inline_requests package¶
Submodules¶
inline_requests.generator module¶
inline_requests.utils module¶
Module contents¶
-
inline_requests.
inline_requests
(method_or_func)[source]¶ A decorator to use coroutine-like spider callbacks.
Example:
class MySpider(Spider): @inline_callbacks def parse(self, response): next_url = response.urjoin('?next') try: next_resp = yield Request(next_url) except Exception as e: self.logger.exception("An error occurred.") return else: yield {"next_url": next_resp.url}
You must conform with the following conventions:
- The decorated method must be a spider method.
- The decorated method must use the
yield
keyword or return a generator. - The decorated method must accept
response
as the first argument. - The decorated method should yield
Request
objects without neithercallback
norerrback
set.
If your requests don’t come back to the generator try setting the flag to handle all http statuses:
request.meta['handle_httpstatus_all'] = True