Because of #19 , the type hint for Spider.crawl_result broke, and it was temporarily replaced with Dict[str, Dict[str, Any]].
This should be fixed to actually reflect the contents of crawl_result, which has the following format:
crawl_result = {
"url1":{
"urls":["some url", "some other url", ...],
"body": "the html of the page"
},
"url2":{
"urls":["some url", "some other url", ...],
"body": "the html of the page"
},
}
Where body is only present if the include_body argument is set to True, and as such might not always be present.
See #19 for previous discussions about this.
You can verify the type hint is working if the mypy checks pass.
Because of #19 , the type hint for
Spider.crawl_resultbroke, and it was temporarily replaced withDict[str, Dict[str, Any]].This should be fixed to actually reflect the contents of
crawl_result, which has the following format:Where
bodyis only present if theinclude_bodyargument is set toTrue, and as such might not always be present.See #19 for previous discussions about this.
You can verify the type hint is working if the mypy checks pass.