scrapy - parsing items that are paginatedI have a url of the form: example.com/foo/bar/page_1.html There are a total of 53 pages, each one of them has ~20 rows. I basically want to get all the rows from all the pages, i.e. ~53*20 items. I have working code in my parse method, that parses a single page, and also goes one page deeper per item, to get more info about the item: def parse(self, response): hxs = HtmlXPathSelector(response) res
