Item Pipeline — Scrapy 2.13.3 documentation

First steps Scrapy at a glance Installation guide Scrapy Tutorial Examples Basic concepts Command line tool Spiders Selectors Items Item Loaders Scrapy shell Item Pipeline Writing your own item pipeline process_item() open_spider() close_spider() Item pipeline example Price validation and dropping items with no prices Write items to a JSON lines file Write items to MongoDB Take screenshot of item

[kent-where-the-light-is]

“cleansing HTML data validating scraped data (checking that the items contain certain fields) checking for duplicates (and dropping them) storing the scraped item in a database”