Scrapy – Uploading image files to Amazon S3

Scrapy has a nice built in feature to automatically download and store images that it comes across while scraping a website. There’s some great documentation on how to get started. Theres also some undocumented code that allows you to store your images on Amazon S3. Here’s how to do that: in your settings.py file: ITEM_PIPELINES […]