Scrapy

Scrapy is "an open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way". Scrapy is a site crawling application that is structured to retrieve structured and useful data from websites for the purposes of data mining, information processing or historical archiving. Scrapy facilitates the data extraction from nearly any website by allowing user to write their own Spiders - directions for locating and retrieving website data. Scrapy is fast, powerful, easily extensible, and yet entirely simple. Scrapy is a free, multi-platform program that is compatible with Windows, Mac, and Linux.

Category: