Methanol is a modular, customizable Web crawling system with crawlers optimized for speed. It is designed to allow the administrator to set up any kind of filetype handling, parsing, and indexing rules.
Open source project that offer summary and download services on this page is a project that carry out their development work on other open source development sites. Their work is linked to this page via a feature called OFI, and their work is not carried out here on OSDN site.