An open source .NET web crawler written in C# using SQL 2005/2008. Arachnode.net is a complete and comprehensive .NET web crawler for downloading, indexing and storing Internet content including e-mail addresses, files, hyperlinks, images, and Web pages. Its features include
  <UL>
	<LI>.NET architecture</LI>
	<LI>Configurable Rules and Actions</LI>
	<LI>Lucene.NET Integration</LI>
	<LI>SQL Server 2008 and full-text indexing</LI>
	<LI>.DOC/.PDF/.PPT/.XLS Indexing</LI>
	<LI>HTML to XML and XHTML</LI>
	<LI>Multi-threading and Throttling</LI>
	<LI>Respectful Crawling</LI>
	<LI>Analysis Services</LI>
	<LI>SQL Server 2008 and SSIS</LI>
	<LI>EXIF data extraction</LI>
 </UL>

An open source .NET web crawler written in C# using SQL 2005/2008. Arachnode.net is a complete and comprehensive .NET web crawler for downloading, indexing and storing Internet content including e-mail addresses, files, hyperlinks, images, and Web pages.

Arachnode.net

Discover open source projects across all platforms

Projects

Arachnode.net

TechStack

Tagcloud

Suggested keywords:

Projects

Arachnode.net

TechStack

Tagcloud