Sensei is a distributed data system that was built to support many product initiatives at LinkedIn, including the real-time faceted search in <A HREF="http://www.linkedin.com/signal/" target="_blank">LinkedIn Signal</A> and the news feed and tabs on the Homepage. Sensei is both a search engine and a database. It is designed to query and navigate through documents that consist of unstructured text and well-formed and structured metadata. 
 It supports Fast realtime updates, Complex query language and REST/JSON api, Hadoop integration and lot more.

Sensei is a distributed data system that was built to support many product initiatives at LinkedIn, including the real-time faceted search in LinkedIn Signal and the news feed and tabs on the Homepage. Sensei is both a search engine and a database. It is designed to query and navigate through documents that consist of unstructured text and  well-formed and structured metadata. Sensei is both a search engine and a database.

SenseiDB - Distributed, Realtime, Semi-Structured Database from LinkedIn

Lux is an open source XML search engine using Lucene /Solr and Saxon XQuery/XSLT processor. Lux provides XML-aware indexing, an XQuery 1.0 optimizer that rewrites queries to use the indexes, and a function library for interacting with Lucene via XQuery. These capabilities are tightly integrated with Solr, and leverage its application framework in order to deliver a REST service, application server, and supporting tools. 
 
Lux provides several kinds of index: XML full-text indexes, an XML path index, and configurable XPath indexes.

Lux is an open source XML search engine using Lucene /Solr and Saxon XQuery/XSLT processor. Lux provides XML-aware indexing, an XQuery 1.0 optimizer that rewrites queries to use the indexes, and a function library for interacting with Lucene via XQuery. These capabilities are tightly integrated with Solr, and leverage its application framework in order to deliver a REST service, application server, and supporting tools.

Lux - XML Search engine

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix is designed to respect the robots.txt exclusion directives and META robots tags, and collect material at a measured, adaptive pace unlikely to disrupt normal website activity. It provides web interface for operator control and monitoring of crawls. It stores content to ARC or ISO WARC aggregate/transcript format.

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix is designed to respect the robots.txt exclusion directives and META robots tags, and collect material at a measured, adaptive pace unlikely to disrupt normal website activity.

Heritrix

Bobo Browse is an information retrieval technology that provides navigational browsing into a semi-structured dataset. Beyond the result set from queries and selections, Bobo Browse also provides the facets from this point of browsing. It provides support to sort documents on fields that have multiple values. It is stable and used by LinkedIn.

Bobo - Faceted search library based on Lucene

LuMongo is a real-time distributed search and storage system based on Lucene. LuMongo is designed from the ground up to scale both vertically and horizontally across servers. LuMongo stores Lucene indexes directly into MongoDB. Documents can be stored natively into MongoDB. When stored natively document can be queried as normal out of MongoDB and use of Map-Reduce and the Aggregation Framework is possible.

LuMongo - Realtime Time Distributed Search

Katta is a scalable, failure tolerant, distributed, data storage for real time access. Katta serves large, replicated, indices as shards to serve high loads and very large data sets. These indices can be of different type. Currently implementations are available for Lucene and Hadoop mapfiles. Its features include:
 <ul>
	<li>Makes serving large or high load indices easy</li>
	<li>Serves very large Lucene or Hadoop Mapfile indices as index shards on many servers</li>
	<li>Replicate shards on different servers for performance and fault-tolerance</li>
	<li>Supports pluggable network topologies</li>
	<li>Master fail-over</li>
	<li>Fast, lightweight, easy to integrate</li>
	<li>Plays well with Hadoop clusters</li>
</ul>

Katta is a scalable, failure tolerant, distributed, data storage for real time access. Katta serves large, replicated, indices as shards to serve high loads and very large data sets. These indices can be of different type. Currently implementations are available for Lucene and Hadoop mapfiles.

Katta - Lucene and more in the cloud.

Compass provides real time searchengine built on top of Lucene. It is distributed, transcational, supports Spring MVC, integrates with ORM. Simple Compass provides a simple API for working with Lucene. If you know how to use an ORM, then you will feel right at home with Compass with simple operations for save, and delete & query. Compass provides google-style search, index updates as well as more advanced concepts such as caching and index sharding (sub indexes). Compass also uses built in optimizations for concurrent commits and merges. It provides support to map	Object, JSON, XML to search engine mapping. It allows storing the index with Data Grid products such as GigaSpaces, Coherence and Terracotta.

Compass is a real time searchengine. It is built on top of lucene. It is transactional, distributed, supports Spring MVC, integrates with Hibernate.    

compass - Searchengine built on top of Lucene

Discover open source projects across all platforms

Projects

SenseiDB - Distributed, Realtime, Semi-Structured Database from LinkedIn

Lux - XML Search engine

Heritrix

Bobo - Faceted search library based on Lucene

LuMongo - Realtime Time Distributed Search

Katta - Lucene and more in the cloud.

compass - Searchengine built on top of Lucene

Tagcloud

License

Suggested keywords:

Projects

SenseiDB - Distributed, Realtime, Semi-Structured Database from LinkedIn

Lux - XML Search engine

Heritrix

Bobo - Faceted search library based on Lucene

LuMongo - Realtime Time Distributed Search

Katta - Lucene and more in the cloud.

compass - Searchengine built on top of Lucene

Tagcloud

License