YaCy (read "ya see") is a free distributed search engine, built on principles of peer-to-peer (P2P) networks. It is distributed on several hundred computers so-called YaCy-peers. Each YaCy-peer independently crawls through the Internet, analyzes and indexes found web pages, and stores indexing results in a common database which is shared with other YaCy-peers using principles of P2P networks.

YaCy - Decentralized Web Search

Open Distro for Elasticsearch is an Apache 2.0-licensed distribution of Elasticsearch enhanced with Enterprise Security, Alerting, SQL, Index Management, k-Nearest Neighbor Search, Performance Analyzer and more. 
 
It provides set of features to help you keep your data secure and stay compliant with regulations such as GDPR, HIPAA, PCI, and ISO. Whether you want to encrypt data in-transit, authenticate users against Active Directory, use Kerberos or JSON web tokens for single sign-on (SSO), or monitor and log any malicious access attempts, Open Distro for Elasticsearch has you covered.
 
It provides support to extract insights out of Elasticsearch using the familiar SQL query syntax. Use aggregations, group by, and where clauses to investigate your data. Read your data as JSON documents or CSV tables so you have the flexibility to use the format that works best for you

Open Distro for Elasticsearch is an Apache 2.0-licensed distribution of Elasticsearch enhanced with Enterprise Security, Alerting, SQL, Index Management, k-Nearest Neighbor Search, Performance Analyzer and more. 


Open Distro for Elasticsearch  - Elasticsearch enhanced with enterprise security, alerting, SQL, and more

Sensei is a distributed data system that was built to support many product initiatives at LinkedIn, including the real-time faceted search in <A HREF="http://www.linkedin.com/signal/" target="_blank">LinkedIn Signal</A> and the news feed and tabs on the Homepage. Sensei is both a search engine and a database. It is designed to query and navigate through documents that consist of unstructured text and well-formed and structured metadata. 
 It supports Fast realtime updates, Complex query language and REST/JSON api, Hadoop integration and lot more.

Sensei is a distributed data system that was built to support many product initiatives at LinkedIn, including the real-time faceted search in LinkedIn Signal and the news feed and tabs on the Homepage. Sensei is both a search engine and a database. It is designed to query and navigate through documents that consist of unstructured text and  well-formed and structured metadata. Sensei is both a search engine and a database.

SenseiDB - Distributed, Realtime, Semi-Structured Database from LinkedIn

Lux is an open source XML search engine using Lucene /Solr and Saxon XQuery/XSLT processor. Lux provides XML-aware indexing, an XQuery 1.0 optimizer that rewrites queries to use the indexes, and a function library for interacting with Lucene via XQuery. These capabilities are tightly integrated with Solr, and leverage its application framework in order to deliver a REST service, application server, and supporting tools. 
 
Lux provides several kinds of index: XML full-text indexes, an XML path index, and configurable XPath indexes.

Lux is an open source XML search engine using Lucene /Solr and Saxon XQuery/XSLT processor. Lux provides XML-aware indexing, an XQuery 1.0 optimizer that rewrites queries to use the indexes, and a function library for interacting with Lucene via XQuery. These capabilities are tightly integrated with Solr, and leverage its application framework in order to deliver a REST service, application server, and supporting tools.

Lux - XML Search engine

JVector is a pure Java, zero dependency, embedded vector search engine, used by DataStax Astra DB.
JVector uses state of the art graph algorithms inspired by DiskANN and related research that offer high recall and low latency. It uses the Panama SIMD API to accelerate index build and queries. It is memory efficient, compresses vectors using product quantization so they can stay in memory during searches.
JVector is concurrent, its index can scale linearly to 32 threads.

JVector - The most advanced embedded vector search engine

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix is designed to respect the robots.txt exclusion directives and META robots tags, and collect material at a measured, adaptive pace unlikely to disrupt normal website activity. It provides web interface for operator control and monitoring of crawls. It stores content to ARC or ISO WARC aggregate/transcript format.

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix is designed to respect the robots.txt exclusion directives and META robots tags, and collect material at a measured, adaptive pace unlikely to disrupt normal website activity.

Heritrix

Bobo Browse is an information retrieval technology that provides navigational browsing into a semi-structured dataset. Beyond the result set from queries and selections, Bobo Browse also provides the facets from this point of browsing. It provides support to sort documents on fields that have multiple values. It is stable and used by LinkedIn.

Bobo - Faceted search library based on Lucene

Norconex HTTP Collector is a web spider, or crawler that aims to make Enterprise Search integrators and developers's life easier. It is Portable, Extensible, reusable, Robots.txt support, Obtain and manipulate document metadata, Resumable upon failure and lot more.

Norconex HTTP Collector - A Web Crawler in Java

LuMongo is a real-time distributed search and storage system based on Lucene. LuMongo is designed from the ground up to scale both vertically and horizontally across servers. LuMongo stores Lucene indexes directly into MongoDB. Documents can be stored natively into MongoDB. When stored natively document can be queried as normal out of MongoDB and use of Map-Reduce and the Aggregation Framework is possible.

LuMongo - Realtime Time Distributed Search

Para was designed to be a simple and modular backend framework for object persistence and retrieval. It helps you build applications faster by taking care of the backend. It works on three levels - objects are stored in a NoSQL data store or any old relational database, then automatically indexed by a search engine and finally, cached. Para is also multitenant and has a clean and easy-to-use JSON API.

Para - The backend for busy developers

Katta is a scalable, failure tolerant, distributed, data storage for real time access. Katta serves large, replicated, indices as shards to serve high loads and very large data sets. These indices can be of different type. Currently implementations are available for Lucene and Hadoop mapfiles. Its features include:
 <ul>
	<li>Makes serving large or high load indices easy</li>
	<li>Serves very large Lucene or Hadoop Mapfile indices as index shards on many servers</li>
	<li>Replicate shards on different servers for performance and fault-tolerance</li>
	<li>Supports pluggable network topologies</li>
	<li>Master fail-over</li>
	<li>Fast, lightweight, easy to integrate</li>
	<li>Plays well with Hadoop clusters</li>
</ul>

Katta is a scalable, failure tolerant, distributed, data storage for real time access. Katta serves large, replicated, indices as shards to serve high loads and very large data sets. These indices can be of different type. Currently implementations are available for Lucene and Hadoop mapfiles.

Katta - Lucene and more in the cloud.

Compass provides real time searchengine built on top of Lucene. It is distributed, transcational, supports Spring MVC, integrates with ORM. Simple Compass provides a simple API for working with Lucene. If you know how to use an ORM, then you will feel right at home with Compass with simple operations for save, and delete & query. Compass provides google-style search, index updates as well as more advanced concepts such as caching and index sharding (sub indexes). Compass also uses built in optimizations for concurrent commits and merges. It provides support to map	Object, JSON, XML to search engine mapping. It allows storing the index with Data Grid products such as GigaSpaces, Coherence and Terracotta.

Compass is a real time searchengine. It is built on top of lucene. It is transactional, distributed, supports Spring MVC, integrates with Hibernate.    

compass - Searchengine built on top of Lucene

searchcode server is a powerful code search engine with a sleek web user interface. searchcode server works in tandem with your source control system, indexing thousands of repositories and files allowing you and your developers to quickly find and reuse code across teams.

searchcode server - The offical home of searchcode-server where you can run searchcode locally

Discover open source projects across all platforms

Projects