IndexTank search engine powers search in Reddit, Social bookmarking site. IndexTank is acquired by LinkedIn and released the project as open source. It includes features like Variables boosts, Facets, Faceted search, Snippeting, Custom scoring functions, Suggest, and Autocomplete. 
 Homepage: <A HREF="http://indextank.com/" target="_blank">http://indextank.com/</A>

IndexTank search engine powers search in Reddit, Social bookmarking site. IndexTank is acquired by LinkedIn and released the project as open source. It includes features like Variables boosts, Facets, Faceted search, Snippeting, Custom scoring functions, Suggest, and Autocomplete.

IndexTank - Search Engine powers Reddit

JanusGraph is a highly scalable graph database optimized for storing and querying large graphs with billions of vertices and edges distributed across a multi-machine cluster. JanusGraph is a transactional database that can support thousands of concurrent users, complex traversals, and analytic graph queries.
<ul>
 <li>Elastic and linear scalability for a growing data and user base.</li>
 <li>Data distribution and replication for performance and fault tolerance.</li>
 <li>Multi-datacenter high availability and hot backups.</li>
</ul>

JanusGraph is a highly scalable graph database optimized for storing and querying large graphs with billions of vertices and edges distributed across a multi-machine cluster. JanusGraph is a transactional database that can support thousands of concurrent users, complex traversals, and analytic graph queries.

JanusGraph - Distributed graph database

HBase provides support to handle BigTable - billions of rows X millions of columns. It is a scalable, distributed, versioned, column-oriented store modeled after Google's Bigtable and runs on top of HDFS (Hadoop Distributed Filesystem). It features compression, in-memory operation per-column. Data could be replicated between the nodes. HBase is used in Facebook and Twitter.

HBase -  Hadoop database

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).

Celery - Distributed Task Queue

As on the ground microservice practitioners quickly realize, the majority of operational problems that arise when moving to a distributed architecture are ultimately grounded in two areas: networking and observability. It is simply an orders of magnitude larger problem to network and debug a set of intertwined distributed services versus a single monolithic application.
 
Originally built at Lyft, Envoy is a high performance C++ distributed proxy designed for single services and applications, as well as a communication bus and “universal data plane” designed for large microservice “service mesh” architectures. Built on the learnings of solutions such as NGINX, HAProxy, hardware load balancers, and cloud load balancers, Envoy runs alongside every application and abstracts the network by providing common features in a platform-agnostic manner. When all service traffic in an infrastructure flows via an Envoy mesh, it becomes easy to visualize problem areas via consistent observability, tune overall performance, and add substrate features in a single place.
 
Envoy is hosted by the Cloud Native Computing Foundation (CNCF). If you are a company that wants to help shape the evolution of technologies that are container-packaged, dynamically-scheduled and microservices-oriented, consider joining the CNCF.

As on the ground microservice practitioners quickly realize, the majority of operational problems that arise when moving to a distributed architecture are ultimately grounded in two areas: networking and observability. It is simply an orders of magnitude larger problem to network and debug a set of intertwined distributed services versus a single monolithic application.

Envoy - C++ front/service proxy

Diaspora is distributed and contextual social networking. It is decentralized and provides freedom and privacy. Instead of everyone’s data being held on huge central servers owned by a large organization, diaspora* exists on independently run servers (“pods”) all over the world. You choose which pod to register with, and you can then connect seamlessly with the diaspora* community worldwide.
 
In diaspora* you own your data. You don’t sign over rights to a corporation or other interest who could use it. In addition, you choose who sees what you share, using Aspects. diaspora* pioneered the concept of aspects, which means you can organise your contacts according to their role in your life. This means you can share something just with family members or work colleagues, knowing that no one you don’t want to will be able to see what you post.

Diaspora is distributed and contextual social networking. It is decentralized and provides freedom and privacy. Instead of everyone’s data being held on huge central servers owned by a large organization, diaspora* exists on independently run servers (“pods”) all over the world. You choose which pod to register with, and you can then connect seamlessly with the diaspora* community worldwide.

Diaspora - Distributed and contextual social networking

ejabberd is a distributed, fault-tolerant technology that allows the creation of large-scale instant messaging applications. The server can reliably support thousands of simultaneous users on a single node and has been designed to provide exceptional standards of fault tolerance. As an open source technology, based on industry-standards, ejabberd can be used to build bespoke solutions very cost effectively.

ejabberd - Robust, Scalable and Extensible XMPP Server

ProxySQL is a high performance, high availability, protocol aware proxy for MySQL and forks (like Percona Server and MariaDB). It has an advanced multi-core architecture. It's built from the ground up to support hundreds of thousands of concurrent connections, multiplexed to potentially hundreds of backend servers. The largest ProxySQL deployment spans several hundred proxies.
 
ProxySQL’s configuration system is inspired from routers. You can dynamically configure everything, persist the configuration and modify it. All with 0 downtime. Its other feature include Query caching, Query Routing, Application layer proxy, Advanced topology support and lot more.
 
It also acts as a Firewall, In case of offending queries that cause problems to the DB (SQL injection or inefficient retrieval of information via SELECT * without WHERE, for example), ProxySQL acts as a gatekeeper between the application and the DB, allowing DBAs to react quickly.

ProxySQL is a high performance, high availability, protocol aware proxy for MySQL and forks (like Percona Server and MariaDB). It has an advanced multi-core architecture. It's built from the ground up to support hundreds of thousands of concurrent connections, multiplexed to potentially hundreds of backend servers. The largest ProxySQL deployment spans several hundred proxies.

ProxySQL - High-performance MySQL proxy

OrbitDB is a serverless, distributed, peer-to-peer database. OrbitDB uses IPFS as its data storage and IPFS Pubsub to automatically sync databases with peers. It's an eventually consistent database that uses CRDTs for conflict-free database merges making OrbitDB an excellent choice for decentralized apps (dApps), blockchain applications and offline-first web applications.
 
OrbitDB provides various types of databases for different data models and use cases:
<ul>
<li><a href="https://github.com/orbitdb/orbit-db/blob/master/API.md#orbitdblognameaddress" target="_blank">log</a>: an immutable (append-only) log with traversable history. Useful for "latest N" use cases or as a message queue.</li>
<li><a href="https://github.com/orbitdb/orbit-db/blob/master/API.md#orbitdbfeednameaddress" target="_blank">feed</a>: a mutable log with traversable history. Entries can be added and removed. Useful for "shopping cart" type of use cases, or for example as a feed of blog posts or "tweets".</li>
<li><a href="https://github.com/orbitdb/orbit-db/blob/master/API.md#orbitdbkeyvaluenameaddress" target="_blank">keyvalue</a>: a key-value database just like your favourite key-value database.</li>
<li><a href="https://github.com/orbitdb/orbit-db/blob/master/API.md#orbitdbdocsnameaddress-options" target="_blank">docs</a>: a document database to which JSON documents can be stored and indexed by a specified key. Useful for building search indices or version controlling documents and data.</li>
<li><a href="https://github.com/orbitdb/orbit-db/blob/master/API.md#orbitdbcounternameaddress" target="_blank">counter</a>: Useful for counting events separate from log/feed data.</li>
</ul>

OrbitDB is a serverless, distributed, peer-to-peer database. OrbitDB uses IPFS as its data storage and IPFS Pubsub to automatically sync databases with peers. It's an eventually consistent database that uses CRDTs for conflict-free database merges making OrbitDB an excellent choice for decentralized apps (dApps), blockchain applications and offline-first web applications.

orbit-db - Peer-to-Peer Databases for the Decentralized Web

Micro is an operating system built for the cloud. It addresses the key requirements for building services in the cloud. It leverages the microservices architecture pattern and provides a set of services which act as the building blocks of a platform. Micro deals with the complexity of distributed systems and provides simpler programmable abstractions to build on.
 
Micro is built as a microservices architecture and abstracts away the complexity of the underlying infrastructure. It is composed as a single logical server to the user but decompose that into the various building block primitives that can be plugged into any underlying system.
The server is composed of the following services.<ul>
 <li>API - HTTP Gateway which dynamically maps http/json requests to RPC using path based resolution</li>
 <li>Auth - Authentication and authorization out of the box using jwt tokens and rule based access control.</li>
 <li>Broker - Ephemeral pubsub messaging for asynchronous communication and distributing notifications</li>
 <li>Config - Dynamic configuration and secrets management for service level config without the need to restart</li>
 <li>Events - Event streaming with ordered messaging, replay from offsets and persistent storage</li>
 <li>Network - Inter-service networking, isolation and routing plane for all internal request traffic</li>
 <li>Proxy - An identity aware proxy used for remote access and any external grpc request traffic</li>
 <li>Runtime - Service lifecycle and process management with support for source to running auto build</li>
 <li>Registry - Centralised service discovery and API endpoint explorer with feature rich metadata</li>
 <li>Store - Key-Value storage with TTL expiry and persistent crud to keep microservices stateless</li></ul>

Micro is an operating system built for the cloud. It  addresses the key requirements for building services in the cloud. It leverages the microservices architecture pattern and provides a set of services which act as the building blocks of a platform. Micro deals with the complexity of distributed systems and provides simpler programmable abstractions to build on.

Micro - Distributed Cloud OS 

JobRunr is the ultimate library to perform background processing on the JVM. It performs fire-and-forget, delayed, scheduled and recurring jobs inside Java applications using only Java 8 lambda's. CPU and I/O intensive, long-running and short-running jobs are supported.
It can be used in various use cases like batch processing, image processing, database maintenance, daily maintenance like taking backups, sending out notifications etc. 
Its feature include:
<ul>
<li>Simple: just use Java 8 lambda's to create a background job.</li>
<li>Distributed &amp; cluster-friendly: guarantees execution by single scheduler instance using optimistic locking.</li>
<li>Persistent jobs: using either a RDMBS (four tables and a view) or a noSQL data store.</li>
<li>Embeddable: built to be embedded in existing applications.</li>
</ul>

JobRunr is the ultimate library for background processing in Java.

JobRunr - The ultimate library for background processing in Java.

kumofs is a scalable and highly available distributed key-value store. 
Data is replicated over multiple servers. 
Data is partitioned over multiple servers. 
Extreme single node performance; comparable with memcached. 
Both read and write performance got improved as servers added. 
Servers can be added without stopping the system. 
Servers can be added without changing the client applications. 
The system does not stop even if one or two servers crashed. 
The system does not stop to recover crashed servers. 
Scalable from 2 to 60 servers. (more than 60 servers has not be tested yet) 
Optimized for storing a large amount of small data. 
memcached protocol support. (get, set and delete only; expiration time and flags must be 0)

kumofs is a scalable and highly available distributed key-value store.

Kumofs

Mercurial is a free, distributed source control management tool. It offers you the power to efficiently handle projects of any size while using an intuitive interface. It is easy to use and hard to break, making it ideal for anyone working with versioned files. Traditional version control systems such as Subversion are typical client-server architectures with a central server to store the revisions of a project. In contrast, Mercurial is truly distributed, giving each developer a local copy of the entire development history. This way it works independent of network access or a central server. Committing, branching and merging are fast and cheap. Mercurials implementation and data structures are designed to be fast. You can generate diffs between revisions, or jump back in time within seconds. Therefore Mercurial is perfectly suitable for large projects such as OpenJDK (hg) or NetBeans (hg).

Mercurial is fast and powerful. Mercurial offers you the power and speed to efficiently handle projects of any size and kind. Every clone contains the whole project history, so committing, branching, tagging and merging are local operations which makes them fast and convenient. You can use a multitude of workflows and easily enhance its functionality with extensions.

Mercurial

Redisson - Distributed and Scalable Java data structures (Set, SortedSet, Map, ConcurrentMap, List, Queue, Deque, Lock, AtomicLong, CountDownLatch, Publish / Subscribe, HyperLogLog) on top of Redis server. Advanced redis java client. It supports over 28+ data structures and services, Synchronous / asynchronous / reactive interfaces and lot more.

Redisson - Redis based In-Memory Data Grid for Java

Vector is an open source on-host performance monitoring framework which exposes hand picked high resolution system and application metrics to every engineer’s browser. Having the right metrics available on-demand and at a high resolution is key to understand how a system behaves and correctly troubleshoot performance issues.

Vector - Performance Monitoring Framework

Discover open source projects across all platforms

Projects