Voldemort is a distributed key-value storage system. Data is automatically replicated over multiple servers. Data is automatically partitioned so each server contains only a subset of the total data. 
Server failure is handled transparently. It is used at LinkedIn for certain high-scalability storage problems where simple functional partitioning is not sufficient.

Project-voldemort - A distributed database, Clone of Amazon's Dynamo

The Apache Cassandra Project develops a highly scalable second-generation distributed database, bringing together Dynamo's fully distributed design and Bigtable's ColumnFamily-based data model. Cassandra is suitable for applications that can't afford to lose data. Data is automatically replicated to multiple nodes for fault-tolerance. 

 Cassandra provides support for Scale out, load balancing, cluster growth, Flexible schema, Key-oriented queries and CAP theorem (Consistency, Availability, Partition tolerance). It is in use at Digg, Facebook, Twitter, Reddit, Rackspace, Cloudkick, Cisco and more companies.

The Apache Cassandra Project develops a highly scalable second-generation distributed database, bringing together Dynamo's fully distributed design and Bigtable's ColumnFamily-based data model. Cassandra is suitable for applications that can't afford to lose data. Data is automatically replicated to multiple nodes for fault-tolerance.

Cassandra - Scalable Distributed Database

HBase provides support to handle BigTable - billions of rows X millions of columns. It is a scalable, distributed, versioned, column-oriented store modeled after Google's Bigtable and runs on top of HDFS (Hadoop Distributed Filesystem). It features compression, in-memory operation per-column. Data could be replicated between the nodes. HBase is used in Facebook and Twitter.

HBase -  Hadoop database

kumofs is a scalable and highly available distributed key-value store. 
Data is replicated over multiple servers. 
Data is partitioned over multiple servers. 
Extreme single node performance; comparable with memcached. 
Both read and write performance got improved as servers added. 
Servers can be added without stopping the system. 
Servers can be added without changing the client applications. 
The system does not stop even if one or two servers crashed. 
The system does not stop to recover crashed servers. 
Scalable from 2 to 60 servers. (more than 60 servers has not be tested yet) 
Optimized for storing a large amount of small data. 
memcached protocol support. (get, set and delete only; expiration time and flags must be 0)

kumofs is a scalable and highly available distributed key-value store.

Kumofs

Crate is an open source, highly scalable, shared-nothing distributed SQL database. Crate offers the scalability and performance of a modern No-SQL database with the power of Standard SQL. Crate’s distributed SQL query engine lets you use the same syntax that already exists in your applications or integrations, and have queries seamlessly executed across the crate cluster, including any aggregations, if needed.

Crate - The fast, scalable, easy to use SQL database with native full text search

Titan is a scalable graph database optimized for storing and querying graphs containing hundreds of billions of vertices and edges distributed across a multi-machine cluster. Titan is a transactional database that can support thousands of concurrent users executing complex graph traversals. It is a native Blueprints enabled graph database and as such, it supports the full TinkerPop stack of technologies.

Titan - Scalable Graph Database 

FlockDB is much simpler than other graph databases such as neo4j because it tries to solve fewer problems. It scales horizontally and is designed for on-line, low-latency, high throughput environments such as web-sites. Twitter uses FlockDB to store social graphs (who follows whom, who blocks whom) and secondary indices. As of April 2010, the Twitter FlockDB cluster stores 13+ billion edges and sustains peak traffic of 20k writes/second and 100k reads/second.

FlockDB - A distributed, fault-tolerant graph database from Twitter

Cloudata is Distributed Large scale Structured Data Storage, and open source project implementing Google's Bigtable. It's DBMS(Database Management System), but not Relational DBMS. It can store more than Peta bytes.

Cloudata - Structured Data Storage implementing Google's Bigtable.

S4 is a general-purpose, distributed, scalable, partially fault-tolerant, pluggable platform that allows programmers to easily develop applications for processing continuous unbounded streams of data. S4 has been deployed in production systems at Yahoo to process thousands of search queries per second.

s4 - Distributed Stream Computing Platform

Discover open source projects across all platforms

Projects