Hypertable is based on Google's Bigtable Design, which is a proven scalable design that powers hundreds of Google services. Many of the current scalable NoSQL database offerings are based on a hash table design which means that the data they manage is not kept physically ordered. Hypertable keeps data physically sorted by a primary key and it is well suited for Analytics.
 
This project is for the design and implementation of a high performance, scalable, distributed storage and processing system for structured and unstructured data. It is designed to manage the storage and processing of information on a large cluster of commodity servers, providing resilience to machine and component failures. Data is represented in the system as a multi-dimensional table of information. The data in a table can be transformed and organized at high speed by performing computations in parallel, pushing them to where the data is physically stored.

Hypertable is based on Google's Bigtable Design, which is a proven scalable design that powers hundreds of Google services. Many of the current scalable NoSQL database offerings are based on a hash table design which means that the data they manage is not kept physically ordered. Hypertable keeps data physically sorted by a primary key and it is well suited for Analytics.

Hypertable - A high performance, scalable, distributed storage and processing system for structured 

Solr is the popular, blazing fast open source enterprise search platform from the Apache Lucene project. Its major features include powerful full-text search, hit highlighting, faceted search, dynamic clustering, database integration, and rich document (e.g., Word, PDF) handling. Solr is highly scalable, providing distributed search and index replication, and it powers the search and navigation features of many of the world's largest internet sites. Solr is written in Java and runs as a standalone full-text search server within a servlet container such as Tomcat. Solr uses the Lucene Java search library at its core for full-text indexing and search, and has REST-like HTTP/XML and JSON APIs that make it easy to use from virtually any programming language. Solr's powerful external configuration allows it to be tailored to almost any type of application without Java coding, and it has an extensive plugin architecture when more advanced customization is required. Its feature set include Rich Document Parsing and Indexing (PDF, Word, HTML, etc) using Apache Tika, An Administration Interface, Monitorable Logging, Fast Incremental Updates and Index Replication, Highly Scalable Distributed search with sharded index across multiple hosts, HTTP interface with configurable response formats (XML/XSLT, JSON, Python, Ruby, PHP, Velocity) Solrj is embedded solr. It provides Java based API and it takes care of constructing, parsing, sending and receiving HTTP request.

Solr is the popular, blazing fast open source enterprise search platform from the Apache Lucene project. Its major features include powerful full-text search, hit highlighting, faceted search, dynamic clustering, database integration, and rich document (e.g., Word, PDF) handling. Solr is highly scalable, providing distributed search and index replication, and it powers the search and navigation features of many of the world's largest internet sites. 

Solr - Blazing-fast, open source enterprise search platform

Crate is an open source, highly scalable, shared-nothing distributed SQL database. Crate offers the scalability and performance of a modern No-SQL database with the power of Standard SQL. Crate’s distributed SQL query engine lets you use the same syntax that already exists in your applications or integrations, and have queries seamlessly executed across the crate cluster, including any aggregations, if needed.

Crate - The fast, scalable, easy to use SQL database with native full text search

EventQL is a distributed, column-oriented database built for large-scale event collection and analytics. It runs super-fast SQL and MapReduce queries. Its features include Automatic partitioning, Columnar storage, Standard SQL support, Scales to petabytes, Timeseries and relational data, Fast range scans and lot more.
 
Use case: 
<ul>
 <li>
 Storage and analysis of streaming event, timeseries or relational data
 </li>
 <li>
 High volume event and sensor data logging
 </li>
 <li>
 Joining and correlating of timeseries data with relational tables
 </li>
 </ul>

 EventQL is a distributed, column-oriented database built for large-scale event collection and analytics. It runs super-fast SQL and MapReduce queries. Its features include Automatic partitioning, Columnar storage, Standard SQL support, Scales to petabytes, Timeseries and relational data, Fast range scans and lot more.

EventQL - The database for large-scale event analytics

SurrealDB is the ultimate mult--modal database for tomorrow's applications. SurrealDB is an end-to-end cloud-native database designed for modern applications, including web, mobile, serverless, Jamstack, backend, and traditional applications. With SurrealDB, you can simplify your database and API infrastructure, reduce development time, and build secure, performant apps quickly and cost-effectively.
<ul dir="auto">
<li>Reduces development time: SurrealDB simplifies your database and API stack by removing the need for most server-side components, allowing you to build secure, performant apps faster and cheaper.</li>
<li>Real-time collaborative API backend service:&nbsp;SurrealDB functions as both a database and an API backend service, enabling real-time collaboration.</li>
<li>Support for multiple querying languages:&nbsp;SurrealDB supports SQL querying from client devices, GraphQL, ACID transactions, WebSocket connections, structured and unstructured data, graph querying, full-text indexing, and geospatial querying.</li>
<li>Granular access control: SurrealDB provides row-level permissions-based access control, giving you the ability to manage data access with precision.</li>
<li class="task-list-item">Record links and directed typed graph connections</li>
<li>Incrementally computed views for pre-computed advanced analytics</li>
<li>Connect and query directly from web-browsers and client devices</li>
<li>Store and model data in any way with tables, documents, and graph</li>
<li>Simple schema definition for frontend and backend development</li>
<li>Use embedded JavaScript functions for custom advanced functionality</li>
</ul>

SurrealDB is an end-to-end cloud-native database designed for modern applications, including web, mobile, serverless, Jamstack, backend, and traditional applications.

SurrealDB - A scalable, distributed, collaborative, document-graph database, for the realtime web

VoltDB was specifically designed for contemporary software applications that are pushed beyond their limits by high volume data sources. VoltDB provides the ability to capture, store and process incoming data at millions of read/write operations per second. And VoltDB’s relational model opens that data to be analyzed in real-time, using familiar Business Intelligence tools, to identify data patterns and trends, spot anomalies, or perform tracking and alerting. In addition to supporting real-time analytics, VoltDB seamlessly integrates with data warehousing and Hadoop infrastructures, feeding data to those environments for deep analytical exploration. Large database gets automatically sharded to multiple nodes, basically it supports database to scale out. VoltDB implements HA using K-Safety, a configurable, active-active durability model for high performance distributed systems. Client connectors are available in languages like Java, C++, Python, PHP, HTTP/JSON, Ruby and Erlang.

VoltDB was specifically designed for contemporary software applications that are pushed beyond their limits by high volume data sources. VoltDB provides the ability to capture, store and process incoming data at millions of read/write operations per second. And VoltDB’s relational model opens that data to be analyzed in real-time, using familiar Business Intelligence tools, to identify data patterns and trends, spot anomalies, or perform tracking and alerting.

VoltDB - Fast Scalable SQL DBMS with ACID

Presto is an open source distributed SQL query engine for running interactive analytic queries against data sources of all sizes ranging from gigabytes to petabytes. It allows querying data from relational / nosql databases. A single Presto query can combine data from multiple sources, allowing for analytics across your entire organization. It is developed by Facebook.

Presto - Distributed SQL query engine for big data

Pinot is a realtime distributed OLAP datastore, which is used at LinkedIn to deliver scalable real time analytics with low latency. It can ingest data from offline data sources (such as Hadoop and flat files) as well as online sources (such as Kafka). Pinot is designed to scale horizontally, so that it can scale to larger data sets and higher query rates as needed.
 
Its features include: 
<ul>
<li>A column-oriented database with various compression schemes such as Run Length, Fixed Bit Length</li>
<li>Pluggable indexing technologies - Sorted Index, Bitmap Index, Inverted Index</li>
<li>Ability to optimize query/execution plan based on query and segment metadata .</li>
<li>Near real time ingestion from Kafka and batch ingestion from Hadoop</li>
<li>SQL like language that supports selection, aggregation, filtering, group by, order by, distinct queries on fact data.</li>
<li>Support for multivalued fields</li>
<li>Horizontally scalable and fault tolerant</li>
</ul>

Pinot is a realtime distributed OLAP datastore, which is used at LinkedIn to deliver scalable real time analytics with low latency. It can ingest data from offline data sources (such as Hadoop and flat files) as well as online sources (such as Kafka). Pinot is designed to scale horizontally, so that it can scale to larger data sets and higher query rates as needed.

Pinot - A realtime distributed OLAP datastore

Kudu is a storage system for tables of structured data. Kudu provides a combination of fast inserts/updates and efficient columnar scans to enable multiple real-time analytic workloads across a single storage layer. As a new complement to HDFS and Apache HBase, Kudu gives architects the flexibility to address a wider variety of use cases without exotic workarounds.

Kudu - Hadoop storage layer to enable fast analytics on fast data

ConcourseDB is a distributed self-tuning database with automatic indexing, version control and ACID transactions. ConcourseDB provides a more intuitive approach to data management that is easy to deploy, access and scale while maintaining the strong consistency of traditional database systems.

ConcourseDB  - Self-tuning database designed for both transactions and ad hoc analytics across time

Discover open source projects across all platforms

Projects