Countly is an innovative, real-time, open source mobile & web analytics, rich push notifications and crash reporting platform powering more than 2500 web sites, 16000 mobile applications and several desktop applications as of 2019. It collects data from mobile, desktop, web apps including Apple Watch, TvOS and other internet-connected devices, and visualizes this information to analyze application usage and end-user behavior.



Countly Server - Web and Mobile Analytics Server

Cube.js is an open-source analytical API platform. It is primarily used to build internal business intelligence tools or add customer-facing analytics to existing applications. Cube.js was designed to work with serverless data warehouses and query engines like Google BigQuery and AWS Athena. A multi-stage querying approach makes it suitable for handling trillions of data points. Most modern RDBMS work with Cube.js as well and can be further tuned for performance.

Cube.js — Open-Source Analytical API Platform

OmniSciDB is the foundation of the OmniSci platform. OmniSciDB is SQL-based, relational, columnar and specifically developed to harness the massive parallelism of modern CPU and GPU hardware. OmniSciDB can query up to billions of rows in milliseconds, and is capable of unprecedented ingestion speeds, making it the ideal SQL engine for the era of big, high-velocity data.
 
OmniSciDB optimizes the memory and compute layers to deliver unprecedented performance. OmniSciDB was designed to keep hot data in GPU memory for the fastest access possible. Other GPU database systems have taken the approach of storing the data in CPU memory, only moving it to GPU at query time, trading the gains they receive from GPU parallelism with transfer overheads over the PCIe bus.
 
OmniSciDB natively supports standard SQL and returns query results hundreds of times faster than CPU-only analytical database platforms. Analysts and data scientists can still rely on their existing SQL knowledge, querying data using industry-standard SQL.

OmniSciDB is the foundation of the OmniSci platform. OmniSciDB is SQL-based, relational, columnar and specifically developed to harness the massive parallelism of modern CPU and GPU hardware. OmniSciDB can query up to billions of rows in milliseconds, and is capable of unprecedented ingestion speeds, making it the ideal SQL engine for the era of big, high-velocity data.

OmniSciDB - Open Source Analytical Database & SQL Engine

EventQL is a distributed, column-oriented database built for large-scale event collection and analytics. It runs super-fast SQL and MapReduce queries. Its features include Automatic partitioning, Columnar storage, Standard SQL support, Scales to petabytes, Timeseries and relational data, Fast range scans and lot more.
 
Use case: 
<ul>
 <li>
 Storage and analysis of streaming event, timeseries or relational data
 </li>
 <li>
 High volume event and sensor data logging
 </li>
 <li>
 Joining and correlating of timeseries data with relational tables
 </li>
 </ul>

 EventQL is a distributed, column-oriented database built for large-scale event collection and analytics. It runs super-fast SQL and MapReduce queries. Its features include Automatic partitioning, Columnar storage, Standard SQL support, Scales to petabytes, Timeseries and relational data, Fast range scans and lot more.

EventQL - The database for large-scale event analytics

Apache Iceberg is an open table format for huge analytic datasets. Iceberg adds tables to compute engines including Spark, Trino, PrestoDB, Flink and Hive using a high-performance table format that works just like a SQL table. Users don’t need to know about partitioning to get fast queries. Iceberg was built for huge tables. Iceberg is used in production where a single table can contain tens of petabytes of data and even these huge tables can be read without a distributed SQL engine.

Apache Iceberg - Table format for storing large, slow-moving tabular data

TDengine is an open-source big data platform designed and optimized for Internet of Things (IoT), Connected Vehicles, and Industrial IoT. Besides the 10x faster time-series database, it provides caching, stream computing, message queuing and other functionalities to reduce the complexity and costs of development and operations.
 
<ul><li>10x Faster on Insert/Query Speeds: Through the innovative design on storage, on a single-core machine, over 20K requests can be processed, millions of data points can be ingested, and over 10 million data points can be retrieved in a second. It is 10 times faster than other databases.</li><li>1/5 Hardware/Cloud Service Costs: Compared with typical big data solutions, less than 1/5 of computing resources are required. Via column-based storage and tuned compression algorithms for different data types, less than 1/10 of storage space is needed.</li><li>Full Stack for Time-Series Data: By integrating a database with message queuing, caching, and stream computing features together, it is no longer necessary to integrate Kafka/Redis/HBase/Spark or other software. It makes the system architecture much simpler and more robust.</li><li>Powerful Data Analysis: Whether it is 10 years or one minute ago, data can be queried just by specifying the time range. Data can be aggregated over time, multiple time streams or both. Ad Hoc queries or analyses can be executed via TDengine shell, Python, R or Matlab.</li><li>Seamless Integration with Other Tools: Telegraf, Grafana, Matlab, R, and other tools can be integrated with TDengine without a line of code. MQTT, OPC, Hadoop, Spark, and many others will be integrated soon.</li><li>Zero Management, No Learning Curve: It takes only seconds to download, install, and run it successfully; there are no other dependencies. Automatic partitioning on tables or DBs. Standard SQL is used, with C/C++, Python, JDBC, Go and RESTful connectors.</li></ul><div> </div>

TDengine is an open-source big data platform designed and optimized for Internet of Things (IoT), Connected Vehicles, and Industrial IoT. Besides the 10x faster time-series database, it provides caching, stream computing, message queuing and other functionalities to reduce the complexity and costs of development and operations.

TDengine -  Big data platform designed and optimized for the Internet of Things

SurrealDB is the ultimate mult--modal database for tomorrow's applications. SurrealDB is an end-to-end cloud-native database designed for modern applications, including web, mobile, serverless, Jamstack, backend, and traditional applications. With SurrealDB, you can simplify your database and API infrastructure, reduce development time, and build secure, performant apps quickly and cost-effectively.
<ul dir="auto">
<li>Reduces development time: SurrealDB simplifies your database and API stack by removing the need for most server-side components, allowing you to build secure, performant apps faster and cheaper.</li>
<li>Real-time collaborative API backend service:&nbsp;SurrealDB functions as both a database and an API backend service, enabling real-time collaboration.</li>
<li>Support for multiple querying languages:&nbsp;SurrealDB supports SQL querying from client devices, GraphQL, ACID transactions, WebSocket connections, structured and unstructured data, graph querying, full-text indexing, and geospatial querying.</li>
<li>Granular access control: SurrealDB provides row-level permissions-based access control, giving you the ability to manage data access with precision.</li>
<li class="task-list-item">Record links and directed typed graph connections</li>
<li>Incrementally computed views for pre-computed advanced analytics</li>
<li>Connect and query directly from web-browsers and client devices</li>
<li>Store and model data in any way with tables, documents, and graph</li>
<li>Simple schema definition for frontend and backend development</li>
<li>Use embedded JavaScript functions for custom advanced functionality</li>
</ul>

SurrealDB is an end-to-end cloud-native database designed for modern applications, including web, mobile, serverless, Jamstack, backend, and traditional applications.

SurrealDB - A scalable, distributed, collaborative, document-graph database, for the realtime web

Quickwit is a distributed search engine built from the ground up to offer cost-efficiency and high reliability. By mere mortals for mere mortals, Quickwit's architecture is as simple as possible. Quickwit is written in Rust and built on top of the mighty tantivy library. It designed to index big datasets straight from object storage like AWS S3 in a stateless manner. 
<ul>
<li>Scalable distributed search: Host an arbitrary number of indexes on Amazon S3 and answer search queries in less than a second with a small pool of stateless search instances</li>
<li>Stream indexing: Ingest TB of data from your favorite distributed event streaming service. As of today, Quickwit supports Apache Kafka natively. The next releases will bring support for more platforms</li>
<li>Fault-tolerant architecture that won't lose data: Quickwit achieves exactly-once processing for indexing and safely stores your data on highly reliable object storage services such as Amazon S3</li>
<li>Cloud-native, easy to operate: Thanks to true decoupled compute and storage, search instances are stateless, add or remove search nodes within seconds</li>
<li>Sub-second full-text search on cloud / distributed storage: Quickwit Search re-designed indexing and index data structure to open it in less than 60ms on Amazon S3</li>
<li>Time-based sharding: Quickwit shards data by time when enabled. And you can use a second dimension to shard data thanks to our <a href="../../docs/design/querying">tags feature</a>. Time-based queries only access splits (a data piece of the index) that match the time range of the query which leads to significant performance improvements</li>
<li>Painless multi-tenant search: Create indexes for each tenant without hurting query performance. Or group tenants into one index and use tagging to prune irrelevant splits for your tenant query to improve significantly performance</li>
<li>Elasticsearch query language support</li>
<li>Decoupled compute and storage, stateless indexers &amp; searchers</li>
<li>Schemaless analytics</li>
</ul>

Quickwit is a distributed search engine built from the ground up to offer cost-efficiency and high reliability. By mere mortals for mere mortals, Quickwit's architecture is as simple as possible. Quickwit is written in Rust and built on top of the mighty tantivy library. It designed to index big datasets straight from object storage like AWS S3 in a stateless manner.

Quickwit - Fast and cost-efficient distributed search engine for immutable data

VoltDB was specifically designed for contemporary software applications that are pushed beyond their limits by high volume data sources. VoltDB provides the ability to capture, store and process incoming data at millions of read/write operations per second. And VoltDB’s relational model opens that data to be analyzed in real-time, using familiar Business Intelligence tools, to identify data patterns and trends, spot anomalies, or perform tracking and alerting. In addition to supporting real-time analytics, VoltDB seamlessly integrates with data warehousing and Hadoop infrastructures, feeding data to those environments for deep analytical exploration. Large database gets automatically sharded to multiple nodes, basically it supports database to scale out. VoltDB implements HA using K-Safety, a configurable, active-active durability model for high performance distributed systems. Client connectors are available in languages like Java, C++, Python, PHP, HTTP/JSON, Ruby and Erlang.

VoltDB was specifically designed for contemporary software applications that are pushed beyond their limits by high volume data sources. VoltDB provides the ability to capture, store and process incoming data at millions of read/write operations per second. And VoltDB’s relational model opens that data to be analyzed in real-time, using familiar Business Intelligence tools, to identify data patterns and trends, spot anomalies, or perform tracking and alerting.

VoltDB - Fast Scalable SQL DBMS with ACID

Presto is an open source distributed SQL query engine for running interactive analytic queries against data sources of all sizes ranging from gigabytes to petabytes. It allows querying data from relational / nosql databases. A single Presto query can combine data from multiple sources, allowing for analytics across your entire organization. It is developed by Facebook.

Presto - Distributed SQL query engine for big data

Druid is a distributed, column-oriented, real-time analytics data store that is commonly used to power exploratory dashboards in multi-tenant environments. Druid excels as a data warehousing solution for fast aggregate queries on petabyte sized data sets. Druid supports a variety of flexible filters, exact calculations, approximate algorithms, and other useful calculations. Druid can load both streaming and batch data. 

Druid IO - Real Time Exploratory Analytics on Large Datasets

Timelion, pronounced "Timeline", brings together totally independent data sources into a single interface, driven by a simple, one-line expression language combining data retrieval, time series combination and transformation, plus visualization. Every Timelion expression starts with a data source function.

Timelion -  Time series composer for Elasticsearch and beyond

Cubism.js is a D3 plugin for visualizing time series. Use Cubism to construct better realtime dashboards, pulling data from Graphite, Cube and other sources. Cubism fetches time series data incrementally: after the initial display, Cubism reduces server load by polling only the most recent values. Cubism renders incrementally, too, using Canvas to shift charts one pixel to the left. 

Cubism.js - Time Series Visualization

Storm is a distributed real time computation system. Storm makes it easy to reliably process unbounded streams of data, doing for real time processing what Hadoop did for batch processing. Storm has many use cases: realtime analytics, online machine learning, continuous computation, distributed RPC, ETL, and more. 
Storm integrates with the queueing and database technologies you already use. A Storm topology consumes streams of data and processes those streams in arbitrarily complex ways, repartitioning the streams between each stage of the computation however needed.

Storm is a distributed real time computation system. Storm makes it easy to reliably process unbounded streams of data, doing for real time processing what Hadoop did for batch processing. Storm has many use cases: realtime analytics, online machine learning, continuous computation, distributed RPC, ETL, and more. 

Apache Storm - Distributed and fault-tolerant realtime computation

GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal on *nix systems or through your browser. It provides fast and valuable HTTP statistics for system administrators that require a visual server report on the fly. It supports nearly all web log formats (Apache, Nginx, Amazon S3, Elastic Load Balancing, CloudFront, etc) 

GoAccess - Real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser

Discover open source projects across all platforms

Projects