Alluxio (formerly known as Tachyon) is a virtual distributed storage system. It bridges the gap between computation frameworks and storage systems, enabling computation applications to connect to numerous storage systems through a common interface.
 
Alluxio enables data orchestration for compute in any cloud. It unifies data silos on-premise and across any cloud to give you the data locality, accessibility, and elasticity needed to reduce the complexities associated with orchestrating data for today’s big data and AI/ML workloads.

Alluxio (formerly known as Tachyon) is a virtual distributed storage system. It bridges the gap between computation frameworks and storage systems, enabling computation applications to connect to numerous storage systems through a common interface.

Alluxio - Data orchestration for analytics and machine learning in the cloud

Rook is an open source cloud-native storage orchestrator for Kubernetes, providing the platform, framework, and support for a diverse set of storage solutions to natively integrate with cloud-native environments. Rook turns storage software into self-managing, self-scaling, and self-healing storage services. It does this by automating deployment, bootstrapping, configuration, provisioning, scaling, upgrading, migration, disaster recovery, monitoring, and resource management. Rook uses the facilities provided by the underlying cloud-native container management, scheduling and orchestration platform to perform its duties.

Rook is an open source cloud-native storage orchestrator for Kubernetes, providing the platform, framework, and support for a diverse set of storage solutions to natively integrate with cloud-native environments. 

Rook - Storage Orchestration for Kubernetes

Minio is an object storage server, It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB. Minio server is light enough to be bundled with the application stack, similar to NodeJS, Redis and MySQL.
 
MinIO is different in that it was designed from its inception to be the standard in private cloud object storage. Because MinIO is purpose-built to serve only objects, a single-layer architecture achieves all of the necessary functionality without compromise. The result is a cloud-native object server that is simultaneously performant, scalable and lightweight.
 
While MinIO excels at traditional object storage use cases like secondary storage, disaster recovery and archiving, it is unique at overcoming the private cloud challenges associated with machine learning, analytics and cloud-native application workloads.

Minio is an object storage server, It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB. Minio server is light enough to be bundled with the application stack, similar to NodeJS, Redis and MySQL.

Minio - Open source object storage server compatible with Amazon S3 APIs

Cortex provides horizontally scalable, highly available, multi-tenant, long term storage for Prometheus. Cortex can run across multiple machines in a cluster, exceeding the throughput and storage of a single machine. This enables you to send the metrics from multiple Prometheus servers to a single Cortex cluster and run "globally aggregated" queries across all data in a single place. Cortex makes your PromQL queries blazin' fast through aggressive parallelization and caching.
 
When run in a cluster, Cortex can replicate data between machines. This allows you to survive machine failure without gaps in your graphs. Cortex can isolate data and queries from multiple different independent Prometheus sources in a single cluster, allowing untrusted parties to share the same cluster.
 
Cortex supports Amazon DynamoDB, Google Bigtable, Cassandra, S3, GCS and Microsoft Azure for long term storage of metric data. This allows you to durably store data for longer than the lifetime of any single machine, and use this data for long term capacity planning.

Cortex provides horizontally scalable, highly available, multi-tenant, long term storage for Prometheus. Cortex can run across multiple machines in a cluster, exceeding the throughput and storage of a single machine. This enables you to send the metrics from multiple Prometheus servers to a single Cortex cluster and run "globally aggregated" queries across all data in a single place. Cortex makes your PromQL queries blazin' fast through aggressive parallelization and caching.



Cortex - A multitenant, horizontally scalable Prometheus as a Service

CloudServer (formerly S3 Server) is an open-source Amazon S3-compatible object storage server that is part of Zenko, Scality’s Open Source Multi-Cloud Data Controller.CloudServer provides a single AWS S3 API interface to access multiple backend data storage both on-premise or public in the cloud.
 
CloudServer is useful for Developers, either to run as part of a continous integration test environment to emulate the AWS S3 service locally or as an abstraction layer to develop object storage enabled application on the go.

CloudServer (formerly S3 Server) is an open-source Amazon S3-compatible object storage server that is part of Zenko, Scality’s Open Source Multi-Cloud Data Controller.CloudServer provides a single AWS S3 API interface to access multiple backend data storage both on-premise or public in the cloud. 

S3 - Node.js implementation of a server handling the Amazon S3 protocol

Camlistore is a way of storing, syncing, sharing, modeling and backing up content. It is your personal storage system for life. It can store and be accessed like a traditional filesystem, but it specializes in representing higher-level objects which don't need to exist in purely one place in a tree. Save stuff easily without categorizing it or choosing a location. It is represented using content-addressable blob.

Camlistore - Personal Storage System

OpenEBS is containerized block storage written in Go for cloud native and other environments w/ per container (or pod) QoS SLAs, tiering and replica policies across AZs and environments, and predictable and scalable performance.
 
OpenEBS allows you to treat your persistent workload containers, such as DBs on containers, just like other containers. OpenEBS itself is deployed as just another container on your host, and enables storage services that can be designated on a per pod, application, cluster or container level, including:
<ul>
<li>Data persistence across nodes, dramatically reducing time spent rebuilding Cassandra rings for example</li>
<li>Synchronization of data across availability zones and cloud providers</li>
<li>Use of commodity hardware plus a container engine to deliver seriously scale out block storage</li>
<li>Integration with orchestrators, so that developer and application intent flows into OpenEBS configurations automatically</li>
<li>Management of tiering to and from S3 and other targets</li>
<li>Plus we are bringing our experience from BSD based containerization and delivering QoS for customers from our CloudByte experience over to OpenEBS - expect to see more intelligence and manageability</li>
</ul>

OpenEBS is containerized block storage written in Go for cloud native and other environments w/ per container (or pod) QoS SLAs, tiering and replica policies across AZs and environments, and predictable and scalable performance. 


OpenEBS - Containerized Storage for Containers

With Quilt you can build, push, and install data packages. Data packages are versioned, reusable data structures that can be loaded into Python. Quilt is designed to support reproducible, auditable, and compliant workflows. Quilt consists of three source-level components data catalog, data registry and data compiler.
 
Its features include: 
<ul>
<li>Versioning and storage of large data</li>
<li>Transformation of a variety of file formats into data frames (via pandas and pyarrow)</li>
<li>De-duplication of repeated data for reduced disk and network footprint</li>
</ul>

With Quilt you can build, push, and install data packages. Data packages are versioned, reusable data structures that can be loaded into Python. Quilt is designed to support reproducible, auditable, and compliant workflows. Quilt consists of three source-level components data catalog, data registry and data compiler.

Quilt - Data Engineering Infrastructure

OpenStorage is an API abstraction layer providing support for multiple public APIs, including the OpenStorage SDK, CSI, and the Docker Volume API. Developers using OpenStorage for their storage systems can expect it to work seamlessly with any of the supported public APIs. These implementations provide users with the ability to run stateful services in Linux containers on multiple hosts.
 
Not only does OpenStorage allow storage developers to integrated their storage system with container orchestrations systems, but also enables applications developers to use the OpenStorage SDK to manage and expose the latest storage features to their clients.

OpenStorage is an API abstraction layer providing support for multiple public APIs, including the OpenStorage SDK, CSI, and the Docker Volume API. Developers using OpenStorage for their storage systems can expect it to work seamlessly with any of the supported public APIs. These implementations provide users with the ability to run stateful services in Linux containers on multiple hosts.


OpenStorage - A multi-host clustered implementation of the open storage specification

A self hosted platform to store images. It is an alternative to Google Photos.

ImageStore - Open source google photos alternative! 

Discover open source projects across all platforms

Projects