Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).

Celery - Distributed Task Queue

Apache RocketMQ is a distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level capacity and flexible scalability.
 
Its features include: 
<ul>
<li>Pub/Sub messaging model</li>
<li>Scheduled message delivery</li>
<li>Message retroactivity by time or offset</li>
<li>Log hub for streaming</li>
<li>Big data integration</li>
<li>Reliable FIFO and strict ordered messaging in the same queue</li>
<li>Efficient pull&amp;push consumption model</li>
<li>Million-level message accumulation capacity in a single queue</li>
<li>Multiple messaging protocols like JMS and OpenMessaging</li>
<li>Flexible distributed scale-out deployment architecture</li>
<li>Lightning-fast batch message exchange system</li>
<li>Various message filter mechanics such as SQL and Tag</li>
<li>Docker images for isolated testing and cloud isolated clusters</li>
<li>Feature-rich administrative dashboard for configuration, metrics and monitoring</li>
</ul>

Apache RocketMQ is a distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level capacity and flexible scalability.

RocketMQ - Distributed messaging and streaming data platform

NSQ is a realtime distributed messaging platform designed to operate at scale, handling billions of messages per day. It promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high availability coupled with a reliable message delivery guarantee. It scales horizontally, without any centralized brokers. Built-in discovery simplifies the addition of nodes to the cluster.
 
<ul>
 <li>support distributed topologies with no SPOF</li>
 <li>horizontally scalable (no brokers, seamlessly add more nodes to the cluster)</li>
 <li>low-latency push based message delivery </li>
 <li>combination load-balanced and multicast style message routing</li>
 <li>excel at both streaming (high-throughput) and job oriented (low-throughput) workloads</li>
 <li>primarily in-memory (beyond a high-water mark messages are transparently kept on disk)</li>
 <li>runtime discovery service for consumers to find producers</li>
 <li>transport layer security (TLS)</li>
 <li>data format agnostic</li>
 <li>few dependencies (easy to deploy) and a sane, bounded, default configuration</li>
 <li>simple TCP protocol supporting client libraries in any language</li>
 <li>HTTP interface for stats, admin actions, and producers (no client library needed to publish)</li>
 <li>integrates with statsd for realtime instrumentation</li>
 <li>robust cluster administration interface</li>
</ul>

NSQ is a realtime distributed messaging platform designed to operate at scale, handling billions of messages per day. It promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high availability coupled with a reliable message delivery guarantee. It scales horizontally, without any centralized brokers. Built-in discovery simplifies the addition of nodes to the cluster.

NSQ - A realtime distributed messaging platform in Go

Pulsar is a distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API. It is horizontally scalable (Millions of independent topics and millions of messages published per second), Strong ordering and consistency guarantees, Low latency , REST API, Geo Replication and lot more.

Pulsar - Distributed pub-sub Messaging System from Yahoo

A simple, fast, robust job/task queue for Node.js, backed by Redis.Bee-Queue is meant to power a distributed worker pool and was built with short, real-time jobs in mind. A web server can enqueue a job, wait for a worker process to complete it, and return its results within an HTTP request. Scaling is as simple as running more workers.

Bee Queue - A simple, fast, robust job/task queue for Node.js, backed by Redis

Jafka mq is a distributed publish-subscribe messaging system cloned from Apache Kafka.
 
Its features include: 
<ul>
<li>Persistent messaging with O(1) disk structures that provide constant time performance even with many TB of stored messages.</li>
<li>High-throughput: even with very modest hardware single broker can support hundreds of thousands of messages per second.</li>
<li>Explicit support for partitioning messages over broker servers and distributing consumption over a cluster of consumer machines while maintaining per-partition ordering semantics.</li>
<li>Simple message format for many language clients.</li>
<li>Pure Java work</li>
</ul>

Jafka mq is a distributed publish-subscribe messaging system cloned from Apache Kafka.

Jafka - A fast and simple distributed publish-subscribe messaging system (mq)

Pulsar is a distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API. It has run in production at Yahoo scale for over 3 years, with millions of messages per second across millions of topics. It is Horizontally scalable, Low latency, High throughput, Multi-tenancy, Geo-replication, Transparent batching of messages, Transparent handling of partitioned topics, REST API for provisioning, admin and stats and lot more. 

Apache Pulsar - Distributed pub-sub messaging platform

The Spread toolkit provides a high performance messaging service that is resilient to faults across local and wide area networks. Spread functions as a unified message bus for distributed applications, and provides highly tuned application-level multicast, group communication, and point to point support. Spread services range from reliable messaging to fully ordered messages with virtual synchrony delivery guarantees.

Spread toolkit - High performance messaging service that is resilient to faults across local and wide area networks

A high-throughput, persistent, distributed, publish-subscribe messaging system based on memory mapped file and Thrift RPC. 
Its design objectives are: 
<ol>
<li>Fast: close to the speed of direct memory access, both produding and consuming are close to O(1) memory access.</li>
<li>Persistent : all data in the system is persisted on disk and is crash resistant.</li>
<li>High-throughput: even with modest hardware Luxun can support handreds of thousands of messages per second.</li>
<li>Realtime : messages produced by producer threads should be immediately visible to consumer threads.</li>
<li>Distributed : explicit support for partitioning messages over Luxun servers and distributing consumption over a cluster of consumer machines while maintaining per-partition ordering semantics.</li>
<li>Multiple clients support : use <a href="http://thrift.apache.org/">Thrift RPC</a> as communication infrastructure, easy to generate different clients for different platforms.</li>
<li>Flexible consuming semantics : supports typical consume once queue, fan out queue, even supports consume by index when needed.</li>
</ol>

A high-throughput, persistent, distributed, publish-subscribe messaging system based on memory mapped file and Thrift RPC.

Luxun - A high-throughput, persistent, distributed, publish-subscribe messaging system based on memo

CassieQ is a distributed queue built on cassandra. CassieQ leverages bucketing of messages to avoid queues modeled off deletes.

cassieq - A distributed queue built off cassandra

Discover open source projects across all platforms

Projects