Apache Airflow (or simply Airflow) is a platform to programmatically author, schedule, and monitor workflows. When workflows are defined as code, they become more maintainable, versionable, testable, and collaborative.
 
Use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress, and troubleshoot issues when needed.
 
Airflow pipelines are defined in Python, allowing for dynamic pipeline generation. This allows for writing code that instantiates pipelines dynamically.

Apache Airflow (or simply Airflow) is a platform to programmatically author, schedule, and monitor workflows. When workflows are defined as code, they become more maintainable, versionable, testable, and collaborative.

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Schedulix is the Open Source Enterprise Job Scheduling System, which meets the complex requirements of modern IT process automation. It helps to create Complex workflow,  Hierarchical workflow modelling, Workflows can be dynamically submitted or paralleled, Automatic reruns of sub-workflow, Load balancing, Sticky allocations,  Time scheduling and lot more.

Schedulix - Enterprise Job Scheduling System

Rundeck is an open source automation service with a web console, command line tools and a WebAPI. It lets you easily run automation tasks across a set of nodes.  It can schedule a job, more secure replacement of cron. It provides support for runbook automation where it can execute common set of reusable procedures.

Rundeck - Job Scheduler and Runbook Automation

Oozie is a workflow scheduler system to manage Apache Hadoop jobs. Its jobs are Directed Acyclical Graphs (DAGs) of actions and triggered by time (frequency) and data availability. Oozie is integrated with the rest of the Hadoop stack supporting several types of Hadoop jobs out of the box (such as Java map-reduce, Streaming map-reduce, Pig, Hive, Sqoop and Distcp) as well as system specific jobs (such as Java programs and shell scripts).

Apache Oozie - Workflow Scheduler for Hadoop

Cascading is a Data Processing API, Process Planner, and Process Scheduler used for defining and executing complex, scale-free, and fault tolerant data processing workflows on an Apache Hadoop cluster. It is a thin Java library and API that sits on top of Hadoop's MapReduce layer and is executed from the command line like any other Hadoop application.

Cascading - Data Processing Workflows on Hadoop

evQueue is an open source job scheduler and queueing engine. It features an event-driven C++ engine and a PHP / MySQL web control interface which provides tasks monitoring and creation. It provides both simple task execution and complex task chaining (workflow) using an easy to use drag & drop web interface. Workflow description includes output linking to input, conditions, loops... Queues management provides an easy way for task parallelization and resource control.
 
The network API provides an easy way, XML based, for synchronous or asynchronous workflow launching and control, allowing tasks to be launched from external applications or web pages. You can thus execute heavy tasks on a remote (dedicated) machine to save resources on your web frontends while providing live reporing of the treatment to you users.

evQueue is an open source job scheduler and queueing engine. It features an event-driven C++ engine and a PHP / MySQL web control interface which provides tasks monitoring and creation. It provides both simple task execution and complex task chaining (workflow) using an easy to use drag & drop web interface. Workflow description includes output linking to input, conditions, loops... Queues management provides an easy way for task parallelization and resource control.

evQueue - Job scheduler and queueing engine

Discover open source projects across all platforms

Projects

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Schedulix - Enterprise Job Scheduling System

Rundeck - Job Scheduler and Runbook Automation

Apache Oozie - Workflow Scheduler for Hadoop

Cascading - Data Processing Workflows on Hadoop

evQueue - Job scheduler and queueing engine

TechStack

Tagcloud

License

Suggested keywords:

Projects

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Schedulix - Enterprise Job Scheduling System

Rundeck - Job Scheduler and Runbook Automation

Apache Oozie - Workflow Scheduler for Hadoop

Cascading - Data Processing Workflows on Hadoop

evQueue - Job scheduler and queueing engine

TechStack

Tagcloud

License