Apache Airflow (or simply Airflow) is a platform to programmatically author, schedule, and monitor workflows. When workflows are defined as code, they become more maintainable, versionable, testable, and collaborative.
<BR><BR>
Use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress, and troubleshoot issues when needed.
<BR><BR>
Airflow pipelines are defined in Python, allowing for dynamic pipeline generation. This allows for writing code that instantiates pipelines dynamically.

Apache Airflow (or simply Airflow) is a platform to programmatically author, schedule, and monitor workflows. When workflows are defined as code, they become more maintainable, versionable, testable, and collaborative.

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Rundeck is an open source automation service with a web console, command line tools and a WebAPI. It lets you easily run automation tasks across a set of nodes.  It can schedule a job, more secure replacement of cron. It provides support for runbook automation where it can execute common set of reusable procedures.

Rundeck - Job Scheduler and Runbook Automation

Oozie is a workflow scheduler system to manage Apache Hadoop jobs. Its jobs are Directed Acyclical Graphs (DAGs) of actions and triggered by time (frequency) and data availability. Oozie is integrated with the rest of the Hadoop stack supporting several types of Hadoop jobs out of the box (such as Java map-reduce, Streaming map-reduce, Pig, Hive, Sqoop and Distcp) as well as system specific jobs (such as Java programs and shell scripts).

Apache Oozie - Workflow Scheduler for Hadoop

Discover open source projects across all platforms

Projects

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Rundeck - Job Scheduler and Runbook Automation

Apache Oozie - Workflow Scheduler for Hadoop

TechStack

Tagcloud

Suggested keywords:

Projects

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Rundeck - Job Scheduler and Runbook Automation

Apache Oozie - Workflow Scheduler for Hadoop

TechStack

Tagcloud