Apache Airflow (or simply Airflow) is a platform to programmatically author, schedule, and monitor workflows. When workflows are defined as code, they become more maintainable, versionable, testable, and collaborative.
 
Use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress, and troubleshoot issues when needed.
 
Airflow pipelines are defined in Python, allowing for dynamic pipeline generation. This allows for writing code that instantiates pipelines dynamically.

Apache Airflow (or simply Airflow) is a platform to programmatically author, schedule, and monitor workflows. When workflows are defined as code, they become more maintainable, versionable, testable, and collaborative.

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

The purpose of Luigi is to address all the plumbing typically associated with long-running batch processes. You want to chain many tasks, automate them, and failures will happen. These tasks can be anything, but are typically long running things like Hadoop jobs, dumping data to/from databases, running machine learning algorithms, or anything else.
 
There are other software packages that focus on lower level aspects of data processing, like Hive, Pig, or Cascading. Luigi is not a framework to replace these. Instead it helps you stitch many tasks together, where each task can be a Hive query, a Hadoop job in Java, a Spark job in Scala or Python, a Python snippet, dumping a table from a database, or anything else. It's easy to build up long-running pipelines that comprise thousands of tasks and take days or weeks to complete. Luigi takes care of a lot of the workflow management so that you can focus on the tasks themselves and their dependencies.
 
Luigi internally used at <a href="https://www.spotify.com">Spotify</a> to run
thousands of tasks every day, organized in complex dependency graphs.
Most of these tasks are Hadoop jobs. Luigi provides an infrastructure
that powers all kinds of stuff including recommendations, toplists, A/B
test analysis, external reports, internal dashboards, etc

The purpose of Luigi is to address all the plumbing typically associated with long-running batch processes. You want to chain many tasks, automate them, and failures will happen. These tasks can be anything, but are typically long running things like Hadoop jobs, dumping data to/from databases, running machine learning algorithms, or anything else.

Luigi - Python module that helps you build complex pipelines of batch jobs

OpenERP is a comprehensive suite of business applications including Sales, CRM, Project management, Warehouse management, Manufacturing, Financial management, Human Resources, Point of Sale, Accounting, Purchase, Marketing and lot more. It has over 700 OpenERP modules available on Launchpad. The package contains OpenERP server, web client and GTK client. 

 OpenERP CRM helps to track sales team best leads and opportunities, Run Marketing Campaigns, Customize your sales cycle, Analyse the stats and forcast the sales. 
 Its Accounting and Finance module provides support to manage payments, Control the invoice, Generate analytic accounting operations integrated with timesheets, projects, invoices, warehouse, etc. It supports mult-country, multi-company and multi-currency. 
 It also includes an ORM, a Workflow engine, report designers, a dashboard designer, a module system, an automated migration engine, and much more. 
 Source code: <A HREF="https://launchpad.net/openobject" target="_blank">https://launchpad.net/openobject</A>
 <img src="/AppImages/Article/openerp-img1.jpg" alt="" class="float-center">

OpenERP is a comprehensive suite of business applications including Sales, CRM, Project management, Warehouse management, Manufacturing, Financial management, Human Resources, Point of Sale, Accounting, Purchase, Marketing and lot more. It has over 700 OpenERP modules available on Launchpad. The package contains OpenERP server, web client and GTK client.

OpenERP -  Open Source Business Applications

Pyres is a Resque clone. Resque is a great implementation of a job queue by the people at github. 

Pyres - Job queue in Python

Obsei is a low code AI powered automation tool. It can be used in various business flows like social listening, AI based alerting, brand image analysis, comparative study and more. 
It consist of Observer, Analyzer and Informer. Observer observes the platform like Twitter, Facebook, App Stores, Google reviews, Amazon reviews, News, Website etc and feed that information. Analyzer performs text analysis like classification, sentiment, translation, PII on the analyzed data. Informer sends it to ticketing system, data store, dataframe etc for further action and analysis.

Obsei - Low code AI powered automation tool

Discover open source projects across all platforms

Projects

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Luigi - Python module that helps you build complex pipelines of batch jobs

OpenERP - Open Source Business Applications

Pyres - Job queue in Python

Obsei - Low code AI powered automation tool

Tagcloud

License

Suggested keywords:

Projects

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Luigi - Python module that helps you build complex pipelines of batch jobs

OpenERP - Open Source Business Applications

Pyres - Job queue in Python

Obsei - Low code AI powered automation tool

Tagcloud

License