Vespa is an engine for low-latency computation over large data sets. It stores and indexes your data such that queries, selection and processing over the data can be performed at serving time. Vespa is serving platform for Yahoo.com, Yahoo News, Yahoo Sports, Yahoo Finance, Yahoo Gemini, Flickr.
 Queries can use both structured filters and unstructured text search to select data. All the matching data is then ranked according to a ranking function - typically machine learned - to implement such use cases as search relevance, recommendation, targeting and personalization.
 
Vespa is scalable. System sizes up to hundreds of nodes handling tens of billions of documents are not uncommon, and no harder to set up and modify than single node systems. Since all system components, as well as stored data is redundant and self-correcting, hardware failures are not operational emergencies and can be handled by re-adding capacity when convenient.
Its feature include:
<ul>
<li>Text search - Combine structured query and text search to select data</li>
<li>Advanced Ranking - Machine learned ranking</li>
<li>Aggregation</li>
<li>Elastic, Scalable</li>
<li>High Availability</li>
<li>Auto repair data corruption</li>
<li>Simple HTTP API interface&nbsp;</li>
<li>lot more...</li>
</ul>

Vespa is an engine for low-latency computation over large data sets. It stores and indexes your data such that queries, selection and processing over the data can be performed at serving time. Vespa is serving platform for Yahoo.com, Yahoo News, Yahoo Sports, Yahoo Finance, Yahoo Gemini, Flickr.

Vespa - Yahoo's big data serving engine

Smile (Statistical Machine Intelligence and Learning Engine) is a fast and comprehensive machine learning, NLP, linear algebra, graph, interpolation, and visualization system in Java and Scala. With advanced data structures and algorithms, Smile delivers state-of-art performance.Smile covers every aspect of machine learning, including classification, regression, clustering, association rule mining, feature selection, manifold learning, multidimensional scaling, genetic algorithms, missing value imputation, efficient nearest neighbor search, etc.

Smile - Statistical Machine Intelligence & Learning Engine

The Oryx open source project provides infrastructure for lambda-architecture applications on top of Spark, Spark Streaming and Kafka. On this, it provides further support for real-time, large scale machine learning, and end-to-end applications of this support for common machine learning use cases, like recommendations, clustering, classification and regression.
  

Oryx 2 - Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning

MapD Core is an in-memory, column store, SQL relational database that was designed from the ground up to run on GPUs. MapD Core is the foundational element of a larger data exploration platform that emphasizes speed at scale. By taking advantage of the parallel processing power of the hardware, MapD Core can query billions of rows in milliseconds. Furthermore, by using the graphics pipelines of GPUs, MapD Core can render graphics directly from the server.
 
MapD can operate as a standalone database using the command line or its administration console. MapD can also output to its native interface, MapD Immerse, or to third-party software such as Tableau, Qlik, or Birst, using a variety of connectors. 
 
Within a single server, MapD Core can handle datasets 1.5-3TB in raw size in GPU RAM, or 10-15TB in CPU RAM. In distributed deployments, that number scales linearly.

 MapD Core is an in-memory, column store, SQL relational database that was designed from the ground up to run on GPUs. MapD Core is the foundational element of a larger data exploration platform that emphasizes speed at scale. By taking advantage of the parallel processing power of the hardware, MapD Core can query billions of rows in milliseconds. Furthermore, by using the graphics pipelines of GPUs, MapD Core can render graphics directly from the server.

MapD - The MapD Core database

Zipline is a Pythonic algorithmic trading library. It is an event-driven system that supports both backtesting and live-trading. Zipline is currently used in production as the backtesting and live-trading engine powering Quantopian -- a free, community-centered, hosted platform for building and executing trading strategies.Note: Installing Zipline via pip is slightly more involved than the average Python package. Simply running pip install zipline will likely fail if you've never installed any scientific Python packages before.

<ul>
<li>Ease of use: Zipline tries to get out of your way so that you can
focus on algorithm development. See below for a code example.</li>
<li>Zipline comes "batteries included" as many common statistics like
moving average and linear regression can be readily accessed from
within a user-written algorithm.</li>
<li>Input of historical data and output of performance statistics are
based on Pandas DataFrames to integrate nicely into the existing
PyData eco-system.</li>
<li>Statistic and machine learning libraries like matplotlib, scipy,
statsmodels, and sklearn support development, analysis, and
visualization of state-of-the-art trading systems.</li>
</ul>

Zipline is a Pythonic algorithmic trading library. It is an event-driven system that supports both backtesting and live-trading. Zipline is currently used in production as the backtesting and live-trading engine powering Quantopian -- a free, community-centered, hosted platform for building and executing trading strategies.Note: Installing Zipline via pip is slightly more involved than the average Python package. Simply running pip install zipline will likely fail if you've never installed any scientific Python packages before.

Zipline - A Pythonic Algorithmic Trading Library

MNN is a highly efficient and lightweight deep learning framework. It supports inference and training of deep learning models, and has industry leading performance for inference and training on-device. At present, MNN has been integrated in more than 20 apps of Alibaba Inc, such as Taobao, Tmall, Youku, Dingtalk, Xianyu and etc., covering more than 70 usage scenarios such as live broadcast, short video capture, search recommendation, product searching by image, interactive marketing, equity distribution, security risk control. In addition, MNN is also used on embedded devices, such as IoT.

MNN is a blazing fast, lightweight deep learning framework, battle-tested by business-critical use cases in Alibaba

TensorFlow is a library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) that flow between them. This flexible architecture lets you deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device without rewriting code. 

TensorFlow - Artificial Intelligence Library from Google

The Kubeflow project is dedicated to making deployments of machine learning (ML) workflows on Kubernetes simple, portable and scalable. Kubeflow is a platform for data scientists who want to build and experiment with ML pipelines. Kubeflow is also for ML engineers and operational teams who want to deploy ML systems to various environments for development, testing, and production-level serving.

kubeflow - Machine Learning Toolkit for Kubernetes

Rasa is an open source machine learning framework to automate text-and voice-based conversations. With Rasa, you can build chatbots on Facebook, Slack, Microsoft Bot Framework, Rocket.Chat, Mattermost, Telegram etc. Rasa's primary purpose is to help you build contextual, layered conversations with lots of back-and-forth. To have a real conversation, you need to have some memory and build on things that were said earlier. Rasa lets you do that in a scalable way.

Rasa - Create chatbots and voice assistants

Gorgonia is a library that helps facilitate machine learning in Go. Write and evaluate mathematical equations involving multidimensional arrays easily. If this sounds like Theano or TensorFlow, it's because the idea is quite similar. Specifically, the library is pretty low-level, like Theano, but has higher goals like Tensorflow.
 
Gorgonia can perform automatic differentiation, symbolic differentiation, gradient descent optimizations, numerical stabilization, distributed computing. 
 
The primary goal for Gorgonia is to be a highly performant machine learning/graph computation-based library that can scale across multiple machines. It should bring the appeal of Go (simple compilation and deployment process) to the ML world. It's a long way from there currently, however, the baby steps are already there.

Gorgonia is a library that helps facilitate machine learning in Go. Write and evaluate mathematical equations involving multidimensional arrays easily. If this sounds like Theano or TensorFlow, it's because the idea is quite similar. Specifically, the library is pretty low-level, like Theano, but has higher goals like Tensorflow.



Gorgonia - Library that helps facilitate machine learning in Go

Argo has set of open source tools for Kubernetes to run workflows, manage clusters, and do GitOps right. 
 
<ul><li>Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition)</li><li>Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes</li><li>Argo Events is an event-driven workflow automation framework for Kubernetes. It allows you to trigger 10 different actions (such as the creation of Kubernetes objects, invoke workflows or serverless workloads) on over 20 different events (such as webhook, S3 drop, cron schedule, messaging queues - e.g. Kafka, GCP PubSub, SNS, SQS)</li><li>Argo Rollouts is a Kubernetes controller and set of CRDs which provide advanced deployment capabilities such as blue-green, canary, canary analysis, experimentation, and progressive delivery features to Kubernetes</li></ul>

Argo has set of open source tools for Kubernetes to run workflows, manage clusters, and do GitOps right. 

Argo Project - Get stuff done with Kubernetes

Streamlit is an open-source Python library that makes it easy to create and share beautiful, custom web apps for machine learning and data science. Build an app in a few lines of code with magically simple API. Adding a widget is the same as declaring a variable. No need to write a backend, define routes etc. Effortlessly share, manage, and collaborate on your apps directly from Streamlit. Apps are deployed directly from Github repo.

Streamlit - The fastest way to build data apps in Python

Apache TVM, a deep learning compiler that enables access to high-performance machine learning anywhere for everyone. TVM’s diverse community of hardware vendors, compiler engineers and ML researchers work together to build a unified, programmable software stack, that enriches the entire ML technology ecosystem and make it accessible to the wider ML community. TVM empowers users to leverage community-driven ML-based optimizations to push the limits and amplify the reach of their research and development, which in turn raises the collective performance of all ML, while driving its costs down.

TVM - Open deep learning compiler stack for cpu, gpu and specialized accelerators

Stanza is a Python NLP Library for Many Human Languages. It contains support for running various accurate natural language processing tools on 60+ languages and for accessing the Java Stanford CoreNLP software from Python. A new collection of biomedical and clinical English model packages are now available, offering seamless experience for syntactic analysis and named entity recognition (NER) from biomedical literature text and clinical notes.

Stanza - A Python NLP Library for Many Human Languages

Jina is a neural search framework that empowers anyone to build SOTA & scalable deep learning search applications in minutes. It helps to build solutions for indexing, querying, understanding multi-/cross-modal data such as video, image, text, audio, source code, PDF.

Jina - Cloud-native neural search framework for any kind of data

Discover open source projects across all platforms

Projects