Open source projects used in LinkedIn
LinkedIn is a social network for professionals. It is built using open source products. They sponsored many projects to open source. Here are the list of open source products used by LinkedIn.
Programming language: LinkedIn uses C++, Java, Scala, Python and Ruby.
Hadoop is a framework for running applications on large clusters built of commodity hardware. http://hadoop.apache.org/
Azkaban is simple batch scheduler for constructing and running Hadoop jobs or other offline processes. It is a workflow system for Hadoop which allows a program to be built out of small reusable pieces that need not know about one another. By declaring dependencies, you can control sequencing. http://sna-projects.com/azkaban/
Pig is a high-level data-flow language and execution framework for parallel computation. http://pig.apache.org/
Zoie is a real-time search and indexing system built on Apache Lucene. http://sna-projects.com/zoie/
Bobo is a Faceted Search implementation in Java, an extension of Apache Lucene. http://sna-projects.com/bobo/
Sensei is a distributed, elastic, realtime, searchable database. It is designed to handle billions of rows. http://sna-projects.com/sensei/
ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. http://zookeeper.apache.org/
Netty provides an asynchronous event-driven network application framework and tools. It is a NIO client server framework which enables quick and easy development of network applications such as protocol servers and clients. http://www.jboss.org/netty
Norbert is a library that provides easy cluster management and workload distribution. It is implemented in Scala. It helps to create a highly scalable architecture capable of handling heavy traffic. http://sna-projects.com/norbert/
Voldemort is a distributed key-value storage system. It is highly scalable as the data gets automatically replicated and partitioned across servers. http://project-voldemort.com/
kafka is a distributed publish/subscribe message queue system. http://sna-projects.com/kafka/
Kamikaze is a utility package for performing operations on compressed arrays of sorted integers. Search indexes, graph algorithms, and certain sparse matrix representations make heavy use of integer arrays, and special compression techniques are needed to get good compression performance on this data. http://sna-projects.com/kamikaze/
Krati is a simple persistent data store with very low latency and high throughput. It relies on hash-based indexing which is suitable for random reads and writes. http://sna-projects.com/krati/
Programming language: LinkedIn uses C++, Java, Scala, Python and Ruby.
Hadoop is a framework for running applications on large clusters built of commodity hardware. http://hadoop.apache.org/
Azkaban is simple batch scheduler for constructing and running Hadoop jobs or other offline processes. It is a workflow system for Hadoop which allows a program to be built out of small reusable pieces that need not know about one another. By declaring dependencies, you can control sequencing. http://sna-projects.com/azkaban/
Pig is a high-level data-flow language and execution framework for parallel computation. http://pig.apache.org/
Zoie is a real-time search and indexing system built on Apache Lucene. http://sna-projects.com/zoie/
Bobo is a Faceted Search implementation in Java, an extension of Apache Lucene. http://sna-projects.com/bobo/
Sensei is a distributed, elastic, realtime, searchable database. It is designed to handle billions of rows. http://sna-projects.com/sensei/
ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. http://zookeeper.apache.org/
Netty provides an asynchronous event-driven network application framework and tools. It is a NIO client server framework which enables quick and easy development of network applications such as protocol servers and clients. http://www.jboss.org/netty
Norbert is a library that provides easy cluster management and workload distribution. It is implemented in Scala. It helps to create a highly scalable architecture capable of handling heavy traffic. http://sna-projects.com/norbert/
Voldemort is a distributed key-value storage system. It is highly scalable as the data gets automatically replicated and partitioned across servers. http://project-voldemort.com/
kafka is a distributed publish/subscribe message queue system. http://sna-projects.com/kafka/
Kamikaze is a utility package for performing operations on compressed arrays of sorted integers. Search indexes, graph algorithms, and certain sparse matrix representations make heavy use of integer arrays, and special compression techniques are needed to get good compression performance on this data. http://sna-projects.com/kamikaze/
Krati is a simple persistent data store with very low latency and high throughput. It relies on hash-based indexing which is suitable for random reads and writes. http://sna-projects.com/krati/