The Portland Group
Oakridge Top Right

Since 1986 - Covering the Fastest Computers
in the World and the People Who Run Them

Language Flags

Visit additional Tabor Communication Publications

Enterprise Tech
HPCwire Japan

Gemini Releases Real-Time Log Processing Based on Flume and Cassandra

FOSTER CITY, Calif., March 3, 2011 -- Gemini Mobile Technologies ("Gemini") released a Real-Time Log Processing System based on Flume and Cassandra ("Flume-Cassandra Log Processor") as open source today. The Flume-Cassandra Log Processor enables massive volumes of production system logs to be collected and processed into graphical reports, in real-time. In addition, logs from multiple data centers can be simultaneously aggregated and analyzed in a single database. With its ability for real-time analysis at unprecedented volumes, Gemini's Flume-Cassandra Log Processor enables businesses to vastly improve both the quality and timeliness of business intelligence gained from their online operations. This dramatic scalability at low cost and small footprint is enabled by NOSQL (Not Only SQL) technology, which originates from Cloud Storage technologies at Google, Facebook, and Amazon. 

Logs record online activities of users and applications, in real-time.  Web service providers, telecom carriers, eCommerce providers, and corporate websites analyze logs to improve customer experience and their business operations.  Traditional systems process logs offline because log files are too big for relational databases, typically producing reports days to weeks after the time when logs were generated.  Gemini's solution uses Flume to stream log entries as they occur, data is extracted on the fly and inserted in real-time into a Cassandra NOSQL database.  This allows reports to be updated in real-time, less than a second after the log event.  Offline analysis is also supported by querying the Cassandra database and running MapReduce jobs.  The Flume-Cassandra Log Processor can be easily scaled from a few PCs to multiple clusters, allowing hundreds of Terabytes of logs to be stored, analyzed, and automatically deleted upon expiration. 

"Business intelligence is the key ingredient for enterprises and service providers to provide a personal and effective web experience to their customers," said Michael Tso, co-founder and COO of Gemini.  "Our Flume-Cassandra Log Processor delivers business intelligence in real-time, at data volumes and velocities that were previously unattainable.  We are pleased to release it as open source, allowing easy customization and improvement by the community." 

"Gemini's real-time log processing solution is an impressive demonstration of Cassandra's capabilities for solving big data problems in real-time," said Matt Pfeil, CEO of DataStax, the commercial leader in Apache Cassandra.  "Cassandra is quickly gaining momentum as the scalable platform for web and enterprise applications. We look forward to continuing our innovation and collaboration with Gemini." 

The Flume-Cassandra Log Processor can be downloaded from Github:

About Gemini

Gemini is a leading provider of high-performance, cloud-enabled messaging platforms.  Gemini is a pioneer in real-time Big Data and NOSQL database technology, developing the Hibari® Key Value Store, Hibari Gigabyte-Maildir Email Store, and the Cloudian™ Multi-Tenant Cloud Storage System.  Gemini has offices in San Francisco, Tokyo and Beijing. Gemini's customers include NTT DOCOMO, Softbank Mobile, Vodafone, and Nextel International; the company also has OEM partnerships with Alcatel-Lucent and Bytemobile.  Gemini is backed by Goldman Sachs, Mitsubishi-UFG Capital, Nomura Securities, Mizuho Capital, Access, and Aplix.  For more information, visit


Source: Gemini

Most Read Features

Most Read Around the Web

Most Read This Just In

Most Read Blogs

Sponsored Whitepapers

Breaking I/O Bottlenecks

10/30/2013 | Cray, DDN, Mellanox, NetApp, ScaleMP, Supermicro, Xyratex | Creating data is easy… the challenge is getting it to the right place to make use of it. This paper discusses fresh solutions that can directly increase I/O efficiency, and the applications of these solutions to current, and new technology infrastructures.

A New Ultra-Dense Hyper-Scale x86 Server Design

10/01/2013 | IBM | A new trend is developing in the HPC space that is also affecting enterprise computing productivity with the arrival of “ultra-dense” hyper-scale servers.

Sponsored Multimedia

Xyratex, presents ClusterStor at the Vendor Showdown at ISC13

Ken Claffey, SVP and General Manager at Xyratex, presents ClusterStor at the Vendor Showdown at ISC13 in Leipzig, Germany.

HPCwire Live! Atlanta's Big Data Kick Off Week Meets HPC

Join HPCwire Editor Nicole Hemsoth and Dr. David Bader from Georgia Tech as they take center stage on opening night at Atlanta's first Big Data Kick Off Week, filmed in front of a live audience. Nicole and David look at the evolution of HPC, today's big data challenges, discuss real world solutions, and reveal their predictions. Exactly what does the future holds for HPC?


Stay informed! Subscribe to HPCwire email Newsletters.

HPCwire Weekly Update
HPC in the Cloud Update
Digital Manufacturing Report
HPCwire Conferences & Events
Job Bank
HPCwire Product Showcases


HPC Job Bank

Featured Events

HPCwire Events