Asia/Pacific Expansion – Objectivity establishing key Partnerships to drive growth in the Big Data market

Contributed by Graham Freeman, Managing Director Asia Pacific Region – Our plans for expansion into Asia/Pacific are now well on track to drive toward initial growth objectives and expand our customer reach. A key element of our strategy has been to identify Strategic Partners who can help develop new applications that are specifically focused on the needs of customers in this Region. Our new Partners need to have a combination of skills to handle the business development, programming, consulting and support that our customer require.

On a recent trip to Japan to meet with potential Partners, the interest in Big Data/NOSQL solutions and the Cloud was extraordinary. Our timing could not have been better. The combination of Objectivity/DB and InfiniteGraph seem to hit the mark where the traditional, relational DB products struggle with scale and performance in this new, testing environment. As we continue to build Partnerships in SE Asia, Japan and Australia I fully expect that we can scale the business quickly to match the demands from our client base. While we can leverage our successes in the US and EMEA, it won’t be long before some of our early systems implementations in these markets may well provide a lead for similar solutions to be deployed in other parts of the world.

We head back to Singapore and Japan next week and given the growing number of requests for meetings, I have no doubt we will pave the way for an excellent run through year end 2012 and build a strong pipeline for opportunities in 2013 and beyond.

Real-time relationship analytics from large-scale graph processing

Real-time relationship analytics from large-scale graph processing

Cassandra excels at storing large, active, decentralized datasets. Additionally, Cassandra’s rich data model allows efficient use for many applications beyond simple associative arrays. One interesting application is the processing of large-scale graph structures.

I have devised a graph application layer to extract and process social network analysis data from Cassandra, using InfiniteGraph (which you can download and use for free). I have written more about the technical benefits of the social-graph-extract application layer and its use of graph-oriented processing on blog.stavi.sh.

Social network analysis is one application of a more general category, relationship analytics, as defined by Curt Monash. The relationship analytics problem domain maps well to the unique features of the Cassandra-InfiniteGraph hybrid system:

  • dedicated vertex/edge API
  • data can be clustered according to vertex/edge proximity
  • disk-based/memory-centric access
  • peer-to-peer communication from InfiniteGraph node to Cassandra node
  • bidirectional updates between raw Cassandra data and Infinitegraph analytics
  • parallel streaming and caching from InfintiteGraph
  • modeling flexibility to support a variety of sources
  • redundancy and high-availability
  • precision and speed for graph analytics
  • finding extremely long paths, all paths, unknown paths, or paths of nontrivial or indeterminate length

Current business problems that can utilize these features:

  • analyzing high-frequency trading
  • discovering high degrees of mutual interconnection in social networks
  • data mining subtle retail correlations
  • product recommendation engines
  • determining terrorist or criminal behavior inferred from known relationships
  • finding a pattern of relationships for fraud detection
  • investigating the directed relationships between proteins and genes
  • checking which entity has the shortest average connection to a group of others for cyber security (botnet controller)

The working codebase for this Cassandra / InfiniteGraph integration can be retrieved from GitHub. This project was originally coded using an early beta of InfiniteGraph, but I haven’t seen any issues with the latest version of InfiniteGraph. Forking of the main project is welcome (including downstream updates). If you have any questions or suggestions, please contact @toddstavish.

Todd Stavish

Todd Stavish is a Senior Systems Engineer for Objectivity, Inc. (the company behind InfiniteGraph), and is focused on our federal and government business. Todd Stavish has expertise in a range of distributed computing applications. He has worked in telecommunications, process control, auitomation and scientific computing. Todd specializes in advising customers about complex modeling, performance optimization and building fault tolerance applications.