Turning Data into Intelligence for Improved Security

Most people today are aware that businesses and infrastructure are increasingly at risk of a broad spectrum of security events, including terrorism, Advanced Persistent Threats (APT), fraud, insider attacks, and other criminal activities. While many are focused on creating technologies to counter such threats, others are developing methods that can proactively track suspicious behavior patterns and activities in order to reveal hidden threats and dangers in enough time to take immediate action to prevent them from occurring in the first place. One approach, known as Information Fusion, is seeing significant inroads as a means to stop threats before they can be executed. For example, governments and large organizations currently have custom-built, complex solutions to capture, store, and search large amounts of real-time, streaming data from different sensors and sources such as satellites, monitoring systems, communication networks, mobile devices, and digital media. Built on extremely scalable and distributed technologies, these solutions enable analysts to build on top of the data and add human inferences to help create a realistic view of the situation being monitored. This incorporation of human reasoning as part of the data being collected is the differentiation between basic Data Integration and complex Information Fusion. The value of Information Fusion systems is to provide in-time access to all relevant components of Big Data for analytic applications. This enables the discovery of hidden connections and relationships within disparate data, allowing analysts to “connect-the-dots” and develop a common view of multiple data streams to support advanced analytic applications. Objectivity’s products, Objectivity/DB and InfiniteGraph, are based on object and graph oriented technologies, making them well-suited to support this type of...
New and Notable Features of IG 3.4

New and Notable Features of IG 3.4

Submitted by nquinn on April 6, 2015 - 10:51am   Introduction InfiniteGraph’s 3.4 release provides many performance enhancements that affect various operations, from adding data to performing navigation queries. Most enhancements are behind the scenes and can be leveraged through existing API methods, but some enhancements are available only through new API. Some notable API enhancements include: A built-in result handler called a PathCollector that can optimize complex navigations by collecting results into an array instead of iterating over the results as they are found A transaction policy called PreloadCacheForHandlesPolicy that optimizes object access when using methods on the Vertex interface These methods can lead to a huge performance boost in certain circumstances, as described below. Path Collector There are two approaches for handling the results of navigational queries. You can iterate over results as they are encountered, or you can collect the results and examine them when the navigation is complete. The iterative approach is preferred when the graph database is a backend engine to a visualization or web application that presents results to the user in real-time. Alternately, in many analytic applications, the entire result set is preferred over iterative results. If you want the entire result set or if you simply want to optimize the navigation as much as possible, you can use a new type of built-in result handler called a PathCollector. An iterating built-in or custom result handler will allow you to utilize the default asynchronous behavior of the navigation engine. This will be useful in two cases: long running navigations in which any results are preferred over the entire set navigation results are...
Building a Recommendation Engine Using a Graph Database

Building a Recommendation Engine Using a Graph Database

Submitted by nquinn on April 6, 2015 - 12:51pm Introduction A very common use case for Graph Databases is to support client applications that provide suggestions for their customers or rate products or services based on user behavior or preference. For the InfiniteGraph 3.4 release, we built a Podcast Recommendation Sample using the features available in IG 3.4 and previous releases. A recommendation engine is typically built using a graph database for three reasons: Recommendations use multiple factors that have complex relationships Recommendations are given in real-time Recommendations utilize multiple levels of relationships For example, recommending purchases might involve looking at purchase event information and the many types of relationships between purchases, including customer preferences, website, store or brand loyalty, and general popularity. It also may be beneficial to look at relationships between customers, including trends within communities, socio-economic groups, etc. This type of relationship analytics is most successful after looking at multiple levels of depth to see how pervasive trends may be. This blog will look at complex relationships and multiple levels of relationship in graph data sets to see how they can be mined to develop an interesting and compelling recommendation algorithm. Design To create a successful recommendation service, care must be taken to determine what types of data are required to create a compelling recommendation engine. For example, if we recommend podcasts based only on popularity, then we would only be offering recommendations along one differentiating factor. All queries would return the same top set of podcast titles based on that one factor. Even two factors of differentiation is uninteresting. Consider using two factors like category...
Everybody Forgets about Path Qualification

Everybody Forgets about Path Qualification

Is a single traversal of your graph database taking an insanely long time or is it crashing your application because it is consuming all of your free memory? It is possible that you have neglected to filter the search space for your navigation using path qualification. This is a review of all of the powerful ways that InfiniteGraph gives you to filter your result set before you even execute a navigation.