|
Download a PDF version of this Article Table of contents: Objectivity/DB - Flexible Deployment
|
System Latency The time between the arrival or generation of data and its availability to users or other systems can be thought of as a form of system latency. It may vary according to the type of data, because groups of data need to arrive and be recognized, or because some of the processing involved takes a significant amount of time. It is possible to configure the various stages in a data ingest and processing pipeline to reduce the amount of additional indexing required. If arriving objects can be correlated, linked together with their partners, or quickly matched to existing objects, there may be no need for indices, which carry their own storage and processing overheads. A separate White Paper – “Building a high throughput data repository with high query performance”, describes some of the techniques used in a recent benchmark. Relational systems performing the same task had a latency of up to fifteen minutes. The Objectivity/DB implementation made the new data available within less than a second after the arrival of all of the related data. There is also a subtle difference between the ways that relational DBMSs and Objectivity/DB handle queries. An RDBMS locates all of the data required to satisfy a query, assembles it into a tabular view, then makes it available to the client one or more rows at a time. A sequential scan across a one Petabyte database could take many years and no results would return until all of the qualified rows were found.
Latency generally drops dramatically when Objectivity/DB is deployed. However, what happens if a very large amount of remote data has to be queried and very few objects are likely to qualify? In this case it will be better to configure a Parallel Query Engine and to locate its agents close to the databases to be searched.
|
Copyright © Objectivity, Inc. 2000-2007. All Rights Reserved.