Object Oriented Database Learning Center

Objectivity Home - Government - Objectivity/DB - Webinars - Download Software

Download a PDF version of this Article

Learning Center Home

Table of contents:

Building High-perfomance Object Oriented Databases

• The Problem
• High ingest rates
• High delete/archival rates
• Complex relationships
• Flexibility
• Performance Optimization
• Very Large Cardinalities
• A Solution
• Persistence
• The Hierarchical Solution
• The Relational Solution
• An Object-oriented approach to persistence
• Query/Retrieval
• Query by Attribute
• Query by Traversal
• Cache
• Concurrency
• Schema Control and Data Migration
• Transactions (Commit / Rollback)
• Single, Consistent View of Data
• A Practical Example
• Logical Schema
• Message Class
• Attachments and Relationships
• Sender, Recipients, CC Lists and Addresses
• Physical Data Placement
• Using Databases and Containers for Fast Archival and Deletes
• Using Databases and Containers for Fast Queries
• Using Databases, Containers and Range Lists for Parallel Queries
• Hardware Architecture Design and Sizing
• Objectivity/DB Terminology

Using Databases and Containers for Fast Archival and Deletes: In order to sustain the rate of incoming messages, we’ll need to be able to archive older messages and delete them as fast as the incoming rate. We discussed earlier how this can be a problem if one is using queries to find the messages and delete them.

The preferred design partitions data according to time so that large numbers of messages and attachments can be archived at once. This can be accomplished easily with Objectivity/DB by using the physical database and container concepts.

In the diagram to the left, each blue cylinder represents a database and each yellow block represents a container. Each database can repre sent a time period (e. g. an hour) and each container a smaller time period (e. g. a second).

In this case, each database would have 3600 containers of data, one for each second in its hour. Two weeks of data would require 14*24 = 336 databases.

If we run parallel threads or processes for ingesting the data, we’ll want to provide separate databases (and consequently separate containers) for each process or thread. This would multiply the number of databases required by the number of threads. We’ll examine how to optimize the number of concurrent threads later in the paper.

Since Objectivity/DB uses a separate file for each database, copying a specific hour’s worth of data to an archive and deleting it requires little more than using the operating system calls to copy or move the appropriate file(s). Deleting an hour’s worth of data could actually be accomplished in a matter of seconds or less.

This does not prevent optimizing queries that are not time-based, nor does this design require us to organize all the federation’s data by time. Logical query optimizations can be employed using indexes and customizable optimizations like hash maps.