Input/Output Capabilities
Input/Output Capabilities
There are two important factors to consider when deciding how to select or position persistent storage:
- Access speeds:
- Serial access speeds (milliseconds)
- Random access speeds (milliseconds)
- Input/Output bandwidth (MB or GB per second)
Modern disks hold many orders of magnitude more data in significantly less space and for a much lower price than they did twenty years ago. Serial access speeds have increased significantly but not dramatically and random access speeds are about the same (or even less) than they were in earlier devices.
RAID technology has increased the reliability of disk subsystems, but numerous studies have shown that the failure rate of individual drives, no matter what technology is used, and is actually quite high. Large installations can expect to use a highly utilized random access disk for about a year before it fails. It is always prudent to replicate or backup important groups of data.
It is also important to benchmark the random and serial access capabilities of a storage device or subsystem with multiple users before finalizing a configuration. The figures provided by hardware manufacturers should be viewed as an indication of performance under ideal conditions, which hardly ever arise in deployed systems.
Many applications that use Objectivity/DB perform large numbers of random access operations, so it is almost always better to use many small, low latency disks than a large, slower disk. It may be better to ingest fast streams of raw data onto small, fast disks, process it from there, and then move it to large, slower disks for occasional users of the raw data. You can compare the “Seek Time” figures for disks to find likely candidates. Figure 5 (below) summarizes typical configurations.
The rate at which a disk, or a disk subsystem, can deliver data to the processors that need it depends on the technology used for the mechanism itself and its connections, which may be: direct; via shared memory or a switch; or over a network. The overall throughput required of a disk configuration depends on the actual amount of traffic to and from the disk on behalf of multiple users. Caching near the disk can dramatically increase the perceived disk throughput, providing near to memory access times for frequently used data. Objectivity/DB has its own caching and replication mechanisms (discussed later), which may obviate the need for some kinds of hardware or file system cache.
Single disks built into commodity processors are unlikely to match the peak throughput of a more sophisticated, cached storage subsystem. However, racks of low-end disks may be more suitable for some archival and query operations than tape based solutions.
