Supercomputing 2009 - Higher Performance Computing

Warren Davidson and I attended Supercomputing 2009 (SC'09) in Portland, OR in mid-November. We didn't see any breakthrough technologies this year. Exhibitors were mostly showcasing incremental changes to their products.

Overall Impressions


The large DBMS vendors were noticeable absent and some of the largest storage vendors were missing too. That may be a sign that the recession is causing people to use their marketing budgets more carefully.

However, Supercomputing isn't really a sales venue for us. Most of the potential customers there are already Objectivity/DB users, so it's more of a venue for networking with customers and partners, plus an important opportunity to see what's going on in the HPC community.

Some companies, such as SiCortex and 3Com had either gone out of business or been taken over since last year's conference. Oracle and EMC were notably absent. Microsoft had a very large stand, as usual, but nothing really new to show in their HPC operating system lineup. There were far fewer independent cluster assemblers than last year.

Display Technology


There are generally a lot of extremely large displays, but they were noticeably absent from this year's show, perhaps because of controlled spending on the part of the exhibitors. The only 3D display, a goggles-free version, was there last year too. We did spot one display that takes up to 8 feeds from separate computers, which is handy in some environments, such as surveillance, simulation and C4IStar. FusionIO/ioDrive had an array of HD screens displaying thousands of discrete video streams being refreshed from solid state storage. It was impressive.

Networking


ICANN had a stand there for the first time ever. They're trying to get the word out that IPv6 is needed and real. They believe that there will be a significant government systems mandate about a month from now. Objectivity/DB Release 10 (scheduled for general availability later this quarter) supports IPv6.

Network bandwidth has been a major theme at all HPC events over the past two decades. The SciNet consortium had a very impressive array of equipment (worth $20M) demonstrating 40 and 100 Gigabit Ethernet and 12x QDR Infiniband. The WAN Bandwidth was 400 Gigabits per second - that's 200,000 times more than an average home connected to broadband and would be capable of transmitting 100 Blu-Ray movies in one minute. Several stands had audio/video links to remote locations, notably Cisco and the University of Georgia, the latter offering an opportunity to talk with a diver in an aquarium in Atlanta.

Processors


NVidea was the only major GPU manufacturer there last year. This year there was at least a dozen companies showing hybrid systems that combine conventional CPUs and GPUs or software to support highly parallel, hybrid environments. GPUs tend to help a lot with streaming data or highly repetitive, simple calculations, such as conversion from one architecture to another. IBM, HP, Dell, Cray and Penguin Computing had large stands. ORNL had chunks of various supercomputers in an armored glass case, raging from a feeble 50 Gigaflop board from 10 years ago to a 160 Teraflop board. The current quest is to build Petaflop machines, but a few places are looking at two or three orders of magnitude above that within 10 years.

Software


We saw a lot more parallel compiler development tools than last year. The price is dropping to around $600 per seat too. I also noted some interesting open source alternatives to Hadoop, such as Sector. Objectivity/DB is a distributed DBMS, so there's not much opportunity to parallelize much more in the kernel and all of the servers can exploit multi-core CPUs. However, parallelization tools may be of interest to application developers, so I'll watch out for good ones. If you're already using one, please let us know via the Comment link below.

Grid and Cloud Computing


There are a lot of cloud and grid dashboard companies out there and about half of the grid specialists now have a cloud story. Most of the major players, apart from Cloudera and GoogleCloud, were there. Objectivity/DB is now available on the Amazon EC2 Cloud, of course.

Storage


DDN had a huge stand, but, as I said above, other than solid state storage, IBM and HP, the storage industry was largely MIA. As has been the trend for the past decade, most advances were in storage density rather than random access or steaming speeds. It's odd to realize that a Terabyte of IBM RAMAC storage would have cost about $100 billion in 1957, versus around $100 now, but the access speeds are still about the same.

It's Easy Going Green With Objectivity/DB

IT departments are very aware of the need to save energy, colloquially labeled "going green". We can all do our bit to help by taking a few simple steps, such as changing the energy saving settings on our computers at work and home. I've recently started moving my personal web sites to a "green" web host provider. They guarantee that 100% of the energy they use in their data center comes from renewable resources. Objectivity's headquarters had a smart energy control system to reduce our overall energy consumption. Whether or not you believe in "saving the planet", it makes sense to save money.

If you're an Objectivity/DB user you're implicitly saving energy. When we benchmarked Objectivity/DB against RDBMSs a couple of years ago we needed one quarter of the processor power and 60% less storage than our competitors. Not all applications will show such a large saving, but there is always a CPU saving due to the lack of a mapping layer and a disk storage saving because of the lack of join tables and unnecessary indices. That can translate to better performance and greater throughput. It can also mean running fewer processors to achieve a task.

Objectivity/DB associations (called "relationships" in Java and C#) can take between 8 and 24 bytes per bi-directional link. In-line links can be 4 bytes to other objects within a container and 8 bytes for connections that span containers or databases. Dynamic links are always 12 bytes. Associations are extremely efficient relative to stored join tables and their accompanying B-Tree indices. Scalable collections also use less disk space than their relational equivalents. The more complex the application and data, the greater the saving.

You'll also be saving energy if it takes you less time to accomplish a development or administration task. Just remember to go home early and turn off the lights. Good luck on the going home early part! :-)