Oracle loses another DTrace creator

Adam Leventhal, another one of the three Sun engineers who developed DTrace, is leaving Oracle. Bryan Cantrill, another member of the team, left Oracle in July. This leaves only one of the original DTrace team – Mike Shapiro – still with Oracle. Shapiro is one of the signatories of the leaked memo which announced the closing of the OpenSolaris project.
Leventhal said, in a blog posting, that at Sun he had found himself "surrounded by superlative engineers" and that he felt lucky to have worked with Cantrill and Shapiro on DTrace. Most recently, Leventhal had been working on Fishworks, Sun's Solaris based storage system technology. Leventhal does not say what he will be doing next, only that he is "off to look for my next remarkable place and time beyond the walls of Oracle". It is possible he could follow in Cantrill's footsteps; within days of leaving, it was announced he had become Vice President of Engineering at Joyent, one of the companies involved in the OpenSolaris derivative Illumos which was launched at the beginning of August

The 2010 Linux Storage and Filesystem Summit, day 2

The second day of the 2010 Linux Storage and Filesystem Summit was held on August 9 in Boston. Those who have not yet read the coverage from day 1 may want to start there. This day's topics were, in general, more detailed and technical and less amenable to summarization here. Nonetheless, your editor will try his best.

Writeback

The first session of the day was dedicated to the writeback issue. Writeback, of course, is the process of writing modified pages of files back to persistent store. There have been numerous complaints over recent years that writeback performance in Linux has regressed; the curious reader can refer to this article for some details, or this bugzilla entry for many, many details. The discussion was less focused on this specific problem, though; instead, the developers considered the problems with writeback as a whole.

Sorin Faibish started with a discussion of some research that he has done in this area. The challenges for writeback are familiar to those who have been watching the industry; the size of our systems - in terms of both memory and storage - has increased, but speed of those systems has not increased proportionally. As a result, writing back a given percentage of a system's pages takes longer than it once did. It is always easier for the writeback system to fail to keep up with processes which are dirtying pages, leading to poor performance.

His assertion is that the use of watermarks to control writeback is no longer appropriate for contemporary systems. Writeback should not wait until a certain percentage of memory is dirty; it should start sooner, and, crucially, be tied to the rate with which processes are dirtying pages. The system, he says, should work much more aggressively to ensure that the writeback rate matches the dirty rate.

From there, the discussion wandered through a number of specific issues. Linux writeback now works by flushing out pages belonging to a specific file (inode) at a time, with the hope that those pages will be located nearby on the disk. The memory management code will normally ask the filesystem to flush out up to 4MB of data for each inode. One poorly-kept secret of Linux memory management is that filesystems routinely ignore that request - they typically flush far more data than requested if there are that many dirty pages. It's only by generating much larger I/O requests that they can get the best performance.

What to Look for in Open Source Systems Management Products

"You could call them the "Little 4," a play on the "Big 4" systems management companies: BMC, CA Technologies, HP, and IBM.

"The Little 4 may be smaller in terms of their bottom lines, but in terms of systems management capabilities, there's nothing little about the open source offerings from Zenoss, Hyperic, GroundWork Open Source, and The OpenNMS Group.

"This is a biosphere within the open source ecosystem that has seen a lot of flux in the past few years. In 2006, Zenoss was one of six vendors (the others being Ayamon, Emu Software, Qlusters, Symbiot, and Webmin) that launched the Open Management Consortium (OMC), "an effort to promote the adoption, development, and integration of systems and network management software based on open source and open standards technologies."

Hard Drive Caching with SSDs

The hard drives we use today are very complex beasts. Cramming that much data onto a very small space while insuring that the data will be there when we need it, is not something easy or simple. The pace of data density increase in hard drives has been nothing short of amazing. At the same time, we all want faster and faster performance from our drives. To help improve hard drive performance, manufacturers introduced a small amount of RAM to the drive. It can be used to hold data for a period of time before writing it to disk, allowing the drive to tell the operating system that the data is actually on the disk, improving application performance. It can also be used to pre-fetch data for reading. In either case, a small amount of RAM can make a very large improvement in performance. If you don’t believe me, run some benchmarks with the disk cache turned on and then off and compare the two results.