Tuesday, February 10, 2015
The proponents of the Berlin Games react as thin-skinned
Thursday, August 19, 2010
Oracle loses another DTrace creator
Leventhal said, in a blog posting, that at Sun he had found himself "surrounded by superlative engineers" and that he felt lucky to have worked with Cantrill and Shapiro on DTrace. Most recently, Leventhal had been working on Fishworks, Sun's Solaris based storage system technology. Leventhal does not say what he will be doing next, only that he is "off to look for my next remarkable place and time beyond the walls of Oracle". It is possible he could follow in Cantrill's footsteps; within days of leaving, it was announced he had become Vice President of Engineering at Joyent, one of the companies involved in the OpenSolaris derivative Illumos which was launched at the beginning of August
The 2010 Linux Storage and Filesystem Summit, day 2
Writeback
The first session of the day was dedicated to the writeback issue. Writeback, of course, is the process of writing modified pages of files back to persistent store. There have been numerous complaints over recent years that writeback performance in Linux has regressed; the curious reader can refer to this article for some details, or this bugzilla entry for many, many details. The discussion was less focused on this specific problem, though; instead, the developers considered the problems with writeback as a whole.
Sorin Faibish started with a discussion of some research that he has done in this area. The challenges for writeback are familiar to those who have been watching the industry; the size of our systems - in terms of both memory and storage - has increased, but speed of those systems has not increased proportionally. As a result, writing back a given percentage of a system's pages takes longer than it once did. It is always easier for the writeback system to fail to keep up with processes which are dirtying pages, leading to poor performance.
His assertion is that the use of watermarks to control writeback is no longer appropriate for contemporary systems. Writeback should not wait until a certain percentage of memory is dirty; it should start sooner, and, crucially, be tied to the rate with which processes are dirtying pages. The system, he says, should work much more aggressively to ensure that the writeback rate matches the dirty rate.
From there, the discussion wandered through a number of specific issues. Linux writeback now works by flushing out pages belonging to a specific file (inode) at a time, with the hope that those pages will be located nearby on the disk. The memory management code will normally ask the filesystem to flush out up to 4MB of data for each inode. One poorly-kept secret of Linux memory management is that filesystems routinely ignore that request - they typically flush far more data than requested if there are that many dirty pages. It's only by generating much larger I/O requests that they can get the best performance.