The right way to do it is to just not use O_DIRECT. The whole notion of "direct IO" is totally braindamaged. Just say no. This is your brain: O This is your brain on O_DIRECT: . Any questions? Our measurements show that the page cache dramatically increases latency. At the speeds of flash storage, the page cache is disastrous. Linus agreed substantially in his own post: Side note: the only reason