[TriLUG] file copy flushing memory

Timothy A. Chagnon tachagnon at gmail.com
Fri Dec 2 16:28:31 EST 2005


Maybe since you have a small amount of memory in that machine, when
you do the copy, the disk buffer gets filled with the blocks from the
file you are copying.  Other applications with open files no longer
have their blocks in the disk buffer, so they have to wait in line for
disk access.

I did some googling and found a few items about possible ways to skip
the disk buffer and asynchronous I/O (AIO, libaio1):
http://lse.sourceforge.net/io/aio.html
http://www.oracle.com/technology/pub/columns/talkinglinux_0104.html

These imply that with AIO, it is possible to open a file (via the
open(2) function) with the O_DIRECT flag, thus bypassing the disk
buffer.  Googling for this, there are a lot of mailing list threads
about dd having different command line options to use this.

dd conv=direct
dd iflags=direct oflags=direct
dd o_direct=yes
dd --o_direct=512,1024

None of these worked on my Ubuntu 5.10 with kernel 2.6.12 and
coreutils 5.2.1. But grepping for O_DIRECT in /usr/include reveals
that those flags are in the headers of asm-i386/fcntl.h and
asm-x86_64/fcntl.h

I tried writing a quicky C program to see if it would work.  The open
with O_DIRECT did not produce an error, but trying to read or write to
that file descriptor fails with  "EINVAL fd is attached to an object
which is unsuitable for writing."  There's a thing that says O_SYNC
(implied by O_DIRECT) is only implemented on ext2, but that didn't
work either.

Oh, well.  Maybe just try the dd commands or try to get a copy with
those options.  Interesting though.

Tim



More information about the TriLUG mailing list