[TriLUG] file copy flushing memory

[Joseph Mack NA3T]

On Fri, 2 Dec 2005, Timothy A. Chagnon wrote:

> Maybe since you have a small amount of memory in that machine, when
> you do the copy, the disk buffer gets filled with the blocks from the
> file you are copying.  Other applications with open files no longer
> have their blocks in the disk buffer, so they have to wait in line for
> disk access.

the two activities were

o disk to disk copy

o fetch webpage with 10M of images over network

I guess the 10M could be pushed to swap, but when I change 
screens and then back again, the images are still there (and 
don't have to be loaded from swap). I assume then that these 
two activities are orthogonal (that was the intention at 
least).

> I did some googling and found a few items about possible ways to skip
> the disk buffer and asynchronous I/O (AIO, libaio1):
> http://lse.sourceforge.net/io/aio.html
> http://www.oracle.com/technology/pub/columns/talkinglinux_0104.html
>
> These imply that with AIO, it is possible to open a file (via the
> open(2) function) with the O_DIRECT flag, thus bypassing the disk
> buffer.  Googling for this, there are a lot of mailing list threads
> about dd having different command line options to use this.
>
> dd conv=direct
> dd iflags=direct oflags=direct
> dd o_direct=yes
> dd --o_direct=512,1024

OK will give these a try. I first looked at all these flags 
when I was first learning unix and went "groan" and have not 
thought about them since.

As for reprogramming tools to do file copies; I was hoping 
not to have to do this. For the moment, I'll just put up 
with the slowdown.


> Oh, well.  Maybe just try the dd commands or try to get a copy with
> those options.  Interesting though.

I don't see why a copy need flush all of memory by default. 
A copy only needs enough memory (which can't be a whole lot) 
and not ask for any more.

Thanks for the ideas

Joe

-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml 
Homepage http://www.austintek.com/ It's GNU/Linux!