[TriLUG] Growing OS cache in RAM causing high system CPU usage

[Daniel Sterling]

I didn't expect this, but amazingly it appears you can use cgroups to
limit page cache usage.

http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/cgroups/memory.txt?id=HEAD

Try setting up a cgroup and use accounting to limit the RAM available
to the processes

-- Dan



On Tue, Jul 8, 2014 at 1:40 PM, Michael Peters
<michael00peters at gmail.com> wrote:
> On Tue, Jul 8, 2014 at 1:28 PM, Bill Farrow <bill at arrowsreach.com> wrote:
>
>> This process is "seamless" to your application but there is still
>> going to be a CPU overhead involved here.
>
> I know there is some overhead, but this seems really excessive. 8 CPUs
> pegged 96-99% for 3 minutes?
>
>> The memory pages being used
>> for OS cache need to be flushed from the CPU caches, and then possibly
>> written out to disk if they are dirty (have changes in them).
>
> There's only a tiny amount of dirty memory in the cache (2060 kB0 and
> no increase in disk IO during these CPU spikes.
>
>> Then
>> they need to be marked as unused and the OS table/list/hash etc needs
>> to be updated accordingly.
>>
>> Once the memory is "unused", it probably has to be initialized to zero
>> before being handed to the application to prevent data leakage.
>>
>> So it is not CPU "free" even if it is seamless to your application.
>
> So given that the memory management overhead of the OS using too much
> cache seems to be causing this how can I limit that? I don't see
> anything in the virtual memory settings to control this. And while I
> can force the cache to be cleared with /proc/sys/vm/drop_caches that's
> a pretty blunt tool that clears everything and will impact performance
> as the OS tries to refill the most needed things. Is there a way to
> clear out the older parts of the cache explicitly?
> --
> This message was sent to: Daniel S. Sterling <sterling.daniel at gmail.com>
> To unsubscribe, send a blank message to trilug-leave at trilug.org from that address.
> TriLUG mailing list : http://www.trilug.org/mailman/listinfo/trilug
> Unsubscribe or edit options on the web  : http://www.trilug.org/mailman/options/trilug/sterling.daniel%40gmail.com
> Welcome to TriLUG: http://trilug.org/welcome