[TriLUG] The Linux kernel has been accidentally hard-coded to a maximum of 8 cores for the past 15 years and nobody noticed

[Tim Shearer via TriLUG]

Dug into it a little bit, and my conclusion is that the author is
somewhat overinflating the effects of what they describe (disclaimer -
I don't have a PhD in OS scheduling). Yes, the documentation (and
behaviour) differs from what's commented/documented, but the overall
effect doesn't "hardcode" the scheduler to using a maximum of 8 cores
- nothing like it.

When systems that have more scheduled work than they have time to
perform that work (i.e. overloaded systems), each scheduled task gets
allotted a minimum amount of CPU time in which to run.The reason for
doing this is to avoid the inefficiencies that arise from lots of
context switching between different tasks (thrashing). This minimum
runnable time is found in /sys/kernel/debug/sched/min_granularity_ns,
and it varies depending on the number of CPU cores. Well, up to 8 CPU
cores, as the author noted.

// Here's the calculation:
  cpus = min(num_online_cpus(), 8)
  factor = 1 + ilog2(cpus);
  sysctl_sched_min_granularity = factor * 750000

// Examples (verified by reading /sys/kernel/debug/sched/min_granularity_ns)
4 cores:
   sched_min_granularity = 3 * 750000
   = 2250000ns

8 cores:
   sched_min_granularity = 4 * 750000
   = 3000000ns

64 cores (same as 8 cores because of the min() above:
   sched_min_granularity = 4 * 750000
   = 3000000ns

Therefore, the 64 core box will run a task for at least 3ms, whereas
the proper documented implementation would equal 5.25ms, the
implication being that there will be almost twice as many context
switches on your fully loaded 64 core box, inferring more scheduling
overhead.

Who here is routinely saturating their 16+ core systems? If so, you
might be affected. Otherwise I wouldn't worry about it too much!

Thanks,
Tim


On Tue, Nov 14, 2023 at 7:14 AM Peter Soper via TriLUG
<trilug at trilug.org> wrote:
>
> I was going to write something similar: this would have been noticed! I suspect the blog will hit Hacker News before too long and the remarks should be interesting.
> Pete
>
> Nov 14, 2023 6:56:16 AM David Both via TriLUG <trilug at trilug.org>:
>
> > I'm not sure how that conclusion can be reached in light of the fact
> > that I have been using 6, 8, 12, 14, and 16 cores (X2 for CPUs with HT)
> > for years. According to all the tools like top, htop, atop, glances, and
> > more, all of those cores and CPUs are in full use. Also the results my
> > systems show on the stats page of World Community Grid shows that all my
> > computers are performing calculations for the full number of cores and
> > CPUs present. So my 16-core, 32-CPU i9 host generates 32 days of
> > processing for each 24-hour day it runs.
> >
> > However it was an interesting read. ;-)
> >
> > --
> >
> >
> > *********************************************************
> > David P. Both, RHCE
> > He/Him/His
> > *********************************************************
> > www.both.org - My personal web site
> > www.Linux-Databook.info - Home of the DataBook for Linux
> > DataBook is a Registered Trademark of David Both
> > *********************************************************
> > The value of any software lies in its usefulness
> > not in its price.
> >
> > — Linus Torvalds
> > *********************************************************
> >
> > On Tue, 14 Nov 2023, brian mullan via TriLUG wrote:
> >
> >> Date: Tue, 14 Nov 2023 06:29:46
> >> From: brian mullan via TriLUG <trilug at trilug.org>
> >> Reply-To: brian mullan <bmullan.mail at gmail.com>,
> >>     Triangle Linux Users Group General Discussion <trilug at trilug.org>
> >> To: Triangle Linux Users Group General Discussion <trilug at trilug.org>
> >> Subject: [TriLUG] The Linux kernel has been accidentally hard-coded to a
> >>     maximum of 8 cores for the past 15 years and nobody noticed
> >> Well this was an interesting article to read at 6am
> >>
> >> *The Linux kernel has been accidentally hardcoded to a maximum of 8 cores
> >> for the past 15 years and nobody noticed*
> >> <https://thehftguy.com/2023/11/14/the-linux-kernel-has-been-accidentally-hardcoded-to-a-maximum-of-8-cores-for-nearly-20-years/>
> >>
> > --
> > This message was sent to: Pete Soper <pete at soper.us>
> > To unsubscribe, send a blank message to trilug-leave at trilug.org from that address.
> > TriLUG mailing list : https://www.trilug.org/mailman/listinfo/trilug
> > Unsubscribe or edit options on the web  : https://www.trilug.org/mailman/options/trilug/pete%40soper.us
> > Welcome to TriLUG: https://trilug.org/welcome
> --
> This message was sent to: timtimminz at gmail.com <timtimminz at gmail.com>
> To unsubscribe, send a blank message to trilug-leave at trilug.org from that address.
> TriLUG mailing list : https://www.trilug.org/mailman/listinfo/trilug
> Unsubscribe or edit options on the web  : https://www.trilug.org/mailman/options/trilug/timtimminz%40gmail.com
> Welcome to TriLUG: https://trilug.org/welcome