Comment by mlyle

Comment by mlyle 6 months ago

> you're trusting software developers either way, whether it be at the app level, the language/runtime level, or the operating system level.

I trust systems to do better based on observed behavior rather than a software engineer's guess of how it will be scheduled. Who knows if, in a given use case, the program is a "small" part of the system or a "large" part that should get preferential placement and scheduling.

> If a threadpool manager is hinted that 4 threads are going to share a lot of memory, they can be allocated on the same l2 cache.

And so this is kind of a weird thing: we know we're going to be performance critical and we need things to be forced to be adjacent... but we don't know the exact details of the hardware we're running on. (Else, just numa_bind and be done...)

Salgat 6 months ago

The beauty is that you don't care what hardware you run on, all you're annotating are very useful but generic properties such as which threads are sharing a lot of memory, or perhaps that a thread should have highest performance priority so that internally it stays on p cores instead of the more scalable e cores. Very simple optional hints.

Reply View 9 replies

mlyle 6 months ago

> should have highest performance priority so that internally it stays on p cores
Everything will decide that it wants P cores; it's not punished for battery or energy impact, and wants to win over other applications for users to have a better experience with it.
And even if not made in bad faith, it doesn't know what else is running on the system.
Also these decisions tend to be unduly influenced by microbenchmarks and then don't apply to the real system.
> which threads are sharing a lot of memory
But if they're not super active, should the scheduler really change what it's doing? And doesn't the size of that L2 matter? It doesn't matter if e.g. the stuff is going to get churned out before there's a benefit from that sharing.
In the end, if you don't know pretty specific details of the environment you'll run on: what the hardware is like, what loading is like, what data set size is like, and what else will be running on the machine -- it is probably better to leave this decision to the scheduler.
If you do know all those things, and it's worth tuning this stuff in depth-- odds are you're HPC and you know what the machine is like.

Reply View | 8 replies
- Salgat 6 months ago
  
  To clarify, what gets scheduled is up to the OS or runtime, all you're doing is setting relative priority. If everything is all the same priority, then it's just as likely to all run on e cores.
  
  Reply View | 7 replies
  
  mlyle 6 months ago
  
  And then, what's the point?
  A system that encourage everyone to jack everything up is pointless.
  A system to tell the OS that the developer anticipates that data is shared and super hot will be mostly lied to (on accident or purpose).
  There's the edge cases: database servers, HPC, etc, where you believe that the system has a sole occupant that can predict loading.
  But libnuma, and the underlying ACPI SRAT/SLIT/HMAT tables are a pretty good fit for these use cases.
  
  Reply View | 6 replies