[ruby-core:65156] [ruby-trunk - Feature #10267] Number of processors

Issue #10267 has been updated by Eric Wong.


 KOSAKI Motohiro <[e-mail]> wrote:
 > On Sat, Sep 20, 2014 at 1:03 PM, Eric Wong <[e-mail]> wrote:
 > > [e-mail] wrote:
 > >> matz accepted this feature including the method name,
 > >> Etc.nprocessors_online.
 > >
 > > I think the name is too long.  How about naming it after the GNU
 > > coreutils nproc(1) command?
 > >
 > >         Etc.nproc(:online)
 > 
 > This is an option. But "proc" is not unclear on ruby context. It is sometimes
 > procedure, and it is sometimes process. I like processor or cpu.
 
 OK.  I mainly want the ability to specify type as arg.
 
 > > I think it is important to be able to get all CPUs, not just
 > > online CPUs, too
 > 
 > > (to workaround old SMP bugs in the kernel):
 > 
 > What bug?
 
 Anything which may involve synchronizing with softirq or workqueues, I
 think...  I suppose my fix in 128dd1759d96ad36c379240f8b9463e8acfd37a1
 (missing barrier in EPOLL_CTL_MOD) is one example.  I was never able to
 reproduce that bug on my HW, but it worries me because some users are
 still on old kernels and potentially affected.
 
 > >         Etc.nproc(:all)
 > 
 > I don't think this is useful. I'd like to explain some kernel internal.
 > Now, Linux has three type of number of cpus.
 > 
 > 1) online
 > 2) present
 > 3) possible
 
 I think these names are fine for Ruby.
 
 > "all" is not clear to me.
 > 
 > If you elaborate why you need nproc(:all), I may be able to suggest better
 > name.
 
 I just chose "all" because it matched the nproc(1) --all option
 and didn't want to use Linux-specific naming.
 But online/present/possible is fine, for now, too.
 
 We could also have extra options like:
 
 :cores - exclude HyperThread  (probably useful for Intel users)
 :physical - number of sockets (maybe for affinity)

----------------------------------------
Feature #10267: Number of processors
https://bugs.ruby-lang.org/issues/10267#change-49001

* Author: Akira Tanaka
* Status: Open
* Priority: Normal
* Assignee: 
* Category: 
* Target version: 
----------------------------------------
How about a method to obtain number of processors?

Several committers discussed this feature at RubyKaigi 2014.

The number is important to use all available processing power.
Too few processes doesn't use all processors.
Too much processes wastes processing power.

For example, test/lib/test/unit has -j N option for
parallel tests and N is the number of processes to run simultaneously.
But we must specify N explicitly.
I think it should have reasonable default.

I propose a primitive method to obtain number of processors for such purpose:
Etc.nprocessors_online.

It is implemented using sysconf(_SC_NPROCESSORS_ONLN).
It is not standardized but available on many platforms: 
GNU/Linux, NetBSD, FreeBSD, OpenBSD, DragonFly BSD,
OpenIndiana, Mac OS X and AIX.

usa-san said he will implement on Windows (after this feature is implemented).

kosaki-san said he will improve the method for container on GNU/Linux.
(If a process is run in a container, usable number of processors may be restricted.
So sysconf(_SC_NPROCESSORS_ONLN) may be not approprate.)

matz accepted this feature including the method name,
Etc.nprocessors_online.

However kosaki-san said he will try to find better name
(by survey container technology).
So I'm waiting.


---Files--------------------------------
etc-nprocessors.patch (1.91 KB)


-- 
https://bugs.ruby-lang.org/