[ruby-core:29586] [Bug #1169] Thread#priority still doesn't work

Issue #1169 has been updated by caleb clausen.


So, you've disabled the test which demonstrates the problem. You've documented that the desired behavior is now going to be optional. And now, you've closed the ticket which reported the problem. However:

          >>>the bug still exists<<<

I do not think this issue should have been closed. Can it be reopened, please? Open bug reports are the collective memory about where the problems in a system are. When you close a report without fixing the reported bug, it's like saying, "We're just going to forget that this problem was ever noticed." But it doesn't make the problem go away; it just makes it harder to know that it's there.

Should thread priorities be respected or not? If they ought to be respected, then it is better to have a note somewhere that keeps track of the fact that they may not be working. I say it's better to leave unresolved bug reports open than to let the bug tracker reflect a false state of affairs.  

I don't think you need access to the realtime scheduling algorithms (SCHED_RR, SCHED_FIFO) to get higher priority threads to receive more cputime. Normal (fair) scheduling (SCHED_OTHER) should be sufficient. After all, the syscalls nice(2), setpriority(2), and pthread_setschedparam(3) presumably work for non-realtime processes/threads.

However, I'm now starting to theorize that there's an interaction here between the scheduler and the GIL which causes priorities to be effectively ignored. Imagine this:

2 threads running, A and B; A.priority > B.priority
initially, A runs (so it holds GIL)
B is waiting on GIL 
after a while (10ms I think) scheduler forces a context switch
so now A unlocks GIL, then immediately relocks it (yielding time to other threads)
but before A can relock, B is scheduled, since it was waiting. Now B owns GIL.
after another 10ms, B yields time back to A
...and so on

So, A and B are effectively alternating timeslices, each getting roughly equal amounts of time, even tho A should have a higher priority. If the os uses priority queues rather than normal fifo queues for the internal queue inside a mutex, then is problem wouldn't occur. (Actually, you need to have more than 2 threads in your system for a priority queue to be helpful here, because of a race condition, but never mind that detail.) However, I think it's quite unlikely that typical desktop or server OSes use priority queues here; an RTOS might use them (I know vxworks has it as an option, or used to anyway) but priority queues require more cpu and memory and are at least tricky (maybe impossible) to implement right in a system with a fair scheduler. (Fair schedulers are actually pretty complicated compared to realtime schedulers.)

So far, this is all just a theory. I have no proof either for the overall theory or my contention that the queue used inside a mutex is (usually, on most systems) a fifo queue. 

If I'm right, then this might be fixable by making timeslices variable length, instead of always 10 ms. Lower priority threads would get shorter timeslices. 
----------------------------------------
http://redmine.ruby-lang.org/issues/show/1169

----------------------------------------
http://redmine.ruby-lang.org