From: Tony Arcieri <tony.arcieri@...>
Date: 2012-07-09T02:58:50+09:00
Subject: Re: Green threads in 1.9.* ?

--000e0cdfcf64c638aa04c4553fca
Content-Type: text/plain; charset=ISO-8859-1

On Sun, Jul 8, 2012 at 2:19 AM, rex goxman <lists@ruby-forum.com> wrote:

> I am reading Erlang documentation right now which which states that an
> Erlang process doesn't amount to much more than a pointer within the
> Erlang VM pointing off to some chunk of code/memory inside the Erlang
> VM, which explains why the processes are so lightweight.
>

Erlang processes have separate heaps and are independently garbage
collected, however within the SMP scheduler they share the same address
space across cores. While they have an affinity for particular schedulers,
since they operate in a shared address space any scheduler can potentially
execute them.


> Erlang didn't change the processes.  It didn't change the threading.  It
> added a scheduler and the infrastructure to allow the green processes
> living within the VM to be scheduled across VMs running on other cores.
> Erlang's own docs recommend running only one VM in a single thread per
> core, because it says you aren't going to get any speedup if you add
> more threads and more VMs per core.
>

You're colluding the word "virtual machine" with schedulers running within
the virtual machine. If I launch BEAM for example:

$ erl
Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:8:8] [rq:8]
[async-threads:0] [hipe] [kernel-poll:false]

This is a single instance of the Erlang virtual machine. However you will
note smp:8 and rq:8. This is because this single instance of the Erlang
virtual machine is running 8 scheduler threads, one for each core of my
computer:

Eshell V5.8.5  (abort with ^G)
1> erlang:system_info(schedulers).
8


> That's entirely incorrect, because you could have one thread running on
> each core, with a single GREEN-THREADED VM running on each of those
> threads (x cores, x threads total, x VMs running total).  This is what
> Erlang does now.  The only thing different now than in the past is that
> the VMs will communicate back and forth and schedule their GREEN THREADS
> across each other.


Again, there's only one instance of BEAM, i.e. there is only one virtual
machine, however there are 8 schedulers running inside it. These schedulers
each have their own thread, however they share an address space and contend
on things like memory allocation (again, because there's only one virtual
machine with a single pool of resources that much be shared among
schedulers).

In the past, when Erlang really did use green threads, it was necessary to
run a single Erlang VM per CPU core and use distributed Erlang (even in a
single system) to get multicore parallelism. However, now that Erlang has
an SMP scheduler, one Erlang VM can run a scheduler thread per CPU core and
affect multicore parallelism that way.

-- 
Tony Arcieri

--000e0cdfcf64c638aa04c4553fca
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div class=3D"gmail_quote">On Sun, Jul 8, 2012 at 2:19 AM, rex goxman <span=
 dir=3D"ltr">&lt;<a href=3D"mailto:lists@ruby-forum.com" target=3D"_blank">=
lists@ruby-forum.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_qu=
ote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex=
">


I am reading Erlang documentation right now which which states that an<br>
Erlang process doesn&#39;t amount to much more than a pointer within the<br=
>
Erlang VM pointing off to some chunk of code/memory inside the Erlang<br>
VM, which explains why the processes are so lightweight.<br></blockquote><d=
iv><br></div><div>Erlang processes have separate heaps and are independentl=
y garbage collected, however within the SMP scheduler they share the same a=
ddress space across cores. While they have an affinity for particular sched=
ulers, since they operate in a shared address space any scheduler can poten=
tially execute them.</div>


<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex">Erlang didn&#39;t change the p=
rocesses. =A0It didn&#39;t change the threading. =A0It<br>
added a scheduler and the infrastructure to allow the green processes<br>
living within the VM to be scheduled across VMs running on other cores.<br>
Erlang&#39;s own docs recommend running only one VM in a single thread per<=
br>
core, because it says you aren&#39;t going to get any speedup if you add<br=
>
more threads and more VMs per core.<br></blockquote><div><br></div><div>You=
&#39;re colluding the word &quot;virtual machine&quot; with schedulers runn=
ing within the virtual machine. If I launch BEAM for example:</div><div>


<br></div><div><div>$ erl</div><div>Erlang R14B04 (erts-5.8.5) [source] [64=
-bit] [smp:8:8] [rq:8] [async-threads:0] [hipe] [kernel-poll:false]</div><d=
iv>=A0</div></div><div>This is a single instance of the Erlang virtual mach=
ine. However you will note smp:8 and rq:8. This is because this single inst=
ance of the Erlang virtual machine is running 8 scheduler threads, one for =
each core of my computer:</div>


<div><br></div><div><div>Eshell V5.8.5 =A0(abort with ^G)</div><div>1&gt; e=
rlang:system_info(schedulers).</div><div>8</div></div><div>=A0</div><blockq=
uote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc =
solid;padding-left:1ex">



<div>That&#39;s entirely incorrect, because you could have one thread runni=
ng on</div>
each core, with a single GREEN-THREADED VM running on each of those<br>
threads (x cores, x threads total, x VMs running total). =A0This is what<br=
>
Erlang does now. =A0The only thing different now than in the past is that<b=
r>
the VMs will communicate back and forth and schedule their GREEN THREADS<br=
>
across each other.</blockquote><div><br></div><div>Again, there&#39;s only =
one instance of BEAM, i.e. there is only one virtual machine, however there=
 are 8 schedulers running inside it. These schedulers each have their own t=
hread, however they share an address space and contend on things like memor=
y allocation (again, because there&#39;s only one virtual machine with a si=
ngle pool of resources that much be shared among schedulers).</div>


<div><br></div><div>In the past, when Erlang really did use green threads, =
it was necessary to run a single Erlang VM per CPU core and use distributed=
 Erlang (even in a single system) to get multicore parallelism. However, no=
w that Erlang has an SMP scheduler, one Erlang VM can run a scheduler threa=
d per CPU core and affect multicore parallelism that way.</div>

<div><br></div></div>-- <br>Tony Arcieri<br><br>

--000e0cdfcf64c638aa04c4553fca--