From: Joseph Jones <joeyi5216@...>
Date: 2015-12-17T21:13:00-07:00
Subject: [ruby-core:72327] [Ruby trunk - Feature #11817] [Open] map.parallel

--567387cc_7f01579b_16c
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Joseph Jones liked your message with Boxer. On December 14, 2015 at 15:53=
:42 MST, eike=40inter.net wrote:Issue =2311817 has been reported by Eike =
Dierks.----------------------------------------=46eature =2311817: map.pa=
rallelhttps://bugs.ruby-lang.org/issues/11817* Author: Eike Dierks* Statu=
s: Open* Priority: Normal* Assignee: ------------------------------------=
----I'd like to suggest a new method: =23parallelThat method should take =
care of executing workloads in parallel across multiple cores.I believe t=
hat =23map can sometimes be distributed to many simultaneous threads.So w=
e would spread out the map blocks into parallel processing,and then colle=
ct the results in order.I believe this would need some new infrastructure=
 like apple did with great=5Fcentral,so I'm talking along these lines.(Ak=
a we only want to open up as many threads as there are cores)This would n=
ot help much with a simple map (because we need to collect and serialize =
the results in the end)but it could be cool for more complex mappings.May=
be this should have been a rails topic first,because there it might make =
a lot of sense to render partials in parallel,this could really speed up =
rails response rendering ;-)But I believe this is a topic for ruby in gen=
eral.Given that we now go to really many cores,it should be at the heart =
of ruby to use that.=46or a start we could just declare =23parallel as ->=
 return self.and than improve on it.A naive implementation would just ope=
n a new thread for every parallel map block,but that would not work well =
-- it actually would be very slow.So we actually need a thread management=
,along the lines of great central.I believe this should be built into the=
 very core libs of the ruby language.I know this is hard stuff, but it ne=
eds to be done.Let's go parallel from the heart of the language.---Please=
 let us discuss if adding the =23parallel method would be a good idea.Let=
's come up with some implementations to find the problems.I'm explicitly =
referring to map=23parallel here.In rails that might be cache.parallel (i=
f you followed)for some math tasks it might also be parallel do =7B=7D-- =
https://bugs.ruby-lang.org/     
--567387cc_7f01579b_16c
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<html><body><div>Joseph Jones liked your message with <a href=3D=22http:/=
/bxr.io/PBIqo=22>Boxer</a>.</div><br/><br/><div><div class=3D=22quote=22>=
On December 14, 2015 at 15:53:42 MST, eike=40inter.net wrote:<br/><blockq=
uote type=3D=22cite=22 style=3D=22border-left-style:solid;border-width:1p=
x;margin-left:0px;padding-left:10px;=22>Issue =2311817 has been reported =
by Eike Dierks.<br /><br />----------------------------------------<br />=
=46eature =2311817: map.parallel<br />https://bugs.ruby-lang.org/issues/1=
1817<br /><br />* Author: Eike Dierks<br />* Status: Open<br />* Priority=
: Normal<br />* Assignee: <br />----------------------------------------<=
br />I'd like to suggest a new method: =23parallel<br /><br />That method=
 should take care of executing workloads in parallel across multiple core=
s.<br /><br />I believe that =23map can sometimes be distributed to many =
simultaneous threads.<br /><br />So we would spread out the map blocks in=
to parallel processing,<br />and then collect the results in order.<br />=
<br />I believe this would need some new infrastructure like apple did wi=
th great=5Fcentral,<br />so I'm talking along these lines.<br />(Aka we o=
nly want to open up as many threads as there are cores)<br /><br />This w=
ould not help much with a simple map (because we need to collect and seri=
alize the results in the end)<br />but it could be cool for more complex =
mappings.<br /><br />Maybe this should have been a rails topic first,<br =
/>because there it might make a lot of sense to render partials in parall=
el,<br />this could really speed up rails response rendering ;-)<br /><br=
 />But I believe this is a topic for ruby in general.<br />Given that we =
now go to really many cores,<br />it should be at the heart of ruby to us=
e that.<br /><br />=46or a start we could just declare =23parallel as -> =
return self.<br />and than improve on it.<br /><br />A naive implementati=
on would just open a new thread for every parallel map block,<br />but th=
at would not work well -- it actually would be very slow.<br /><br />So w=
e actually need a thread management,<br />along the lines of great centra=
l.<br /><br />I believe this should be built into the very core libs of t=
he ruby language.<br /><br />I know this is hard stuff, but it needs to b=
e done.<br />Let's go parallel from the heart of the language.<br /><br /=
>---<br /><br /><br />Please let us discuss if adding the =23parallel met=
hod would be a good idea.<br />Let's come up with some implementations to=
 find the problems.<br /><br />I'm explicitly referring to map=23parallel=
 here.<br />In rails that might be cache.parallel (if you followed)<br />=
<br />for some math tasks it might also be<br />   parallel do =7B=7D<br =
/><br /><br /><br /><br /><br /><br /><br /><br /><br /><br /><br /><br /=
><br /><br /><br /><br /><br /><br /><br /><br /><br /><br /><br /><br />=
<br /><br /><br /><br /><br /><br /><br /><br /><br /><br /><br />-- <br =
/>https://bugs.ruby-lang.org/<br /></blockquote></div></div></body></html=
>
--567387cc_7f01579b_16c--