Re: Windows Compilation Madness

The biggest issue is that although this has been discussed many times  
over, it's yet to be solved - in my mind, this means the discussion is  
still very much open. The OCI has been massively successful, and I am  
grateful, as I started there myself. It is not a real build though,  
it's a package set, and many of us are becoming increasingly dependent  
on a real build process for the whole stack. It's also important to  
note that this is far wider than a Ruby issue, this is about Open  
Source on Windows in general. (Anyone please note that this doesn't  
mean however, that it's not a ruby issue - someone needs to solve  
this, and who better than the ruby community for just getting it done,  
right?)

On 3 Jan 2008, at 14:55, Luis Lavena wrote:
> For me too, but every time I sent a message to get feedback, three
> things happen:
>
> A) don't get any reply (quite common)
> B) "move to a real OS" not so funny comments wasting part of my time
> invested (free) on ruby.
> C) another long thread like this with all the problems windows users
> face, the C-lib (MSVCRT) and all that, all over again.

Same here. Moreover, I think there's more than just you, me and Roger  
who've been solving this issue on our own in order to deal with the  
problems. There's someone around in #ruby-lang, I don't remember who,  
that's building everything on VS 2005 and 2008. That must be a  
nightmare for some apps. Maybe not so bad for the 'normal' rails stacks.

> No matter if I raise this discussion here (ruby-talk) or ruby-core,
> almost every time get the same "feedback".

Well, from my research actually outside of the ruby world, it seems  
realistically that a lot of people aren't certain about what issues  
will arise with these MSVCR differences. The MSDN docs are pretty  
clear on what *can* cause problems, and indeed some software does  
follow what I personally consider evil implementations that cause  
issues - but more importantly, most people can't tell you what inside  
the stdlib(s), will be an issue. Partly because some of this stdlib is  
closed source, and monitoring external calls alone, doesn't cut it, so  
it's not even easy to trace with a tool like IDA. Some of the  
PostgreSQL team actually know a lot more, but it's been a long time  
since I've been anywhere near them.

By empirical evidence from digging around, I find it's also common  
that people stitch together partial solutions to toolchain problems  
all too often in this environment. Indeed most people using MinGW do  
just that - I can say this with some confidence as no one has ever  
addressed build procedures and sub shell paths before - where as I  
have seen more than one instance of people running the makefiles  
individually, after the main ruby make.

Anyway, I'm not here to insult people, I'm here with a goal of moving  
toward doing things properly. In my opinion, that's building full  
stacks from a single build chain, in a single standard toolset, with  
minimal commands. Ideally without braking platform standards or adding  
external patches, unless they've been forwarded and accepted upstream.  
I think most of us agree on this in principle.

Moving on...

> I had more success downloading sources and compile by hand than using
> mingwPORT. After all, there is no way to automate mingwPORT execution.

For some builds yes, I've had a few fail badly on me, and some are  
more complex (e.g. the cyclic dependancies i.e. gettext + iconv IIRC)

I have had some of the ports building as part of a build script, but  
it's complex and environment setup and subshell problems are a real  
pain. My dollar for a truly working 'export'.

> Is funny you mention this James, it seems you made some progress since
> you contacted me back in november.


Yes, with varying levels of automation. I have 3 sets of scripts  
currently, in several flavours supporting building in COMSPEC, MSYS,  
and Rake independently, with varying levels of success. I've also been  
developing off of E:\, and this has issues in some places that I still  
need to produce and submit patches for. Lack of time has prevented me  
from producing solid documentation to date, although I also have some  
draft blog posts coming along too, that really would be better in a  
wiki. I wonder if any of the implementors have any opinions on this,  
wrt strategic documentation positioning. Ofc the JRuby team don't  
quite have the same issue, but they may have some advice.


> Too bad we are overlapping each other on this. I though we can
> collaborate since both aimed the same goal, but it seems isn't.

Well, we should. There's no reason why a single build chain can't  
solve all our issues, if we help with patches and sensible designs.  
I'm getting relatively close, we'll compare notes ASAP.

> Roger Pack suggested me to bundle MinGW in a gem...:
>
> 1) a 8MB gem (!!!)

Pre-built gems must account for a higher percentage of bandwidth on  
rubyforge, I mean the OCI is pretty big as is. 8mb isn't too bad by  
comparison, but, I'm not sure this is the right solution for many  
people.

> 2) because mingw is inside the gem, will not be easy hook it into PATH

Still trying to decide about PATH. Environment preparation is easy  
under the current methodology of launching ruby etc under Windows, as  
before, I'm looking into rubygems in more detail wrt this problem.  
Possibly even some of the rubigen stuff would help, but I've gotta  
spend some time researching my ideas. There's also the idea of  
emulating execve(7?), which is yet another branch of possiblity that  
I've got stabs for. (albeit cheating by adding false functionality to  
a pure-ruby 'env' that runs all extension-less files).

> 3) that didn't solve the rbconfig issues.

Falsifying a build is an odd thing, again rubigen type generation  
might make some sense, who knows. Maybe we want to look more closely  
at the build stack, or maybe we can deal with the problems purely with  
upstream patches. I suspect all of the above might eventually be  
required in some ways.

Personally, I think the important first stage is to release an  
automated build-chain build-chain. Once we can automate the whole  
stack, we can release it at several independent levels of compilation.

On 3 Jan 2008, at 15:22, Joel VanderWerf wrote:
> That would be great, if it can be made to work. I have ruby programs  
> that require C compilation to work (code is generated based on user  
> input in a ruby-based DSL). The hard part is setting up the compiler  
> on the user's computer.


And this is why... :-)

We're also more likely to get help from individuals if the entire  
build chain setup procedure can be replicated in more environments  
than our development machines. One of the big issues with C/C++ build  
chains is the 'speciality' of a developers machine. This happens oh so  
frequently, even in a professional project, and it makes me angry  
because it's whoever not documenting some dependancy, or not being  
aware of what their using. Sure, it's not easy, but it's our job!

---
On 3 Jan 2008, at 15:30, Luis Lavena wrote:
> Still, Ruby with with VC6 need VC6 to compile, or a hacked rbcofnig
> file, which need tweaks "per installation", since noone install ruby
> in the same path, drive, etc.

Right. I don't think this is the way to go. The OCI and garbage  
collect builds should remain as the authors intend them. This will be  
the standard for a long time yet, and to impose would make life worse  
I think.

The build scripts can be fixed externally, and I don't mean patching  
something installed locally. We should provide full-stack builds for  
people to use, and provide options for distribution packaging - it's  
not as large or insurmountable as it may feel to some.

A 7z self extracting exe can package up ruby and the stdlib into a 4mb  
exe. This is another little experimental branch I've been playing with  
- originally built to provide a few random tools like a standalone  
port forwarder internally in our environments when it was required for  
a telco, but it turns out to be an interesting stab against a couple  
of installer / distribution ideas. It actually turned out to be as  
fast, and as tidy as rubyscript2exe, for what that pattern is worth.

Anyway I'm in danger of digressing, and I have other things to do  
today... :-)

When I am back in my home country and have access to my build chain,  
we should have a more solid discussion than we did some months ago,  
probably off-list. :-)

Anyone who wishes to join or even just state an opinion or desire,  
PLEASE come and join us. I'll set something up as a central place, or  
hopefully someone else might, and announce it here.

>
> --
> Luis Lavena
>