[ruby-core:116613] [Ruby master Misc#20240] Unable to build ruby 3.1.0 on macOS when shared due to dylibs (libgmp) not found when running miniruby
From:
"jmarrec (Julien Marrec) via ruby-core" <ruby-core@...>
Date:
2024-02-06 23:48:00 UTC
List:
ruby-core #116613
Issue #20240 has been updated by jmarrec (Julien Marrec).
Thank you for your reponse!
Ideally I'd like to support it.
I've tried it, I get similar issues for some of the other dylibs, during make install. Psych is built correctly, just can't be installed. Zlib looks to be the same issue.
``` shell
ruby/3.1.0: Calling package()
ruby/3.1.0: RUN: make install DESTDIR=/Users/julien/.conan2/p/b/rubyc85244d60c2fd/p -j10
Downloading bundled gem rbs-3.1.0...
Downloading bundled gem debug-1.6.3...
BASERUBY = /usr/bin/ruby --disable=gems
CC = clang
LD = clang
LDSHARED = clang -dynamiclib
CFLAGS = -fPIC -O3 -fPIC -O3 -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wdivision-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wextra-tokens -Wundef -std=gnu99 -fno-common -pipe
XCFLAGS = -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -fvisibility=hidden -DRUBY_EXPORT -I. -I.ext/include/arm64-darwin23 -I/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/include -I/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src -I/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/enc/unicode/13.0.0
CPPFLAGS = -DNDEBUG -DNDEBUG -I/Users/julien/.conan2/p/b/zlib1f8e7d96319f0/p/include -I/Users/julien/.conan2/p/b/opense854e464e8ff6/p/include -I/Users/julien/.conan2/p/b/libff05fe9d5b96f79/p/include -I/Users/julien/.conan2/p/b/libyae2f0aa15c9e92/p/include -I/Users/julien/.conan2/p/b/readl0d0041a63fa03/p/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT
DLDFLAGS = -Wl,-undefined,dynamic_lookup -L/Users/julien/.conan2/p/b/zlib1f8e7d96319f0/p/lib -L/Users/julien/.conan2/p/b/opense854e464e8ff6/p/lib -L/Users/julien/.conan2/p/b/libff05fe9d5b96f79/p/lib -L/Users/julien/.conan2/p/b/libyae2f0aa15c9e92/p/lib -L/Users/julien/.conan2/p/b/readl0d0041a63fa03/p/lib -install_name @executable_path/../lib/libruby.3.1.dylib -compatibility_version 3.1 -current_version 3.1.0 -fstack-protector-strong -framework CoreFoundation -fstack-protector-strong -framework CoreFoundation
SOLIBS = -lpthread -ldl -lobjc
LANG = en_US.UTF-8
LC_ALL = en_US.UTF-8
LC_CTYPE =
MFLAGS = - --jobserver-fds=4,5 -j
Apple clang version 15.0.0 (clang-1500.1.0.2.5)
Target: arm64-apple-darwin23.2.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
generating arm64-darwin23-fake.rb
generating enc.mk
arm64-darwin23-fake.rb updated
making srcs under enc
making enc
make[1]: Nothing to be done for `enc'.
make[1]: Nothing to be done for `srcs'.
generating transdb.h
Extracting bundle gem debug-1.6.3...
transdb.h unchanged
generating makefiles ext/configure-ext.mk
making trans
make[1]: Nothing to be done for `/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/enc/trans'.
making encs
ext/configure-ext.mk updated
Extracting bundle gem rbs-3.1.0...
make[1]: Nothing to be done for `encs'.
Unpacked gems/debug-1.6.3.gem
generating makefile exts.mk
exts.mk unchanged
Unpacked gems/rbs-3.1.0.gem
/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/revision.h unchanged
make[1]: Nothing to be done for `note'.
./miniruby -I/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib -I. -I.ext/common /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/runruby.rb --extout=.ext -- --disable-gems -r./arm64-darwin23-fake /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb --make="/Library/Developer/CommandLineTools/usr/bin/make" --dest-dir="/Users/julien/.conan2/p/b/rubyc85244d60c2fd/p" --extout=".ext" --ext-build-dir="./ext" --mflags="- --jobserver-fds=4,5 -j" --make-flags=" --jobserver-fds=4,5 -j -- DESTDIR=/Users/julien/.conan2/p/b/rubyc85244d60c2fd/p" --data-mode=0644 --prog-mode=0755 --installed-list .installed.list --mantype="doc" --gnumake --install=all --exclude=doc
installing binary commands: //bin
installing base libraries: //lib
installing arch files: //lib/ruby/3.1.0/arm64-darwin23
installing pkgconfig data: //lib/pkgconfig
installing extension objects: //lib/ruby/3.1.0/arm64-darwin23
installing extension objects: //lib/ruby/site_ruby/3.1.0/arm64-darwin23
installing extension objects: //lib/ruby/vendor_ruby/3.1.0/arm64-darwin23
installing extension headers: //include/ruby-3.1.0/arm64-darwin23
installing extension scripts: //lib/ruby/3.1.0
installing extension scripts: //lib/ruby/site_ruby/3.1.0
installing extension scripts: //lib/ruby/vendor_ruby/3.1.0
installing extension headers: //include/ruby-3.1.0/ruby
installing command scripts: //bin
installing library scripts: //lib/ruby/3.1.0
installing common headers: //include/ruby-3.1.0
installing manpages: //share/man/man1
installing default gems from lib: //lib/ruby/gems/3.1.0
abbrev 0.1.0
base64 0.1.1
benchmark 0.2.0
bundler 2.3.3
/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/yaml.rb:3: warning: It seems your ruby installation is missing psych (for YAML output).
To eliminate this warning, please install libyaml and reinstall your ruby.
/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/installer.rb:343:in `rescue in install': uninitialized constant Gem::Installer::Zlib (NameError)
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/installer.rb:293:in `install'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:899:in `block in install'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:713:in `no_write'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:899:in `install'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:1002:in `block in install_default_gem'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:989:in `each'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:989:in `install_default_gem'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:927:in `block in <main>'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:1127:in `block in <main>'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:1124:in `each'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:1124:in `<main>'
<internal:/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/core_ext/kernel_require.rb>:85:in `require': dlopen(/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/build-release/.ext/arm64-darwin23/psych.bundle, 0x0009): Library not loaded: @rpath/libyaml.dylib (LoadError)
Referenced from: <C2079A68-F56A-3359-8762-39D3F1940B29> /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/build-release/.ext/arm64-darwin23/psych.bundle
Reason: tried: '/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/build-release/libyaml.dylib' (no such file), '/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/build-release/libyaml.dylib' (no such file) - /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/build-release/.ext/arm64-darwin23/psych.bundle
from <internal:/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/core_ext/kernel_require.rb>:85:in `require'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/build-release/.ext/common/psych.rb:13:in `<top (required)>'
from <internal:/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/core_ext/kernel_require.rb>:85:in `require'
from <internal:/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/core_ext/kernel_require.rb>:85:in `require'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/yaml.rb:4:in `<top (required)>'
from <internal:/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/core_ext/kernel_require.rb>:85:in `require'
from <internal:/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/core_ext/kernel_require.rb>:85:in `require'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems.rb:619:in `load_yaml'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/config_file.rb:346:in `load_file'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/config_file.rb:189:in `initialize'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems.rb:331:in `new'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems.rb:331:in `configuration'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/user_interaction.rb:167:in `verbose'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/installer.rb:544:in `generate_bin_script'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:905:in `generate_bin_script'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:873:in `generate_bin_script'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/installer.rb:506:in `block in generate_bin'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/installer.rb:482:in `each'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/installer.rb:482:in `generate_bin'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/installer.rb:323:in `install'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:899:in `block in install'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:713:in `no_write'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:899:in `install'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:1002:in `block in install_default_gem'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:989:in `each'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:989:in `install_default_gem'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:927:in `block in <main>'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:1127:in `block in <main>'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:1124:in `each'
from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:1124:in `<main>'
make: *** [do-install-nodoc] Error 1
```
----------------------------------------
Misc #20240: Unable to build ruby 3.1.0 on macOS when shared due to dylibs (libgmp) not found when running miniruby
https://bugs.ruby-lang.org/issues/20240#change-106624
* Author: jmarrec (Julien Marrec)
* Status: Open
* Priority: Normal
----------------------------------------
I am trying to develop a conan (the C/C++ package manager) recipe for Ruby. The recipe would allow downstream users to 1) get a runnable ruby executable, and 2) be able to link to ruby, or embbed it in a C/C++ program if built statically, in an easy way.
Currently there is an existing ruby 3.1.0 recipe that I'm trying to adapt, so I have to support this version.
First off, let me say that I can succesfully build with 3.3.0, so I know something has changed for the better since then. I'm just at a lost when figuring out what I need to backport to make 3.1.0 work.
The original issue is that it appears miniruby is looking for some dylibs and not finding them. Even if I do define `LD_LIBRARY_PATH`, `DYLD_LIBRARY_PATH` or `DYLD_FALLBACK_LIBRARY_PATH` (any combinations of these three) in my env.
``` shell
dsymutil exe/ruby; { test -z '' || codesign -s '' -f exe/ruby; }
./miniruby \
-e 'prog, dest, inst = ARGV; dest += "/ruby"' \
-e 'exit unless prog==inst' \
-e 'unless prog=="ruby"' \
-e ' begin File.unlink(dest); rescue Errno::ENOENT; end' \
-e ' File.symlink(prog, dest)' \
-e 'end' \
ruby exe ruby
dyld[59344]: Library not loaded: @rpath/libgmp.10.dylib
Referenced from: <356E0011-6223-321A-9179-D55618D248D0> /Users/julien/.conan2/p/b/ruby9cafa28a7060d/b/build-release/miniruby
Reason: no LC_RPATH's found
make: *** [exe/ruby] Abort trap: 6
make: *** Deleting file `exe/ruby'
```
It seems that something is unsetting the variables, because this for eg works fine
```shell
DYLD_LIBRARY_PATH=/Users/julien/.conan2/p/b/zlib1f8e7d96319f0/p/lib:/Users/julien/.conan2/p/b/opense854e464e8ff6/p/lib:/Users/julien/.conan2/p/b/libyae2f0aa15c9e92/p/lib:/Users/julien/.conan2/p/b/libff05fe9d5b96f79/p/lib:/Users/julien/.conan2/p/b/readl0d0041a63fa03/p/lib:/Users/julien/.conan2/p/b/termc22b5bb1515971/p/lib:/Users/julien/.conan2/p/b/gmp676fa41eaa3d6/p/lib: /Users/julien/.conan2/p/b/ruby9cafa28a7060d/b/build-release/miniruby -e "puts 'Hello, world'"
```
My configure call is like this:
```shell
./configure --enable-shared --disable-static --prefix=/ '--bindir=${prefix}/bin' '--sbindir=${prefix}/bin' '--libdir=${prefix}/lib' '--includedir=${prefix}/include' '--oldincludedir=${prefix}/include' --disable-install-doc --enable-load-relative --with-zlib-dir=/Users/julien/.conan2/p/b/zlib1f8e7d96319f0/p --with-openssl-dir=/Users/julien/.conan2/p/b/opense854e464e8ff6/p --with-libffi-dir=/Users/julien/.conan2/p/b/libff05fe9d5b96f79/p --with-libyaml-dir=/Users/julien/.conan2/p/b/libyae2f0aa15c9e92/p --with-readline-dir=/Users/julien/.conan2/p/b/readl0d0041a63fa03/p --with-gmp-dir=/Users/julien/.conan2/p/b/gmp676fa41eaa3d6/p --with-opt-dir=/Users/julien/.conan2/p/b/opense854e464e8ff6/p:/Users/julien/.conan2/p/b/libff05fe9d5b96f79/p:/Users/julien/.conan2/p/b/libyae2f0aa15c9e92/p:/Users/julien/.conan2/p/b/readl0d0041a63fa03/p:/Users/julien/.conan2/p/b/gmp676fa41eaa3d6/p --disable-jit-support
```
I have tried to backport https://github.com/ruby/ruby/pull/6296/files and https://github.com/ruby/ruby/commit/48644e71096c70132be9dfdcbfb414ec2e68d18b and https://github.com/ruby/ruby/pull/8730 amongst other things but I can't make it work. (I even tried a more brute force approach patching a lot of files by diffing 3.3.0 with 3.1.0, but please note I don't know what I'm doing... and I can get to the install step but then I get some errors about Psych / libymal and undefined Gem::Install:Zlib).
I would **greatly** appreciate if someone can spare some time to help me wrap this up (I've been trying to make the recipe work for so long that I'm about to give up...)
--
https://bugs.ruby-lang.org/
______________________________________________
ruby-core mailing list -- ruby-core@ml.ruby-lang.org
To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/