From: "postmodern (Hal Brodigan)" Date: 2022-01-13T05:27:19+00:00 Subject: [ruby-core:107093] [Ruby master Feature#18481] Porting YJIT to Rust (request for feedback) Issue #18481 has been updated by postmodern (Hal Brodigan). Maintainer of ruby-install here along with @havenwood. By design ruby-install will install all build and runtime dependencies from the system's package manager needed to compile/run a ruby with default configuration options; which is how one would normally install dependencies when compiling software manually. The currently supported package managers are: * apt (Debian/Ubuntu) * yum (CentOS) * dnf (RHEL/Fedora, is replacing yum) * zypper (OpenSUSE) * pacman (Arch Linux) * brew (aka Homebrew) * port (aka MacPorts) * pkg (aka FreeBSD Ports) Note: OpenBSD/NetBSD's pkg_add is currently not supported. Note: ruby-install currently does not support tracking additional or optional dependencies, but this could be added; it has been requested in the past for compiling CRuby with jemalloc support. If CRuby does add rust (more specifically rustc and cargo) as build dependencies, I would only ask that a survey of the above mentioned package managers be done to verify that: 1. they provide packages for rustc and cargo. 2. those packages are relatively up-to-date. To make things easier, I would only worry about the currently maintained and stable releases of OSes/Linux Distros. Users still on older unmaintained OS/Distro versions should upgrade in order to take advantage of security updates. * Debian (stable): * https://packages.debian.org/bullseye/rustc * https://packages.debian.org/bullseye/cargo * Ubuntu (20.04 LTS) rust packages: * https://packages.ubuntu.com/focal/rustc * https://packages.ubuntu.com/focal/cargo * Fedora: * https://src.fedoraproject.org/rpms/rust * https://src.fedoraproject.org/rpms/cargo or https://src.fedoraproject.org/rpms/rust-cargo ? * OpenSUSE: * https://software.opensuse.org/package/rustc-1_9 * https://software.opensuse.org/package/cargo * Arch Linux: https://archlinux.org/packages/extra/x86_64/rust/ * Homebrew: https://formulae.brew.sh/formula/rust * MacPorts: https://ports.macports.org/port/rust/ * FreeBSD Ports: https://www.freshports.org/lang/rust A problem I have seen in the past with LLVM-based languages, is that they either target an extremely recent version of LLVM not yet provided by the various package managers, or they target a much older version of LLVM which contains known bugs/issues they must workaround, or they give up and simply download and build their own copy of LLVM. Targeting a very recent version of rust could make it difficult to build on systems who's package manager still has an older version of rust. Targeting a much older "stable" version of rust might also cause issues when the YJIT team runs into known bugs/issues. Downloading your own copy of rust (precompiled or building from source) may seem like the path of least resistance, however this can consume additional disk space, bandwidth, CPU, and time; and can lead to friction and grumbling among developers (ex: nokogiri compiles it's own copy of libxml2 each time you update it). While rust does recommend using their `rustup` script, which can be installed via some (but not all) package managers, I would defer to using the system's approved package that has been QAed by the package maintainers and is assured to work with the rest of the system; and make life easier for the ruby package maintainers. ---------------------------------------- Feature #18481: Porting YJIT to Rust (request for feedback) https://bugs.ruby-lang.org/issues/18481#change-95938 * Author: maximecb (Maxime Chevalier-Boisvert) * Status: Open * Priority: Normal ---------------------------------------- TL;DR: The YJIT team wants to explore using Rust to help develop YJIT. The rest of CRuby will continue to build without Rust tools and building YJIT will remain optional. We���re currently exploring the possibility of porting YJIT to Rust and working on a small proof of concept that should be ready next month. The motivation behind this is that we are facing challenges in terms of code maintainability. As you know, JIT compilers can get very complex, and C99 doesn't offer many tools to manage this complexity. There are no classes and methods, limited type checking, and it's hard to fully separate code into modules, for instance. We believe that having access to object oriented programming and a more expressive type system would help us manage growing complexity better and also improve the safety/robustness of YJIT. For instance we would like to add Windows support and a new backend to YJIT. That means we���ll have two separate backends (x86, arm64) and we���ll need to support two different calling conventions (Microsoft, SystemV), but currently, we have limited tools to build the abstractions needed, such as preprocessor macros and if-statements. We���ve discussed the idea of porting YJIT to Rust with some of the Ruby core developers (@ko1, @k0kubun, @mame), and it seems they would be open to merging something like this if it works well. I���m opening this ticket so that everyone can have a chance to provide feedback and participate in the discussion. We realize that adding Rust code to the CRuby codebase could be challenging and that there are open questions. We are planning to make it so that YJIT will only need the Rust compiler and `cargo` to build. Building YJIT would then require the Rust compiler to be installed, but CRuby could build without YJIT and without the Rust compiler. There would be no new dependencies for the compiled binary. Rust is supported on Mac, Windows, BSDs, and Linux, which covers all the platforms on which we plan to support YJIT. Since Rust is based on LLVM, it has good support for cross-compilation. We would like to solicit input from Ruby distributors who create `.deb` and `.rpm` packages. We will likely remain conservative when updating Rust versions to make OS packaging easier. We believe that in the general, the resulting code should be easier to maintain because it will be better organized, but the YJIT team will help out with YJIT-related backports and will be available to help if needed. Value proposition: - Rust type systems will catch more bugs early, help prevent new bugs - Easier to manage growing complexity of YJIT - Easier to maintain codebase, fewer ���footguns��� - Easier for newcomers because the compiler catches more bugs - Better performance because we can implement more sophisticated optimizations - Easier to add support for new platforms (which adds complexity) - Rust has mature and easy-to-install tools such as source code formatter and editor plugins - Rust as a programming language community has a great deal of enthusiasm behind it. This could translate to more enthusiasm for YJIT and for Ruby as well. Integration: - YJIT will only depend on the Rust language and the standard library, no other dependencies - YJIT will be able to build without an internet connection - Rust has good support for cross-compilation - Rust is supported on all platforms on which we plan to support with YJIT (Mac, Linux, Windows) - The compiled CRuby binary won���t have any new dependencies on shared libraries - CRuby will still be able to build without `rustc`, with YJIT disabled -- https://bugs.ruby-lang.org/ Unsubscribe: