From: "kou (Kouhei Sutou) via ruby-core" Date: 2026-06-08T02:40:26+00:00 Subject: [ruby-core:125638] [Ruby Feature#19315] Lazy substrings in CRuby Issue #19315 has been updated by kou (Kouhei Sutou). Dan0042 (Daniel DeLorme) wrote in #note-29: > > `RSTRING_CSTR()` doesn't check whether `\0` is included in the target content or not. > > That is a very good point. In that case, perhaps the new NUL-terminated macro should instead alias or behave similarly to `StringValueCStr()`. Otherwise, we risk allowing a NUL-terminated string that contains embedded NUL bytes in the middle. I struggle to see a valid use case for that scenario; it feels like an invitation for bugs and undefined behavior. I think that we don't need a new macro for the `CStr` behavior. We can just use existing `StringValueCStr()`. > > I think `RSTRING_END()` must now guarantee null-termination in the same way as `RSTRING_PTR()`. Otherwise, they could point to different buffers after unsharing. > > I would like to point out that this introduces the exact opposite problem for raw pointers: if someone uses `RSTRING_RAW_PTR` alongside `RSTRING_END`, they could end up pointing to different buffers. To maintain consistency, would we then need to introduce `RSTRING_RAW_END` as well? Correct. The proposed PR includes a note about the case: > `@warning` The returned pointer is invalidated if the string is > subsequently unshared. Operations such as RSTRING_PTR() > and RSTRING_END() may unshare the string and reallocate > its buffer, making this pointer dangling. Only call this > function after all unsharing operations on `str` have > completed, or when `str` is known to be independent. To > obtain the end pointer without risking invalidation, use > `RSTRING_RAW_PTR(str) + RSTRING_LEN(str)`. I think that we don't need to introduce `RSTRING_RAW_END()` or something. We can just recommend `RSTRING_RAW_PTR() + RSTRING_LEN()` instead of `RSTRING_END()`. ---------------------------------------- Feature #19315: Lazy substrings in CRuby https://bugs.ruby-lang.org/issues/19315#change-117487 * Author: Eregon (Benoit Daloze) * Status: Open ---------------------------------------- CRuby should implement lazy substrings, i.e., "abcdef"[1..3] must not copy bytes. Currently CRuby only reuse the char* if the substring is until the end of the buffer. But it should also work wherever the substring starts and ends. Yes, it means RSTRING_PTR() might need to allocate to \0-terminate, so be it, it's worth it. There is already code for this (`SHARABLE_MIDDLE_SUBSTRING`), but it's disabled by default and `RSTRING_PTR()` needs to be changed to deal with this. It seems a good idea to introduce a variant of `RSTRING_PTR` which doesn't guarantee \0-termination, so such callers can then use the existing bytes always without copy. There are countless workarounds for this missing optimization, all not worth it with lazy substring and all less readable: * https://bugs.ruby-lang.org/issues/19314 * https://bugs.ruby-lang.org/issues/18598#note-3 * https://github.com/ruby/net-protocol/pull/14 * Manual lazy substrings which track string + index + length * More but I don't remember all now, feel free to comment or link more urls/tickets. -- https://bugs.ruby-lang.org/ ______________________________________________ ruby-core mailing list -- ruby-core@ml.ruby-lang.org To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/