[ruby-core:118214] [Ruby master Misc#20488] Document source file size restrictions

Issue #20488 has been updated by kddnewton (Kevin Newton).

Status changed from Open to Closed

Thank you!

----------------------------------------
Misc #20488: Document source file size restrictions
https://bugs.ruby-lang.org/issues/20488#change-108703

* Author: kddnewton (Kevin Newton)
* Status: Closed
----------------------------------------
I was hoping we might be able to decide on an official maximum size for a s=
ource file in terms of bytes/lines/columns/etc.

Ruby uses fixed integer sizes to represent line numbers, column numbers, of=
fsets, etc. These can overflow with files that are too big. Sometimes they =
will fail with cryptic messages like:

```
ruby: negative string size (or size too big) (ArgumentError)
```

(That's for 2^31 "a"s in a row.) Note that for python for the same file you=
 will get:

```
OverflowError: Parser column offset overflow - source line is too big
```

and for perl you will get:

```
Identifier too long at test.pl line 1.
```

For files with 2^32 newlines, Ruby just crashes on my machine with the curr=
ent parser (prism finishes, but its newline counter overflows so it gets al=
l the offsets wrong).

Would it be okay to say:

* maximum line: 31 bits
* maximum column: 32 bits
* maximum file byte size: 32 bits

This would also help with memory savings =E2=80=94 I would like to only use=
 32 bits for offsets in the file, as opposed to the current 64 bits I'm usi=
ng (which seems unnecessarily large).



--=20
https://bugs.ruby-lang.org/