[ruby-core:118606] [Ruby master Bug#20030] `Ripper.tokenize('"\\C-あ"')` separates encoding valid string to encoding invalid string.
From:
"nagachika (Tomoyuki Chikanaga) via ruby-core" <ruby-core@...>
Date:
2024-07-15 13:02:37 UTC
List:
ruby-core #118606
SXNzdWUgIzIwMDMwIGhhcyBiZWVuIHVwZGF0ZWQgYnkgbmFnYWNoaWthIChUb21veXVraSBDaGlr YW5hZ2EpLg0KDQpCYWNrcG9ydCBjaGFuZ2VkIGZyb20gMy4yOiBSRVFVSVJFRCB0byAzLjI6IERP TkUNCg0KcnVieV8zXzIgY29tbWl0OmE4MDRkNTUxNGM3YzA2MDhiOWZiNTI0MjZlYzNlYzczODQy MGFkMjkgbWVyZ2VkIHJldmlzaW9uKHMpIGNvbW1pdDpkNTAzZTFiOTVhNDBlNDVkNzc2N2UwMTc1 ZGU2MDA5MmRlNGJhNTRlLg0KDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tDQpCdWcgIzIwMDMwOiBgUmlwcGVyLnRva2VuaXplKCciXFxDLeOBgiInKWAgc2VwYXJhdGVz IGVuY29kaW5nIHZhbGlkIHN0cmluZyB0byBlbmNvZGluZyBpbnZhbGlkIHN0cmluZy4NCmh0dHBz Oi8vYnVncy5ydWJ5LWxhbmcub3JnL2lzc3Vlcy8yMDAzMCNjaGFuZ2UtMTA5MTM2DQoNCiogQXV0 aG9yOiB0b21wbmcgKHRvbW95YSBpc2hpZGEpDQoqIFN0YXR1czogQ2xvc2VkDQoqIHJ1YnkgLXY6 IHJ1YnkgMy4zLjBkZXYgKDIwMjMtMTEtMzBUMTY6MjM6MjVaIG1hc3RlciBkMDQ4YmFlOTZiKSBb eDg2XzY0LWxpbnV4XQ0KKiBCYWNrcG9ydDogMy4yOiBET05FDQotLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tDQp+fn5ydWJ5DQpSaXBwZXIudG9rZW5pemUgJyJcXEMt44GC IicgIyBvciBSaXBwZXIudG9rZW5pemUgIlwiXFxDLVx1MzA0MlwiIg0KIyA9Pg0KWyJcIiIsICJc eDgxIiwgIlx4ODIiLCAiXCIiXQ0Kfn5+DQpJIGV4cGVjdCBhbGwgdG9rZW5zIHRvIGJlIHZhbGlk X2VuY29kaW5nIGlmIHRoZSBzb3VyY2Ugc3RyaW5nIGlzIHZhbGlkX2VuY29kaW5nLg0KDQpUaGlz IGlzIGNhdXNpbmcgSVJCIGNyYXNoIHdoZW4gdHlwaW5nIGAiXEMt44GCImAuDQoNCg0KDQoNCi0t IA0KaHR0cHM6Ly9idWdzLnJ1YnktbGFuZy5vcmcvDQogX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fXwogcnVieS1jb3JlIG1haWxpbmcgbGlzdCAtLSBydWJ5LWNv cmVAbWwucnVieS1sYW5nLm9yZwogVG8gdW5zdWJzY3JpYmUgc2VuZCBhbiBlbWFpbCB0byBydWJ5 LWNvcmUtbGVhdmVAbWwucnVieS1sYW5nLm9yZwogcnVieS1jb3JlIGluZm8gLS0gaHR0cHM6Ly9t bC5ydWJ5LWxhbmcub3JnL21haWxtYW4zL2xpc3RzL3J1YnktY29yZS5tbC5ydWJ5LWxhbmcub3Jn Lw==