From: nobu@...
Date: 2006-04-21T10:52:45+09:00
Subject: Re: Strange problem with PTY.spawn()

Hi,

At Thu, 20 Apr 2006 23:42:03 +0900,
Reto Schuettel wrote in [ruby-talk:189572]:
> But what happens if we add something before the call to PTY.spawn()
>  | $ ruby1.8 -rpty -e   'puts "blub"; so, si, pid = PTY.spawn("/bin/true"); ' 
>  | blub
>  | -e:1: [BUG] rb_sys_fail(can't get Master/Slave device) - errno == 0
>  | ruby 1.8.4 (2005-12-24) [sparc-solaris2.8]
>  | 
>  | Abort
> 
> Oops? Different error? (Nobus patch is applied on this machine).

Seems grantpt() on Solaris doesn't set errno perperly, or even
the return value is not correct.

Try this patch, but I don't know if it is in only particular
version of Solaris, or more.


Index: ext/pty/pty.c
===================================================================
RCS file: /cvs/ruby/src/ruby/ext/pty/pty.c,v
retrieving revision 1.19.2.3
diff -p -U2 -r1.19.2.3 pty.c
--- ext/pty/pty.c	8 Sep 2005 05:59:40 -0000	1.19.2.3
+++ ext/pty/pty.c	21 Apr 2006 01:52:56 -0000
@@ -349,9 +349,14 @@ getDevice(master,slave)
     extern int grantpt(int);
 
+#if defined(__solaris__) && defined(__sparc__)
+#define succeed(xxx) ((errno = 0), (xxx) != -1 && errno != 0)
+#else
+#define succeed(xxx) ((xxx) != -1)
+#endif
     if((i = open("/dev/ptmx", O_RDWR, 0)) != -1) {
 	s = signal(SIGCHLD, SIG_DFL);
-	if(grantpt(i) != -1) {
+	if(succeed(grantpt(i))) {
 	    signal(SIGCHLD, s);
-	    if(unlockpt(i) != -1) {
+	    if(succeed(unlockpt(i))) {
 		if((pn = ptsname(i)) != NULL) {
 		    if((j = open(pn, O_RDWR, 0)) != -1) {


-- 
Nobu Nakada