28

I need to run an interactive Bash instance in a separated process in Python with it's own dedicated TTY (I can't use pexpect). I used this code snippet I commonly see used in similar programs:

master, slave = pty.openpty()

p = subprocess.Popen(["/bin/bash", "-i"], stdin=slave, stdout=slave, stderr=slave)

os.close(slave)

x = os.read(master, 1026)

print x

subprocess.Popen.kill(p)
os.close(master)

But when I run it I get the following output:

$ ./pty_try.py
bash: cannot set terminal process group (10790): Inappropriate ioctl for device
bash: no job control in this shell

Strace of the run shows some errors:

...
readlink("/usr/bin/python2.7", 0x7ffc8db02510, 4096) = -1 EINVAL (Invalid argument)
...
ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7ffc8db03590) = -1 ENOTTY (Inappropriate ioctl for device)
...
readlink("./pty_try.py", 0x7ffc8db00610, 4096) = -1 EINVAL (Invalid argument)

The code snippet seems pretty straightforward, is Bash not getting something it needs? what could be the problem here?

Dima Tisnek
  • 10,388
  • 4
  • 61
  • 113
TKKS
  • 453
  • 1
  • 4
  • 9
  • 2
    That's quite normal — you got an **interactive shell** without **job control**. – Dima Tisnek Jan 09 '17 at 09:23
  • 2
    If you want job control too, you need your shell to become a process leader — that is start new "session", it's achieved with `start_new_session=True` keyword argument to `Popen` (since Python 3.2). If you need more control, use `preexec_fn=...` – Dima Tisnek Jan 09 '17 at 09:28
  • Ok, that sound reasonable. I understand that the `start_new_session=True` is only relevant to >3.2. Is there an equivalent in 2.7? Sorry, probably should have mentioned the python version in the question. – TKKS Jan 09 '17 at 10:28
  • 2
    You can do that by hand by calling `setsid()` in `preexec_fn` via `ctypes` – Dima Tisnek Jan 09 '17 at 10:38
  • I think this question is about same fundamentals as http://stackoverflow.com/questions/23826695/handling-keyboard-interrupt-when-using-subproccess/23839524#23839524 http://stackoverflow.com/questions/33119213/run-program-in-another-process-and-receive-pid-in-python/33120039#33120039 http://stackoverflow.com/questions/37737649/how-to-destroy-an-exe-filenot-converted-from-py-by-run-as-the-same-script/37776347#37776347 http://stackoverflow.com/questions/13243807/popen-waiting-for-child-process-even-when-the-immediate-child-has-terminated/13256908#13256908 it could be considered a duplicate. – Dima Tisnek Jan 10 '17 at 10:26
  • I don't think any of them is really about using the pseudo-terminal with Popen like this. I don't think I could have solved this issue with any of these other questions. I will publish my solution code. – TKKS Jan 11 '17 at 14:04
  • How to do this on Windows, using the cmd shell instead of bash? – K.Mulier Feb 23 '21 at 21:16

2 Answers2

20

This is a solution to run an interactive command in subprocess. It uses pseudo-terminal to make stdout non-blocking(also some command needs a tty device, eg. bash). it uses select to handle input and ouput to the subprocess.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os
import sys
import select
import termios
import tty
import pty
from subprocess import Popen

command = 'bash'
# command = 'docker run -it --rm centos /bin/bash'.split()

# save original tty setting then set it to raw mode
old_tty = termios.tcgetattr(sys.stdin)
tty.setraw(sys.stdin.fileno())

# open pseudo-terminal to interact with subprocess
master_fd, slave_fd = pty.openpty()


try:
    # use os.setsid() make it run in a new process group, or bash job control will not be enabled
    p = Popen(command,
              preexec_fn=os.setsid,
              stdin=slave_fd,
              stdout=slave_fd,
              stderr=slave_fd,
              universal_newlines=True)

    while p.poll() is None:
        r, w, e = select.select([sys.stdin, master_fd], [], [])
        if sys.stdin in r:
            d = os.read(sys.stdin.fileno(), 10240)
            os.write(master_fd, d)
        elif master_fd in r:
            o = os.read(master_fd, 10240)
            if o:
                os.write(sys.stdout.fileno(), o)
finally:
    # restore tty settings back
    termios.tcsetattr(sys.stdin, termios.TCSADRAIN, old_tty)
JnBrymn
  • 22,802
  • 27
  • 102
  • 142
Paco
  • 351
  • 2
  • 8
  • From docs, `pty.spawn` seem to do similar stuff but simpler interface. https://docs.python.org/3/library/pty.html#example Are they doing the same? – balki Mar 26 '19 at 15:26
  • @balki [pty.spawn](https://docs.python.org/3/library/pty.html#pty.spawn) is blocking: "It is expected that the process spawned behind the pty will eventually terminate, and when it does spawn will return." – moi Mar 18 '20 at 20:40
  • @balki I've looked at the source code on https://github.com/python/cpython/blob/master/Lib/pty.py#L151. I think `pty.spawn` is doing basically the same thing as this code snippet does. So yes, use `pty.spawn` as it would make application code simpler. – Paco Jul 19 '20 at 05:45
  • 1
    if the process exits without output this hangs waiting for input (setting a timeout on `select.select` or setting it to nonblocking "fixes" this). this also needs a little work for window sizing (SIGWINCH, etc.) – Anthony Sottile Sep 05 '20 at 17:47
  • Is it possible to separate stdout/stderr ? – Noortheen Raja Feb 27 '22 at 20:05
  • in my case the subprocess required an interactive tty ... so when i tried doing bash `app.py &` it error'd out. then i tried passing `stdin=subprocess.DEVNULL` and that didn't work. then i tried passing an open file descriptor and that didn't work. finally i opened the `pty.openpty()` and passed in to the subprocess `stdin=slave_fd` and it worked. – Trevor Boyd Smith Jun 02 '22 at 15:03
7

This is the solution that worked for me at the end (as suggested by qarma) :

libc = ctypes.CDLL('libc.so.6')

master, slave = pty.openpty()
p = subprocess.Popen(["/bin/bash", "-i"], preexec_fn=libc.setsid, stdin=slave, stdout=slave, stderr=slave)
os.close(slave)

... do stuff here ...

x = os.read(master, 1026)
print x
TKKS
  • 453
  • 1
  • 4
  • 9