4

I am a beginner in multiprocessing, can anyone tell me why this does not produce any output?

import multiprocessing

def worker(num):
    """thread worker function"""

    print('Worker:', num)

if __name__ == '__main__':
    jobs = []

    for i in range(4):
        p = multiprocessing.Process(target=worker, args=(i,))
        jobs.append(p)
        p.start()
Will
  • 22,773
  • 13
  • 90
  • 102
user128751
  • 429
  • 2
  • 5
  • 9

2 Answers2

3

You're starting your Process(), but never waiting on it to complete, so your program's execution ends before the background process finishes. Try this, with a call to Process.join():

import multiprocessing
import sys

def worker(num):
    """thread worker function"""

    print('Worker:', num)
    sys.stdout.flush()

if __name__ == '__main__':
    jobs = []

    for i in range(4):
        p = multiprocessing.Process(target=worker, args=(i,))
        jobs.append(p)
        p.start()

    map(lambda p: p.join(), jobs)

Here we use map() to call join() on each process in the jobs list.

Here's an example of this working:

In [127]: %paste
import multiprocessing
import sys

def worker(num):
    """thread worker function"""

    print('Worker:', num)
    sys.stdout.flush()

if __name__ == '__main__':
    jobs = []

    for i in range(4):
        p = multiprocessing.Process(target=worker, args=(i,))
        jobs.append(p)
        p.start()

    map(lambda p: p.join(), jobs)

## -- End pasted text --
Worker: 2
Worker: 1
Worker: 0
Worker: 3

In [128]:

As for why this isn't working in IDLE, see this answer:

However, to see the print output, at least on Windows, one must start IDLE from a console like so.

C:\Users\Terry>python -m idlelib
hello bob

(Use idlelib.idle on 2.x.) The reason is that IDLE runs user code in a separate process. Currently the connection between the IDLE process and the user code process is via a socket. The fork done by multiprocessing does not duplicate or inherit the socket connection. When IDLE is started via an icon or Explorer (in Windows), there is nowhere for the print output to go. When started from a console with python (rather than pythonw), output goes to the console, as above.

Community
  • 1
  • 1
Will
  • 22,773
  • 13
  • 90
  • 102
  • running as a script work, but using IDLE it does not work. – user128751 Jun 01 '16 at 17:53
  • Weird thing. In ipython when I run the **code with join** after the **code without join** the missing messages from the **code without join** are also flushed. So this is not about premature exit, but smth about stdout caching in ipython. – robyschek Jun 01 '16 at 17:57
  • @user128751 I just verified that if you start IDLE from the console, as I said in the quoted text, Will's revision with join work's when run from an IDLE editor. – Terry Jan Reedy Jun 01 '16 at 21:13
2

You probably need to flush the output. You can use sys.stdout.flush():

import multiprocessing
import sys

def worker(num):
    """thread worker function"""
    print('Worker:', num)
    sys.stdout.flush()
    return

if __name__ == '__main__':
    jobs = []
    for i in range(4):
        p = multiprocessing.Process(target=worker, args=(i,))
        jobs.append(p)
        p.start()
rofls
  • 4,795
  • 3
  • 25
  • 37