3

I'm trying to run some multiprocessing in a Jupyter notebook, using python version 3.7.0. However, even a really simple example seems to hang indefinitely. After reading this answer I tried explicitly calling .close and .join on the pool, but it still hangs. Example code is below, can anyone tell me what's wrong?

import multiprocessing as mp

def fun(x):
    return 2*x

with mp.Pool() as pool:
    args = list(range(10))
    res = pool.map(fun, args)
    pool.close()
    pool.join()
bjarkemoensted
  • 2,177
  • 2
  • 17
  • 35
  • 1
    `multiprocessing` has problems working in an interactive prompt. save your code as a .py file, and run it with the python interpreter. (it can be made to work, but it's much simpler to just run the file rather than execute code interactively) – Aaron Mar 02 '21 at 05:00
  • If you look at the console where you started up Jupyter Notebook you will see a bunch of errors. You need to put function `fun` in an external file, for example, `workers.py` in the same directory as your ipynb file and then add `from workers import fun`. In other words, worker functions need to be imported. – Booboo Mar 05 '21 at 18:54
  • Does this answer your question? [Jupyter notebook never finishes processing using multiprocessing (Python 3)](https://stackoverflow.com/questions/47313732/jupyter-notebook-never-finishes-processing-using-multiprocessing-python-3) – Booboo Mar 05 '21 at 18:56

1 Answers1

0

For me, it worked the solution proposed by @Booboo.

  • Write your function in an external file
  • import it to your .ipynb file
Eduardo Massieu
  • 13
  • 3
  • This should probably be a comment rather than an answer. But thank you for chiming in! – Apolymoxic Mar 11 '22 at 16:09
  • This does not provide an answer to the question. Once you have sufficient [reputation](https://stackoverflow.com/help/whats-reputation) you will be able to [comment on any post](https://stackoverflow.com/help/privileges/comment); instead, [provide answers that don't require clarification from the asker](https://meta.stackexchange.com/questions/214173/why-do-i-need-50-reputation-to-comment-what-can-i-do-instead). - [From Review](/review/late-answers/31266240) – Muhammad Mohsin Khan Mar 14 '22 at 17:19