8

Using the SciPy/NumPy libraries, Python is a pretty cool and performing platform for scientific computing. I just wonder: When I have to go parallel (multi-thread, multi-core, multi-node, gpu), what does Python offer?

I'm mostly looking for something that is fully compatible with the current NumPy implementation. The coolest thing would be an implementation of NumPy and some other basic functionality (list comprehension for example) that is fully parallel and also can use GPUs. But I'm probably asking for a bit too much ;-)

(It's not that we are unable to switch to solution in C/C++, it's more that we'd really like to leverage the advantages of Python above these languages also in a parallel enivronment.)

Michael
  • 1,463
  • 11
  • 22
  • 2
    What scientific computing library are you using in C++? Petsc, for instance, has Python bindings, so essentially you get the same functionalities. – Federico Poloni Jun 22 '16 at 14:18

3 Answers3

6

Here's a few options that are relatively easy to work with:

  • One node - multiprocessing is the most straightforward thing to do. multiprocessing.map works well for an embarrasingly parallel problem.
  • GPU - pyCUDA allows you to do some GPU level programming with just python. I haven't toyed with it that much.
  • MPI - mpi4py- Exposes MPI interface at the python level. If you're familiar with MPI, this is useful.

There's a ton of options out there. Here's a page that attempts to catalog some of them.

(I wanted to give more links, but apparently I don't have the rep for that. A quick search on these terms should turn up documentation)

Danielle
  • 61
  • 3
2

Among the things you can try is pyMPI for MPI support in python. We have a good number of users at our HPC site that use it for parallelism. It's not an automated way to get there through NumPy, but you should look at its features carefully.

Bill Barth
  • 10,905
  • 1
  • 21
  • 39
2

IPython has a package for parallel computing called ipyparallel.

An overview from their documentation:

This architecture abstracts out parallelism in a very general way, which enables IPython to support many different styles of parallelism including:

  • Single program, multiple data (SPMD) parallelism.
  • Multiple program, multiple data (MPMD) parallelism.
  • Message passing using MPI.
  • Task farming.
  • Data parallel.
  • Combinations of these approaches.
  • Custom user defined approaches.

Most importantly, IPython enables all types of parallel applications to be developed, executed, debugged and monitored interactively. Hence, the I in IPython. The following are some example usage cases for IPython:

  • Quickly parallelize algorithms that are embarrassingly parallel using a number of simple approaches. Many simple things can be parallelized interactively in one or two lines of code.
  • Steer traditional MPI applications on a supercomputer from an IPython session on your laptop.
  • Analyze and visualize large datasets (that could be remote and/or distributed) interactively using IPython and tools like matplotlib/TVTK.
  • Develop, test and debug new parallel algorithms (that may use MPI) interactively.
  • Tie together multiple MPI jobs running on different systems into one giant distributed and parallel system.
  • Start a parallel job on your cluster and then have a remote collaborator connect to it and pull back data into their local IPython session for plotting and analysis.
  • Run a set of tasks on a set of CPUs using dynamic load balancing.
leekaiinthesky
  • 121
  • 3