Does OOP work easily and efficiently in parallel computations?

Question

I am working on a task for which the object oriented (OO) approach fits well. I am doing it in MATLAB, since I am in the prototyping phase. However, I know that later I will surely have to perform larger scale computations (my university has a cluster). On method is to use mex functions for the computational heavy parts. But it does not work well for MATLAB classes. The other option is to write the code in C++, whose OOP is different from that of MATLAB, but there are many similarities, and the idea is the same. How easy is to use then OpenMP, MPI, PETSC, etc with C++ classes to parallelize my code? The third option would be to neglect OOP, but then I sacrifice the elegance and extensibility and of my program. My questions:

1) Do you recommend me to remain with OOP, or switch to the procedural way?

2) Which parallelization technique do you recommend me (OpenMP, MPI, PETSC, etc.)? I do not want to invest enormous amount of time in it. I am quite skilled in MATLAB, but have only a basic knowledge in C and nothing in C++.

EDIT

From one of the comments it turned out that there is no difference if I use standard variables or objects. So to reformulate question 1)

1) Is it sure that OOP does not make my life harder when I am going to do the parallelization? I will create a specific application not a general tool; in this case how much OO C++ is difficult to learn? I won't need special data structures, just the loops, if statements and the call of parallelization libraries. Is that a viable solution just to make the class methods parallel (so that the implementation remains hidden from outside) or a complete rewriting is required?

Answer 1

Empirically, your comment that "most numerical libraries 'do not like' OOP, that is why many software are written in C or Fortran" is not correct. Instead, I would say that almost all software libraries that have been written over the last 20 years are in one way or another object oriented. For example, PETSc is object oriented (even though it is written in C, but it basically uses classes, inheritance, and virtual functions under the hood), and so is the other large linear algebra library, Trilinos (written in C++). In the finite element context, libMesh, FEniCS/Dolphin, and deal.II (my own project) are all written in C++ in an object oriented way. In fact, even MPI is written in an object oriented way, where communicators are like classes and most functions operate on them (in C++ one would write them as member functions).

So in reality, regardless of language, the object oriented programming paradigm has long won the battle in scientific computing. This is true whether you write your code in C++ or in C/Fortran -- basically all codes of significant size use the OOP paradigm.

Answer 2

Without knowing more about your problem, let me just say some general thoughts.

You can definitely parallelize OOP code using either openMP or MPI. In fact basically all of the professional/commercial software solving computational science problems take advantage of the OOP paradigm because OOP provides significant advantages in encapsulation, extensibility, maintainability, modularization, inheritance etc. I should stress that openMP and MPI are fundamentally different parallelization paradigms:

openMP

1. Shared memory paradigm
2. Useful for speeding up code on a given processor. For example instead of using just 1 of your 4 cores on a quad care processor you can use all 4 cores.
3. All your threads see everything - i.e. all cores can read an write to the same arrays, vectors, etc

MPI

1. Distributed memory paradigm
2. Useful for breaking up your problem and distributing it across many machines in a cluster.
3. Each separate node acts like a separate computer. Your problem must be broken up and given to each node/computer in the cluster. That node then works on that data only.
4. If you need to access data on a different node then you have to do that through communication calls between the nodes.
5. Communication is relatively expensive and should be minimized. This coupled with the fact that the problem must be broken up means that quite often sequential algorithms must be completely re-thought to work in a distributed environment.

Hybrid

1. Here we use both MPI and openMP. For example you might break up your problem and distribute it across many nodes on a cluster. Each node might have 4, 6, 8 ,16 or more cores. On each node you then use openMP to gain further speed-up.

I would say that in general parallelization is not a trivial endeavor - especially with MPI. Many times you will need to completely re-think your algorithms so that they work efficiently on a distributed environment. Because of this I think it would be very challenging to learn both C++ and MPI in a short amount of time. C++ alone can take years to truly master. MPI on the other hand isn't difficult from a syntactic perspective - you may only need to know 10-15 subroutine calls - but can be difficult in terms of adjusting your algorithms to handle a distributed memory model.

For a beginner openMP is probably the easiest and is less likely to require fundamental algorithm changes. You have a slow for loop? You might be able to speed this section of the code up with a simple #pragma statement.

Conclusion

I would definitely recommend using OOP if your code is large and you want it to be able to grow and be maintained over time. Heck even if your code is small OOP is great for creating well designed, modularized code. Learning a language like C++ - especially since you know C already - wont be too difficult (but will take time to become an expert) and it will give you dividends over the long haul. If you are serious about parallelizing your code yourself then you will want to learn both openMP and MPI eventually. In the meantime openMP is probably the easiest and if you are using gcc doesn't require installing as it operates by calling pre-processor flags.

Answer 3

I think Python is also worth considering along with the languages you mention, as it has several advantages over Matlab for rapid development, but insulates you from a lot of the low-level details that come with using C/C++.

The ecosystem for scientific computing in Python is quite rich. You mentioned PETSc, for which there are Python bindings; there are also MPI bindings for Python; and that's not even scratching the surface of what you can do with Cython, Numba, etc. As for finite element methods, fenics is fairly popular, though by no means the only choice. I picked up Python in a matter of months, but I only felt competent in C++ after ~4 years of using it.

Python cannot beat optimized C/C++ code for speed, but the performance gap can be as little as 10-40% with tools like Numba and Cython, together with outsourcing computationally intensive code to PETSc and Trilinos. In my experience, you will have to worry about memory management and other low-level details when programming in C/C++, which might not be the best use of your time if you're only interested in algorithms.

That said, learning Python does not impede you from learning C++; quite the contrary, the more languages you know, the easier it is to pick up new ones. A practicing computational scientist has to know C, C++, Python, and probably also Fortran and x86-64 assembly. I happen to like Lisp and OCaml too, but those are just for fun.

As for object-oriented programming, using this paradigm should not, in principle, hamper the scalability or speed of a large scientific application. The principle performance concern when using OOP is the added cost of virtual function dispatch. It would take too long to talk about that in any amount of detail, but suffice it to say that this cost is often completely negligible compared to the other operations being performed in a typical scientific code, e.g. linear algebra operations. The benefits to the developer, on the other hand, are immense. If anything, good use of OO design principles can help you write parallel code more easily.