17

Concurrent programming is quite difficult to me: even looking at a basic slide seems challenging to me. It seems so abstract.

What are the benefits to knowing Concurrent programming concepts well? Will it help me in regular, sequential programming? I know there is a satisfaction to understanding how our programs work, but what else?

Caffeinated
  • 656
  • 3
    I think it's kind of off topicky, but can get kind of on topicky if you edit out all the personal stuff (although concurrent programming is quite difficult to everyone) and ask for the concrete technical merits of the concepts. – yannis Oct 21 '11 at 02:47
  • 1
    The benefits should be quite obvious. You can write programs that can take advantage of all of the performance enhancements available through division of work that concurrent programming offers. It's not easy for anyone. It's a very challenging concept today. – Rig Oct 21 '11 at 03:04
  • @Rig - Thanks so much. I will just dive in then, piece by piece. I think I must keep testing myself too. – Caffeinated Oct 21 '11 at 03:10
  • 3
    We can't help you get motivation to learn something, but the general question about why one ought to know about concurrency is on-topic enough. –  Oct 21 '11 at 05:59
  • 2
    I know concurrent programming. I can tell that the slide you provide does not help to understand. Instead, go to a party with dining philosophers. – mouviciel Oct 21 '11 at 07:37
  • 1
    Relax, even the greatest find it difficult: http://www.informit.com/articles/article.aspx?p=1193856 – SK-logic Oct 21 '11 at 10:32
  • 1
    I updated my answer to include DB work on the web. –  Oct 22 '11 at 02:50
  • 1
    What is the bounty for? You've got 7 answers, one of which you've already accepted as the correct one. Perhaps you can clarify more specifically what you are looking for? – Robert Harvey Nov 01 '13 at 16:16
  • @RobertHarvey - OK i know this didn't make too much sense. How do I reverse bounty? – Caffeinated Nov 01 '13 at 16:20
  • 1
    Uh... Well, you can flag the question with a custom moderator flag and ask for a reversal, but it will be up to them. If you want something more specific from your question, just say so. – Robert Harvey Nov 01 '13 at 16:21
  • @RobertHarvey - understood, I will do so. thanks very much Sir! – Caffeinated Nov 01 '13 at 16:22
  • To all those who think concurrent programming is difficult (yes, including the revered Donald Knuth) - Stick to these keywords: Actor model, Thread-safe queue (also known as channels in most concurrent languages), and a choice of either isolation or immutability. Then, throw in some cores / thread pools, and voila you have multicore programming. See full answer by Robert Love. – rwong Nov 02 '13 at 07:01

7 Answers7

32

Here's a quick and easy motivation: If you want to code for anything but the smallest, weakest systems, you will be writing concurrent code.

Want to write for the cloud? Compute instances in the cloud are small. You don't get big ones, you get lots of small ones. Suddenly your little web app is a concurrent app. If you designed it well, you can just toss in more servers as you gain customers. Else you have to learn how while your instance has its load average pegged.

OK, you want to write desktop apps? Everything has a dual-or-more-core-CPU. Except the least expensive machines. And people with the least expensive machines probably aren't going to fork over for your expensive software, are they?

Maybe you want to do mobile development? Hey, the iPhone 4S has a dual-core CPU. The rest won't be far behind.

Video games? Xbox 360 is a multi-CPU system, and Sony's PS3 is essentially a multi-core system.

You just can't get away from concurrent programming unless you are working on tiny, simple problems.

2016 update: The current iteration of the $35 Raspberry Pi is built around a quad-core system on a chip intended for cell phones. Dramatic advances in AI have been made in part due to the availability of high-end graphics cards as parallel compute engines.

ObscureRobot
  • 604
  • 4
  • 7
  • 1
    While I agree in principle, to say that Everything has a dual-or-more-core-CPU. Except the least expensive machines. seems a little preposterous. Lots of people have single-core machines, not because it was cheap, but because they are happy with what they have and see no need to upgrade. That said, thinking in terms of concurrency will help the scheduler on a single-core system as well, so it is not wasted effort anywhere you can assume preemptive multitasking, either (which is about every multitasking environment most developers will come in contact with, these days). – user Oct 21 '11 at 11:38
  • 1
    ok - that's a bit of an exaggeration, but new hardware tends overwhelmingly to be multi-core. I don't see that going away. So if you are a student today thinking about the work you will be doing professionally in the future it is safe to assume that you will be working on multicore systems. – ObscureRobot Oct 21 '11 at 12:09
  • I wonder if i disagree with your answer as much as you would disagree with mine lol ;) –  Oct 22 '11 at 01:26
  • 1
    The way I read it, there is a kernel of similarity to what we are both saying, @acidzombie24. I'm saying that a developer should know how to deal with concurrency because it will be everywhere. You are saying that you don't have to be good at concurrent programming as long as you ... avoid the pitfalls of concurrent systems :) – ObscureRobot Oct 22 '11 at 02:10
  • I agree it's very useful to know about concurrency, but disagree you can only get away from it for "tiny, simple problems". You can get away from concurrency a great deal, even in non-trivial systems, for example if you rely on existing frameworks and application servers. Once the infrastructure is in place, I can be a junior developer writing new services for a web app and know almost NOTHING about concurrency or parallelism. – Andres F. Nov 01 '13 at 16:39
  • No mention of GPUs? These are some the most ubiquitous massively-parallel multi-core systems around, and virtually every computer has one, and none of them are single-core. – greyfade Nov 01 '13 at 19:05
  • @AndresF. that is a recipe for much future pain. – ObscureRobot Nov 02 '13 at 21:53
  • @greyfade GPUs are highly concurrent, but if you aren't a game developer or working on extreme high performance computing, you probably shouldn't be talking directly to the hardware. – ObscureRobot Nov 02 '13 at 21:55
  • @ObscureRobot I disagree. I'd say the average web developer needs to know very little about concurrency, and can still be very productive for standard web apps. Often the only shared resource is the database. I strongly disagree with your comment about "tiny, simple problems". The fact is the software infrastructure already solves most of the problems, unless it's a very specialized case. – Andres F. Nov 02 '13 at 21:59
  • @ObscureRobot: Just because GPUs are often used for games doesn't mean that that's their only use. Modern GPUs are general-purpose compute engines, and can run quite general code. They happen to be very good at floating-point math, so they get used a lot in a lot of different kinds of simulation workloads. If you're at all interested in doing highly-concurrent loads, you do want to talk to the hardware. – greyfade Nov 02 '13 at 22:00
  • @AndresF. Please tell me where you work so that I can be sure to never take a job there. Infrastructure does not magically make concurrency go away, any more than garbage collection makes it unnecessary to know anything about memory management. – ObscureRobot Nov 02 '13 at 23:27
  • @greyfade I would draw your attention to the portion of my comment where I said "or working on extremely high performance computing". – ObscureRobot Nov 02 '13 at 23:29
  • @ObscureRobot I cannot disclose where I work, so you'll have to take my word for it: it's a successful business which requires a scalable solution. Note I didn't say concurrency (or memory management, whatever) "is unnecessary to know"; I was just challenging your mistaken assertion that you can only disregard concurrency for tiny, simple problems. Infrastructure does make some of the concurrency internals go away. For a whole class of non-tiny apps, a server taking care of handling requests, and the database being the single point of shared state, means you can hand-wave lots of stuff. – Andres F. Nov 02 '13 at 23:34
  • @ObscureRobot Of course, if you start doing other tricky stuff, for example involving other shared resources, then you must know about concurrency. But that's going beyond your initial simplistic assertion, isn't it? Just admit you were wrong in your oversimplififcation; it won't hurt you :) – Andres F. Nov 02 '13 at 23:36
  • @ObscureRobot PS: and of course, you can write single-user, non-tiny desktop apps without having the faintest idea of what concurrency is. I'm sure you can think of other examples, come on. – Andres F. Nov 02 '13 at 23:38
  • Instead, why don't you write your own devil's advocate answer to the question? Though briefly amusing, trolling in the comments isn't all that helpful. – ObscureRobot Nov 03 '13 at 06:47
22

From 1970 to about 2002 processors doubled in speed about every 18 months. So as a programmer all you had to do was wait and your program would go faster. The problem is that around 2002 the rules changed. Now they are not making bigger fast processors they are making smaller slower processors but putting them out in groups. The computer I am working on now has 4 cores, and Chips with up to 8 cores (and 4 threads per core) exist. Soon enough we will have chips with a lot more cores.

So if you write a program that is not at all concurrent you will find that you are using 1 core or thread, but the rest of the CPU is sitting there doing nothing. So if you have 16 cores 1 will be running your program and the other 15 are sitting there!

The problem with concurrency is that it is non deterministic. Which is to say that you don't know exactly what order different threads will do things in. Traditionally programmers have tried to solve this by using locks and the like. This has lead to a LOT of pain. Having some form of mutable state that more than one thread can access freely is often a formula for pain and heisnebugs!

Of late the trend has be to moving to functional languages which tightly control mutable state. There are two basic ways that functional languages handle concurrency. The first is by using message passing. This is best shown by Erlang. In Erlang there is in general no Shared state between processes. They communicate not by sharing memory but my passing messages. This should make sense to you as we are doing it right now. I am sending this information to you by sending you a message, not by you remembering it out of my brain! By switching to message passing most of the locking bugs simply go away. In addition messages can be passed over the network as well as within one node.

The other method is STM, which stands for Software Transcriptional Memory, This is present in clojure and Haskell (and others). In STM memory is shared but changes can only be made via a transaction. As the Database folks figured all this stuff out in the 1970's it is pretty easy to ensure that we get it right.

Actually I over simplified a bit, Clojure and Haskell can both do message passing, and Erlang can do STM.

Disclaimer I am the author of Programming Web Services with Erlang, which will be out in Early release in the next few weeks.

Zachary K
  • 10,413
  • 2
  • 38
  • 55
  • 1
    @Zachary K: are there approaches that blend functional languages with native languages so that computation-intensive parts are implemented in the native language but they provide interfaces that can be consumed by a server written in functional language? – rwong Oct 21 '11 at 10:50
  • Not 100% sure, but both Clojure and Scala exist on the JVM, so that is where I would start. Maybe take a look at the Akka framework. I have not used it but did listen to a talk about Akka a while back and it seems like it might be pretty cool. For now I'm doing Erlang and Javascript which is taking up most of my time! – Zachary K Oct 21 '11 at 11:16
  • 1
    @rwong: .NET lets programmers use C# or other non-functional languages for some parts of their apps and F#, a functional language, for others. – Kevin Nov 02 '13 at 00:34
5

Because concurrency can blow up in your face when you expect it the least...

fortran
  • 458
  • 2
  • 10
4

The first rule of concurrent programming is "It's difficult". The second rule of concurrent programming is "It. Is. Difficult"..!!

Seriously though, there are two common approaches to concurrent programming, multi-threading and multi-processing. Multi-processing is the easiest to comprehend since it just means having multiple instances of a process running to accomplish a task. The is pretty easy to do on Unix based systems via calls to fork/join, but not so easy on Windows systems.

Multi-threading is probably the approach most people think of when talking about concurrency. It's not difficult to start multiple threads within an application, but the devil is in the details. You need to co-ordinate data sharing between thread (usually using locks) which can lead to deadlock or data in an invalid state. You also need to understand how to communicate between thread using concepts like semaphores, conditional variables etc etc.

The advantage to all this is that once you understand it you're able to more effectively utilize the underlying hardware. These days it's pretty much the norm for a processor to have multiple cores. By utilizing concurrent programming you can make these cores work for you, and your application will get a speed improvement.

The disadvantage is that you have to start thinking about how you'll split your application up into small parts that can be run on different threads. This is a lot harder than it sounds. Also, highly concurrent solutions can be awkward to unit test as the order of execution is less deterministic.

These days most languages ship with an abstraction over most concurrent primitives to make life a bit easier. For example, .NET 4 ships with the Task Parallel Library which make life a bit easier. In Java land they've got the Concurrency package.

Sean
  • 1,134
  • 1
    It gets orders of magnitude if you run away from locks as fast as possible. Use STM or actors and all of what you said goes away. Of course that means moving away from Java to languages like Scala, Erlang or Clojure. (though I would argue that this is also a good thing) – Zachary K Oct 21 '11 at 11:53
  • @Zachary: It may be a good thing, but if you working in a .NET shop, for example, it's not practical. STM may be an option in the future, but right now it's not an option in mainstream languages. – Sean Oct 21 '11 at 11:57
  • Clojure runs on .net, and there is F#. I would bet that there are also STM implementations for C#. – Zachary K Oct 21 '11 at 11:59
3

I recently had a very interesting task to do in which multiprocessing saved me. I basically had to do a lot of requests to a few separate servers, dealing with very small amounts of data, but many requests.

Working with PHP, I did things the old fashioned way, and the best time I obtained after a few hours of work resulted in ~120 seconds to run a certain test (many requests + network delay + no async)

But that wasn't nearly enough compared to what I needed, and after failing miserably with PHPs multiprocessing, I switched to Python.

After a few hours, I had a running Python multiprocessing script that ran in 20 seconds, and after a bit of fiddling with the timeouts and no. of threads to be used, I got it down to ~10 seconds.

This was for a website written 100% in PHP, except a single, 100 line Python script. And the whole thing is working perfect.

My conclusion would be that even if it won't help you on a day to day basis, you may encounter situations where knowing at least the basics of concurrent programming will help you greatly.

Good luck, and happy coding!

PS: I'm not trying to bash PHP, but PHP simply wasn't the right tool for the job at hand.

PS2: Knowing a new technology, or a new way of doing things can open the door to a whole new world of possibilities.

Vlad Preda
  • 151
  • 4
2

If you do any sort of web development, concurrency comes into play, at least with most languages. For example, I use spring for web development and each new request comes in as its own thread. Therefore, if any request ends up accessing a shared object, where the state can be changed of a variable, concurrency is very much a factor and has to be taken into consideration. If its not, then the data can be edited in unpredictable ways and data corruption can result. Its not critical to know every last detail about concurrency but learning pieces at a time is important to better understand web app programming, if you are working on desktop apps, maybe its not so important unless you need to run multiple threads

programmx10
  • 2,087
-1

Learn the insight of operating systems. Reading the source code of schedulers and device drivers will help; they are definitely concurrent.

jj1bdx
  • 107
  • 3
  • 2
    Concurrency for programmers usually go for your own programs running in multiple instances. –  Oct 21 '11 at 07:54
  • I tried to emphasize you can't write your own concurrent program anyway without knowing the details of the scheduling algorithm of the OS kernel. – jj1bdx Nov 03 '11 at 11:20
  • Why not? If you use the locking mechanisms correctly the OS scheduling algorithm is unimportant. –  Nov 03 '11 at 11:39