13

In his famous The Free Lunch Is Over article from 2005, Herb Sutter predicted a Concurrent Programming Revolution as big as Object-Oriented Revolution. Has this revolution really happend in years 2005 - 2013?

Key points in the article:

  • Processor manufacturers have run out of room with most of their traditional approaches to boosting CPU performance. Instead of driving clock speeds ever higher, they are instead turning to hyperthreading and multicore architectures.

  • Applications will increasingly need to be concurrent if they want to fully exploit CPU throughput gains.

  • “Oh, performance doesn’t matter so much, computers just keep getting faster” statement will be wrong.

  • Efficiency and performance optimization will get more, not less, important. Those languages that already lend themselves to heavy optimization will find new life; those that don’t will need to find ways to compete and become more efficient and optimizable. Expect long-term increased demand for performance-oriented languages and systems.

  • Programming languages and systems will increasingly be forced to deal well with concurrency. We desperately need a higher-level programming model for concurrency than languages offer today.

cpp
  • 281
  • 16
    How many cores does your phone have? – Matt D Sep 30 '13 at 05:13
  • 6
    My Nokia 2700 has one core. – cpp Sep 30 '13 at 05:57
  • Time to upgrade! Even consumer devices like the Ipad routinely now have multiple cores. Single core devices are becoming the exception and not the rule. – Matt D Sep 30 '13 at 06:12
  • 1
    IMHO there has been an evolution, especially in the mainstream languages, but not a revolution. – Doc Brown Sep 30 '13 at 09:25
  • 5
    @MattD Sorry, but why is it "time to upgrade", and what does the number of cores in cpp's phone have to do with it? How do you know a Nokia 2700 isn't enough for his/her needs? – Andres F. Sep 30 '13 at 12:47
  • 2
    I'm going to go on record to say, by a totally non-scientific observation, that there has been such a revolution on the hardware side of things, but the software revolution has been extraordinarily slow-going. Getting quality concurrent tools for all programmers is still a tough gig and concurrency is still something that trips up even seasoned parallel developers in hard-to-reproduce ways. – Jesse C. Slicer Sep 30 '13 at 14:49
  • 2
    I just want to point out that just because the timing of the predications is wrong, it doesn't necessarily make the predictions wrong. It is quite obvious that the key points you have listed above have not obtained the importance implied, but there have been steps in the direction of each of those points. So I'd say the above predictions are going to become true, it is just a question of when. The WHEN will be when it becomes a requirement and not just a desire. Knowing when that threshold will be crossed is why predicting when is so hard. – Dunk Sep 30 '13 at 15:40

2 Answers2

23

Yes, but it depends.

You can’t expect to write nontrivial, high-performance software without both taking advantage of parallel hardware and using concurrency as a program structuring technique. But most software is both trivial and non–performance-critical. A web app isn’t doing much number crunching, and CRUD apps have nothing like the hard timing limits of some simulation and medical software.

Game developers in particular need to care about this, because games are the most common type of application with soft realtime requirements. The problem is salient on a mobile phone, where you want to squeeze as much computing and rendering power as possible out of an integrated chip with two CPU cores and a low-power GPU. That’s another reason that so many developers are looking at Haskell and waiting for languages like Rust to mature—we want safety and performance on modern hardware.

Since 2005 we have gained new and improved tools such as OpenCL, CUDA, OpenMP, and vector instruction sets for working with concurrency and data parallelism in established languages. However, the relative newcomers are designed from early on to do many more interesting things with concurrency.

Haskell’s concurrent runtime allows the language to provide rich support for lightweight parallelism (sparks) and concurrency abstractions (threads, channels, and shared mutable references). Go and Rust also offer lightweight tasks, Go using channels and Rust using message passing.

These systems offer memory safety, performant runtimes, and static protection against certain kinds of races. The by-default immutability of Haskell and Rust make concurrency much easier for humans to manage. Erlang was doing this already in the 80s, but the needs of software and our knowledge about how to design programming systems have also improved since—thank goodness.

Finally, many existing languages—I won’t name names—are ready to decline as credible choices for writing new software. Their burdens of complexity and poor concurrency abstractions make them unsuitable for the considerations of modern applications. We are simply waiting for mature alternatives.

Jon Purdy
  • 20,547
  • 2
    I'm not so sure about decline of certain languages, I'm hoping we'll see code written with more focus on minimizing dependencies than anything else. After all, it's not really the language that's at fault, its how you use it. And the "low hanging fruit" of its use is no longer aligned with multithreaded/concurrency. – Matt D Sep 30 '13 at 06:15
  • 6
    @MattD: Both how you use a language and the language itself can be at fault. Say your program is wrong. You’re the one who wrote it wrong, but the language didn’t necessarily help you to write it right, and that’s just as important. – Jon Purdy Sep 30 '13 at 07:39
6

Here are few data points; decide for yourself if it counts as a revolution.

Parallelized Hardware

Around 2005, both Intel and AMD start mass-producing 2-core desktop x86 CPUs (Pentium D and Athlon 64), with clock speeds around 3 GHz.

In 2006, PlayStation 3 is released, featuring the Cell processor with 8+1 cores at 3.2 GHz.

In 2006, GeForce 8 series is released. It consists of large numbers (~100) of general-purpose 'stream processors', as opposed to graphics-specific units. Around 2007, CUDA 1.0 spec is released, allowing some general-purpose computation run on massively-parallel NVidia hardware.

Since then, the tendencies continued.

Say, now, in 2013, both Intel and AMD offer 4, 8, and 16-core CPUs, with clock speeds slightly above mere 4 GHz. Dual-core and quad-core designs are common for lower-power devices, such as laptops and smartphones.

All this is mass-produced, consumer-grade everyday computer hardware.

Software

CUDA is released in 2007, then OpenCL in 2008, allowing to use massively-parallel GPUs in general (non-graphical) computation. The model becomes popular; many hosting companies (e.g. Amazon) offer GPUs for general computing tasks.

Go is released in 2009, featuring very cheap preemptive threads ("goroutines") and allowing to efficiently express highly concurrent algorithms.

Akka toolkit is released for Java and Scala in 2009, allowing for actor-based concurrency.

Erlang (a highly concurrent language) sees some increase in usage.

Concurrency vs Parallelism

Note that to utilize parallel hardware, one does not necessarily need software concurrency, that is, juggling with threads of execution within a computation. Many problems are solved by parallel, non-interacting processes, where each process is a traditional sequential program.

Parallel processing may utilize more traditional languages and parallel frameworks, like map-reduce or MPC or OpenMP. For such frameworks, presence of multiple cores on the same CPU crystal is not conceptually very different from just having more CPUs in the cluster; the difference is mainly speed.

No free lunch so far

CPU speeds still linger at around 5 GHz at the high end. With better technologies in sight, like graphene transistors, frequencies may again rise in the future, but not very soon probably.

9000
  • 24,232