Multithreading: am I doing it wrong?

Question

I'm working on an application that plays music.

During playback, often things need to happen on separate threads because they need to happen simultaneously. For example, the notes of a chord need to be heard together, so each one is assigned its own thread to be played in. (Edit to clarify: calling note.play() freezes the thread until the note is done playing, and this is why I need three separate threads to have three notes heard at the same time.)

This kind of behavior creates many threads during playback of a piece of music.

For example, consider a piece of music with a short melody and short accompanying chord progression. The entire melody can be played on a single thread, but the progression needs three threads to play, since each of its chords contains three notes.

So the pseudo-code for playing a progression looks like this:

void playProgression(Progression prog){
    for(Chord chord : prog)
        for(Note note : chord)
            runOnNewThread( func(){ note.play(); } );
}

So assuming the progression has 4 chords, and we play it twice, than we are opening 3 notes * 4 chords * 2 times = 24 threads. And this is just for playing it once.

Actually, it works fine in practice. I don't notice any noticeable latency, or bugs resulting from this.

But I wanted to ask if this is correct practice, or if I'm doing something fundamentally wrong. Is it reasonable to create so many threads each time the user pushes a button? If not, how can I do it differently?

Answer 1

One assumption you are making might not be valid: you require (among other things) that your threads execute simultaneously. Might work for 3, but at some point the system is going to need to prioritize which threads to run first, and which one wait.

Your implementation will ultimately depend on your API, but most modern APIs will let you tell in advance what you want to play and take care of the timing and queuing themselves. If you were to code such an API yourself, ignoring any existing system API (why would you?!), an event queue mixing your notes and playing them from a single thread looks like a better approach than a thread per note model.

Answer 2

Do not rely on threads executing in lockstep. Every operating system I know of does not make the guarantee that threads execute in time consistent with each other. This means that if the CPU runs 10 threads, they do not necessarily receive equal time in any given second. They could quickly get out of synch, or they could stay perfectly in synch. That is the nature of threads: nothing is guaranteed, since the behavior of their execution is non-deterministic.

For this specific application, I believe you need to have a single thread that consumes music notes. Before it can consume the notes, some other process needs to merge instruments, staffs, whatever into a single piece of music.

Let us assume that you have e.g. three threads producing notes. I would synchronize them on a single data structure where they can all put their music. Another thread (consumer) reads the combined notes and plays them. There may need to be a short delay here to ensure that thread timing does not cause notes to go missing.

Related reading: producer-consumer problem.

Answer 3

A classic approach for this music problem would be to use exactly two threads. One, a lower priority thread would handle the UI or sound generating code like

void playProgression(Progression prog){
    for(Chord chord : prog)
        for(Note note : chord)
            otherthread.startPlaying(note);
}

(note the concept of only starting the note asynchronously and going on without waiting for it to finish)

and the second, realtime thread would consistently look at all the notes, sounds, samples, etc that should be playing now; mix them and output the final waveform. This part may be (and often is) taken from a polished third party library.

That thread would often be very sensitive to "starvation" of resources - if any actual sound output processing is blocked for longer than your sound card buffered output, then it will cause audible artifacts such as interruptions or pops. If you have 24 threads that are directly outputing audio, then you have a much higher chance that one of them will stutter at some point. This is often considered unacceptable, as humans are rather sensitive to audio glitches (much more than to visual artifacts) and notice even minor stuttering.

Answer 4

Well, yes, you are doing something wrong.

The first thing is that creating Threads is expensive. It has much more overhead than just calling a function.

So what you should do, if you need multiple threads for this job is to recycle the threads.

As it looks to me you have one main thread that does the scheduling of the other threads. So the main thread leads through the music and starts new threads whenever there is a new note to be played. So instead of letting the threads die and restarting them, better keep the other threads alive in a sleep loop, where they check every x milliseconds (or nanoseconds) if there is a new note to be played and otherwise sleep. The main thread then doesn't start new threads but tells the existing thread pool to play notes. Only if there aren't enough threads in the pool it can create new threads.

The next one is the synchronization. Hardly any modern multithreading systems actually guarantee that all threads are executed at the same time. If you have more threads and processes running on the machine than cores (which is mostly the case), then the threads don't get 100% of the CPU time. They have to share the CPU. This means every thread gets a small amount of CPU time and then after it had it's share the next thread gets the CPU for a short time. The system doesn't guarantee that your thread gets the same CPU time as the other threads. This means, one thread might be waiting for another one to finish, and thus be delayed.

You should rather have a look if it isn't possible to play multiple notes on one thread, so that the thread prepares all the notes and then only gives the "start" command.

If you need to do it with threads, at least reuse the threads. Then you don't need to have as many threads as there are notes in the whole piece, but only need as many threads as the maximum number of notes played at the same time.

Answer 5

The pragmatic answer is that if it works for your application and meets your current requirements then you're not doing it wrong :) However if you mean "is this a scalable, efficient, reusable solution" then the answer is no. The design makes many assumptions about how the threads will behave which may or may not be true in different situations (longer melodies, more simultaneous notes, different hardware, etc.).

As an alternative consider using a timing loop and a thread pool. The timing loop runs in it's own thread and continually checks if a note needs to be played. It does this by comparing the system time against the time that the melody was started. The timing of each note can normally be calculated very easily from the tempo of the melody and the sequence of notes. As a new note needs to be played, borrow a thread from a thread pool and call the play() function on the note. The timing loop can work in various ways but the simplest is a continuous loop with short sleeps (probably between chords) to minimise CPU usage.

An advantage of this design is that the number of threads will not exceed be the maximum number of simultaneous notes + 1 (the timing loop). Also the timing loop protects you against slippages in timings which could be caused by thread latency. Thirdly the tempo of the melody is not fixed and can be changed by altering the calculation of the timings.

As an aside, I agree with other commenters that the function note.play() is a very poor API to work with. Any reasonable sound API would allow you to mix and schedule notes in a much more flexible way. That said, sometimes we have to live with what we've got :)

Answer 6

Looks fine to me as a simple implementation, assuming that is the API you have to use. Other answers cover why this isn't a very good API, so I'm not talking about that more, I just assume it is what you have to live with. Your approach will use a large number of threads, but on a modern PC that should be no concern, as long as thread count is in the dozens.

One thing you should do, if feasible (like, playing from a file instead of from user hitting the keyboard), is to add some latency. So you start a thread, it sleeps until specific time of system clock, and starts to play a note at the right time (note, sometimes sleep may be interrupted early, so check clock and sleep more if needed). While there is no guarantee that OS will continue the thread exactly when your sleep ends (unless you use a real-time operating system), it's very likely to be much more accurate than if you just start the thread and start playing without checking the timing.

Then next step, which complicates things a bit but not too much, and will allow you to reduce the above mentioned latency is, use a thread pool. That is, when a thread finishes a note, it doesn't exit, but instead waits for a new note to play. And when you request starting to play a note, you first try to take a free thread from the pool, and add a new thread only if needed. This will require some simple inter-thread communication, of course, instead of your current fire-and-forget approach.