5

Which solutions (standalone or plugins) exist for unmixing/de-mixing sources?

Lets say I have a track with acoustic guitar and string synth in mono, and I want each source separated. Which tools allow me to do this?

(note: before you say this is impossible, check out the research field of source separation algorithms)

Michael Hansen Buur
  • 3,802
  • 15
  • 31

4 Answers4

5

Quick answer - No I haven't heard of any but I've heard about research being done on the field.

The greatest theoretical problem with de-mixing is knowing what to subtract.

When we mix, we add A and B which are both known signals. But when we try to do the opposite, we need to know in advance what B is. This would render the process useless in most cases since, using your case as an example, if we already had the isolated guitar signal, we wouldn't need to de-mix it.

When we do know what B is, then it is a process as straightforward as mixing is. For example, there are plugins that will analyze 'hiss', clicks or any regular noise and subtract it (de-mix) it from the original signal.

I believe that's the reason we haven't seen much progress in de-mixing technology. It would take a lot of very complex self-correlation (comparing a signal to itself) to adequately identify complex sounds like voices or instruments.

Schizomorph
  • 1,573
  • 8
  • 20
  • 1
    Yet we seem to do it quite well everyday ;-) Something tells me that the approach of "de-noisers" is wrong. It is about continuity or content. Inside the field of image processing content-aware processing has been there for some time now. And thats "2D", not "1D" like audio (disregarding the stereo aspect).Someone must have been working on this... – Michael Hansen Buur Sep 04 '17 at 09:59
  • If only we knew how we do it, we could get computers to do it for us :) – Schizomorph Sep 04 '17 at 14:09
  • Yeah, if we know B, then we can remove to get A. But yeah, it would take a self-learning system I think. I wrote an answer similar to this but a bit longer a while back. It's here somewhere. – n00dles Sep 04 '17 at 14:18
  • 2
    Oh, it's here; https://sound.stackexchange.com/questions/40005/what-is-the-current-state-of-the-art-in-source-separation-what-tools-are-availa/40053#40053 Could be dupe actually. I'll comment on question. – n00dles Sep 04 '17 at 14:22
  • 2
    @MarcW Yeah, I've heard about self-correlation algorithms that are used to calculate S/N ratio (or something like that - it's been years since I heard this in a class and I've never used it). Maybe this would make a really good question @ https://dsp.stackexchange.com/ – Schizomorph Sep 04 '17 at 14:34
  • 2
    How many numbers did I add together to get 100? Can you specify what those numbers were? – Noctis Skytower Sep 04 '17 at 17:01
2

There's actually one software company that claims to provide credible unmixing : audionamix.

I am not affiliated with them, neither can I confirm their claims but there are demos on their website.

audionuma
  • 2,834
  • 1
  • 14
  • 26
  • Now we're talking :-) This looks interesting and definitely the sort of software I had in mind. – Michael Hansen Buur Sep 04 '17 at 12:15
  • 1
    This is very interesting indeed but I think I heard the same artifacts you hear when you push a de-noiser a bit much. If this is correct, then it's a very similar algorithm where you get the spectral 'fingerprint' of a sound and you use a set (like, hundreds) of very narrow BPFs to remove it. – Schizomorph Sep 04 '17 at 14:23
  • 1
    Personally, I wouldn't trust it until I tried it for myself. I've heard claims like this before. Imagine the explosion of remixes of old songs when this becomes truly usable. I'd definitely return to a few old projects that screeched to a halt! – n00dles Sep 04 '17 at 14:48
2

In concurrence with other answers; generally "not really feasible" is the answer. I've had a play with iZotope RX5, a very interesting editor; draws the signal as a 2D frequency graph and has tools for selecting regions (including changing regions) to isolate certain sounds and amplify or subdue them, but in the context I was operating it proved impossible to feasibly remove damage to an original recording of mixed sources (a telephone interview where a screaming, playing baby, a microwave and a background television had polluted it)

Your ears can "hear just the piano" so you assume that someone must have been able to make software that can "just hear the piano" and isolate it into its own track. No software does this well, or automatically; computationally it's a massive problem, largely rooted in sounds that aren't there in raw numbers form in the source stream, thanks to psychoacoustics and how your brain interprets the audio from your ears

More discussion: Can any software effectively remove one person/intermittent noises from an audio recording of a three party conversation?

Caius Jard
  • 121
  • 2
  • RX6 has added AI algorithms which are supposed to help with situations like the one you described. How well this works I don't know. – leftaroundabout Sep 04 '17 at 16:36
  • @leftaroundabout I have rx6 (and love it). It still doesn't handle that problem well, but better than anything else i've used –  Oct 19 '17 at 18:06
1

You don't state how your sources were mixed. If we are talking about multitrack session with sources panned in place (the kind of studio mix popular in the seventies), separation works very well.

If we are talking about a single multitrack session with master tape available where we want to deal with bleedover into the "wrong" microphones, stuff is still doable.

If you just have some "acoustic" recording of the whole band where the "mix" happens by being in the same room, stuff is pretty hopeless for enjoyable results: you can still achieve improvements if the aim is preprocessing for the sake of recognition tasks.

"Song with acoustic guitar and string synth" tells very little: does "song" imply "singing" as well? How many microphones were used? Is the synth mixed in as a digital track, as an analog input track, or was it also present in the acoustic signal? How much digital and redigitized and analog post-processing was done?

At any rate: in our current time copyright law is so abysmally stupid that you need to negotiate for every trivial sampling of everything, so if you are thinking about commercial remixing, you can try just to negotiate for access to the original master tapes as well if that would help.

So it might make more sense to restate your question in reference to a much more specific scenario: in "I have a song with acoustic guitar and string synth" is too unspecific with regard to "song" and too unspecific with regard to "I have": what kind of material/media do you actually have, and what kind of legal situation do you actually have?

  • Narrowed my example a little, but I think you miss the generic nature of my question. I'm not really interested in special ad hoc approaches, but something that works in general with a variety of sources. – Michael Hansen Buur Sep 04 '17 at 09:41
  • 1
    Even worse, then. Some separation can sometimes achieved by a phased approach and error and trial, aplying a variety of different techniques. A fully automated system is still pretty far away, I think. – José David Sep 04 '17 at 10:23