10

I am working on a project in Python to detect and classify some bird song, and I have found myself in a position where I need to convert a wave file into frequency vs. time data. This hasn't been too much of a problem, but to be able to classify the different syllables into groups, I need to write something that will detect when the data clusters into a certain shape. To give you an idea of what the data looks like, here is an image of how the data looks when plotted:

enter image description here

I need some way to get each individual syllable (each shape with a separation on either side) and save them either to a variable or to their own files so that I can run Pearson correlation between them using SciPy.

Also, I prefer Python, but I am open to coding in other languages if you have another way to do it.

Thanks!

Phonon
  • 5,216
  • 5
  • 37
  • 62
smckee
  • 201
  • 1
  • 3
  • I'm not completely familiarized with what I'm about to suggest, but looks like Continuous Wavelet Transform with a wavelet that contains the properties of your syllables is something worth to take a look at. – heltonbiker Jul 13 '12 at 20:06
  • Do you need to find a shape or you trying to classify birds? If so does using Hidden Markov Model's sound? – Mikhail Jul 14 '12 at 01:25

1 Answers1

2

Two questions:

1/ Near 8s, we can observe a stable pitch for 100ms or so, then a sudden increase dropping until 8.5s. Does this whole sequence (8s to 8.5s) form a single entity, or do you consider the two stages (stable then decrease) to be two entities?

2/ Do you want to work with or without supervision. Do you know in advance the "patterns" to look for?

  • If you want to work without supervision (say you have gathered recordings and aim at extracting a "structured representation" from it), your problem is akin, in a first step, to voice activity detection. Just use signal intensity, maybe in conjunction with a "pitchiness" metric (say the ratio of the maximum of the autocorrelation in the brid range, 1kHz - 5kHz here) to detect segments where there is an active strong pitched tone. Median-filter the resulting sequence to smooth it, and then threshold it to get the different segments. Once you have broken down your signal into segments, you can do interesting things with them. For example, you could extract for each of them a pitch trajectory (a sequence with the strongest frequency peak for each FFT frame, or something more robust extracted with a true pitch estimator), use DTW to compute a matrix of pairwise distances between each block, and use a clustering algorithm (k-means, agglomerative clustering) to identify groups of similar pitch patterns (the 8:8.5 and 10:10.5 segments). It is likely that an unsupervised approach will over-segment - for example 7.6:8.5 and 9.6:10.5 will be recognized as the repetition of the two same blocks, while to you they could fundamentally be one single pattern, but you could use something like Sequitur to have one level of structure higher.

  • If you have a predefined dictionary of "patterns" you want to label your signal with, you'd better follow the kind of approaches used for speech recognition, with the only major difference that speech recognition does not take pitch into account, while in your case pitch is the sole information to be considered! A speech recognition system tackles both the segmentation and recognition tasks in one single FST decoding operation.

pichenettes
  • 19,413
  • 1
  • 50
  • 69