finding the frequency of audio input

[andrewjmonks]

I'm trying to use my theremin as an interface for ChucK. To do this, I'd need to be able to find the frequency of the adc's input. Because it generates a simple sine wave, it should be easy(er) to accurately find the pitch of whatever I'm playing.
However, one area of ChucK that I really haven't explored is manipulating audio input. I feel like there should be a way to do this with UAnae, but I've been digging through the UAna reference, but I haven't had any luck.

Where should I look now?

[Inventor]

Hi and welcome to the forum. You can do this with an FFT or (preferably) with a ZeroX.

For the FFT, you figure out what the frequency of each bin is, find the largest frequency with a for loop, and multiply it's index by the bin frequency. The problem with the FFT approach is frequency accuracy as you can probably only do FFTs of 1024 or 2048 bins before your computer gets too slow to track in real time. So that's all the frequency resolution you get.

For the ZeroX you set up a while (true) loop spinning on samp => now and count the sample steps between positive ZeroX pulses using the .last() member function. Then you calculate the frequency with sample accuracy. It's also cpu intensive to spin on samp => now, but you're not doing much in the loop so it's OK.

Hope that helps.

[andrewjmonks]

[Inventor]

Here, I made your code work:

[andrewjmonks]

Beautiful, thank you!

One more question, though.
It works great as long as it's getting a good signal, but when it isn't, it just gives out garbled nonsense. I think if I added a simple gate so that it only tried to analyze signal that was above some level it would be perfect.

I think I would be able to code it myself if I knew of a function to find out how loud something is. What's a good way to do that?

[andrewjmonks]

Never mind, I guess that's just more last().

[Inventor]

[Kassen]

If you just want a noise gate Dyno is likely the most simple option; it comes with a noise gate preset. There is a example for a amplitude tracker in /examples/deep that may help as well, it works slightly different from Inventor's.

Another way to prevent garbage from FFT-based pitch tracking might be to simply ignore the signal if the highest value in your FFT array is below a certain value? That would be quite simple indeed with very little computational cost.

[andrewjmonks]

Thanks for all the help.
I think I'll use something like /examples/deep/follower.ck

[Kassen]

I think, BTW, that we could do a zero-crossing-detection-based pitch follower in a more easy way as well. I'll report back with my findings.

[Kassen]

Here we go. Only using UGens to track pitches of simple signals. Not at all suitable for complex signals but it'll track whistling. Will likely be suitable for a Theremin's simple signal but clearly you'd need to add volume tracking as well or there will be garbage (due to adc noise) when the Theremin is silent.

[Inventor]

Spiffy trick, Kassen, nicely done!

[Kassen]

Thanks! I like how many of the UGens are very basic but you can join them up to do quite advanced things.

Here I added comments and functions to make it as accessible as I could but when it comes down to it this is very little code for some quite advanced functionality. Basically 4 UGens and a few settings will do pitch tracking; that's quite nice.

I wonder if it's good enough for Andrew's problem. the Theremin should have a very simple signal but I'm not 100% sure exactly *how* simple it is. It does lag a bit behind the input but it's fundamentally impossible to track pitch without lagging at least a bit and at least this is smooth.

[andrewjmonks]

Wow, Kassen. That works really well.
I never would have thought to find the average that way. I'll definitely play with that some more.

I ended up using ftom() for a bit of pitch correction. I found that it made it sound much less noisy. Also, it sounds cool:

[Kassen]

Great!

I tend to try to use UGens instead of loops running at a 1::samp rate. In most cases that leads to CPU savings but of course you can't always do that. For example; you are using a modulo here and I don't know how to do that in UGens. I wonder how far simple additions (using a "Step") and multiplications (using a "Gain" set to multiply) would get you here but it's not completely clear to me what sort of correction you are after here.

There's yet another way to do pitch tracking, BTW, that is called a "phase locked loop". That might also be suitable but I will have to read up on it again and see whether that can be done efficiently. If not it might be worth considering trying to add that method as a UGen to our "blob" project for home-made UGens.

[DrJustice]

I'm not a ChucK user, just a lurker, so this may be irrelevant, but:

If Chuck (sorry can't be bothered to type that last haxxor'ed K, even though this paranthesis was a lot more typing ) has a complex FFT, then consecutive FFTs can yield exact partial frequencies, i.e. between the bin frequencies, by unwrapping the phases.

DJ
--

[Kassen]

Yes, we do. In fact, chuck (see; it's no big deal ;¬) ) applies it's "strong timing" to the FFT control, meaning you can pull a frame at any moment you want, this need not be a steady rate either, in fact the rate can depend on the results of previous analysis outcomes without a lot of work. Conceivably you could take one frame per sample though obviously that won't run in realtime any time soon. I seem to remember that with some undocumented trickery you can do custom window shapes as well.

So yes; we can, but I'm not quite sure what you mean by "unwrapping the phases". We do get the phase; we get a amplitude and phase as a series of complex numbers in a array.

I think that means we are good to go and I don't think there is anything like that in the examples. If you can explain the technique to me I'll try porting it and we'll see if we can get it in the official examples as that sounds quite useful.

In general I think it's good to build up a bit of a library of methods of pitch detection/tracking as most forms of pitch detection tend to suck in general. There is no guarantee at all that any easily detected frequency in a signal will also be the perceived "pitch" according to the human ear. I feel the "trick" is to line up a suitable method with the type of material you wish to treat so a larger toolbox will definitely help a lot there.

[Kassen]

Please note that I made a small edit to my example above. I hope this didn't confuse or bother anyone.

[DrJustice]

Kassen, the phase unwrapping is what a phase vocoder performs to track an instantaneous frequency per bin.

The principle is simple: you look at the same bin in two consecutive FFTs, and if there is a phase difference you know that the frequency did not fit precisely in the bin. If the phase has advanced the frequency is higher than the bin frequency and vice versa.

In practise you convert the complex spectrum (or just the one bin we are looking at) to a phase and magnitude spectrum. Then to find the phase difference, you just subtract the phase from the previous FFT frame (store the phase each time around so you can pick up the previous one). Then you unwrap the phase difference so that it is in the range -PI to PI. The unwrapping part here is just subtracting or adding PI/2 from/to the phase difference until you are within -PI to PI. Now you can get the frequency with something like:
freq = phasediff * factor + bin_no * fundamental
where:
factor = sample_rate / (2 * PI)
fundamental = the lowest bin frequency

This is the simplest method, there are more esoteric methods out there too. If you want I can dig up and mail you some phase vocoder C sources, or you can search the net for "FFT phase unwrapping" and "phase vocoder". Also have a look at this for a few more possibilities (including the above method).

DJ
--

[dewdrop_world]

In supercollider-land, I've gotten the best pitch tracking results from Nick Collins' Tartini ugen, part of the sc3-plugins sourceforge project.

[Kassen]

DrJ; I think I get it. Do I understand correctly that this is assuming that the pitch we are trying to track will never be a "misfit" for the bin by more then a cycle? I'm not sure I understand why that is a safe assumption (but then there is a lot of stuff in the details of the FFT that are arcane to me :¬) ).

Also; in a practical scenario we'll probably have overlapping frames, say with 50% overlap, in that case we can simply double the phase difference, then perform the same math, right?

James, thanks, that looks very interesting. A quick peek at the source told me that porting that is completely beyond me though. I'll start with something a little easier as a first go at new UGens but it *does* sound like a very nice thing to have. Maybe the current interest in MIR will make Ge or Rebecca have a go at it, based on Nick's work (looking at that source I was feeling it would be really nice if somebody else would've already pulled all the actual analysis bits out of there....).

I have to pick my battles and right now I'm a little more interested in more advanced analysis of controller signals then of audio signals.

[DrJustice]

[DrJustice]

Kassen, in the first example I think you can do :
factor = sample_rate / ((hop_size / n) * 2 * PI)
to factor in the overlap. Not tested though - I may be wrong!

DJ
--

[Kassen]

Check.

I don't think it's that hard. I mean; if the phase differs by X over half a frame it should differ 2·X over a whole frame.

That's not the side of math that I'm scared of; with a system like this we can easily generate tuning signals at a known frequency to verify our attempt and re-trying means re-compiling which is a hot-key and will be done in the blink of a eye. It's when the Greek symbols start flying that I hide below the bed.

One of my friends is a professional mathematician and he'd likely frown at that sort of thing but then I can just do this in secret with the curtains closed and the door locked :¬).

Thanks, I'll give this a go over the weekend.

[radarsat1]

someone needs to port Puckette's fiddle~ to ChucK. ;-)