ChucK Brain

[Inventor]

Since I learned from kijjaz's technique how to create a neural net using ChucK UGens, I made one. It has no training yet, so it just listens to the microphone and outputs a few quiet clicks. i created a delay line of T duration and N delays, feeding into an NxM feedforward neural net.

Next I'm going to read up on some Neural Net training methods like BackProp, Genetic algorithms, or others. I'd like to find something that codes into UGens just like the forward network does, and perhaps BackProp would do that, I dunno.

---

ChucK_Brain2.ck
Description:	A beginning: A neural net coded with ChucK ugens	Download (listen)
Filename:	ChucK_Brain2.ck
Filesize:	2.33 KB
Downloaded:	755 Time(s)

---

[Inventor]

To make the backpropagation algorithm work, I'll need the derivative of the activation function as a UGen. I looked at Gen5 but it only works between 0 and 1. I need a Gen5 that creates the sigmoid function for example, 1/(1+e^(-t/tau)). Either that or a piecewise derivative of the limiter Dyno function... I'll look at the other GenX items, but so far no luck.

[Inventor]

Coded a complete backprop algorithm by feeding back errors with Gains and piece-wise defining the derivative. I put in kind of a flaky error function, which is probably preventing all learning, lol, but it's a start. I'm currently training it on loud rock and roll to see if it does anthing with the sound. My music buddy, haha!

---

ChucK_Brain3.ck
Description:	Chuck Brain, now with half a brain!	Download (listen)
Filename:	ChucK_Brain3.ck
Filesize:	4.64 KB
Downloaded:	751 Time(s)

---

[Inventor]

Finally some success. I ripped out the backprop algorithm and redid the feedback system, plus put in a genetic algorithm of sorts for unsupervised learning. Now the little net, only 3 nodes wide and 5 layers deep, listens to music and learns to play a rhythm. I trained it on "Star Spangled Banner" by Jimmi Hendrix, and it got into some mambo drum beat both times I tried. I think it's really learning. Not sure why a much bigger net slows the system down so much, but it does. Oh, and I gave each neuron a delay so they can learn time-based functions. Very interesting...

---

ChucK_Brain7.ck
Description:	Plays Bass Rhythm in response to "Star Spangled Banner" by Jimmi Hendrix	Download (listen)
Filename:	ChucK_Brain7.ck
Filesize:	5.46 KB
Downloaded:	741 Time(s)

---

[Inventor]

I'm playing around with these neural nets and realizing that I'm just not going to get a whole lot of intelligence out of them because the number of neurons is restricted by processor speed to be about 25 neurons. Still, I think that is enough to create a drum or guitar riff. I still need to add the cross operator to the genetic algorithm, establish neuron offsets, and figure out a better figure of merit before this thing can be said to work properly. Interesting though, as computers get faster this "brain" can have thousands or tens of thousands of real-time audio neurons, all learning by listening to music. I wonder what that will accomplish?

[soundcyst]

you might have better luck using Lisp in conjunction with ChucK to get more neurons.

i'm not sure if there are any packages out there that interface Lisp to OSC, but there are definitely MIDI-outputting interfaces.

[Inventor]

Yeah, good point soundcyst, I can bolt on some other language like lisp that's made for AI, or just code it in ChucK code rather than using UGens. What's slowing everything up as you can see is the fact that all those 25 neurons with their tangled interconnected inputs are processing sound at 44.1 kHz. I could be smarter about it and write a neural net that controls some audio generator which the net then tries to make mimic the sound source. Then maybe the neurons only have to fire 100 times a second or so, and I can get a few hundred *times* more neurons.

But this was an experiment in using the net in the data path and it served it's purpose. The best I get are moaning sounds in response to the rhythms of the music, which is something at least. Next time I get motivation, I'll try something with more, lower frequency, neurons.

[soundcyst]

actually, i hadn't looked at the code until just now =P

i'm getting some clicks about 30 seconds in, but that's about it.

... nevermind. much more interesting stuff after 90 or so seconds.

it looks like you're using gain for its gain values as well as as a neuron.. slick.

makes it more complex to write your own class though...

[Inventor]

Yes, for sure on that soundcyst, if I was to do it up right and all "classy", there'd be a weight class and a neuron class, a layer class, and a net class I guess. One cool thing is that the optimization process is genetic, so feedback can be introduced into the net as well as features like the delays or whatever.

[Kassen]

[Inventor]

I have been busy this week working on another neural net program, and this one is better than any of my previous ones. It learns how to say "electro-music.com" by training on an audio sample. First it trains for what is usually a farily long time, alternately playing the distorted input audio and the neural net output audio. Then it goes into playback mode and continues to alternate, but now increasing amounts of noise are added to the input signal. It does a good job of continuing to recite "electro-music.com" in response to the distorted audio until the noise swamps the input signal and the output of the neural net becomes unintelligible.

It is fun to listen to it train because it starts out just making synthy-sounding noise, and then it begins to pick up on the words and eventually it sounds like a vocoder effect on the input file. Then when the noise kicks in it demonstrates that wonderful characteristic of good AI, graceful degradation. I spent forever tweaking the code out and it could still use a touch of cleanup but overall it's in pretty good shape.

I could go on and on about all the little details, but suffice to say that after days of compulsive programming, I'm pleased with the result. Attached you will find the source code and the input file. Note that the code parameters at the top of the file are optimized for working with this particular audio file of me saying "electro-music.com", so try the program unmodified with that audio file and you'll get good results the first time. Oh, and by the way it's a real CPU hog, it burns up all of my CPU on a 1.25 GHz G5 Mac, though that is a slow computer by today's standards. If your machine is only a few years old you should have no trouble at all. Good luck and have fun!

---

Neural_Song_Learner.wav
Description:	The input audio file for the demonstration. "electro-music.com"
Filesize:	271.54 KB
Duration:	0:03
Viewed:	2019 Time(s)
▶ Play    ⇩ Download

---

---

Neural_Song_Learner6.ck
Description:	AI, getting smarter all the time!	Download (listen)
Filename:	Neural_Song_Learner6.ck
Filesize:	11.92 KB
Downloaded:	943 Time(s)

---

[Inventor]

Oh, and just for fun I put the following text in the source code file. You never know...

[Inventor]

On the subject of active AI programs, one that I thought of is a directory tree dweller. It would be a Perl script that uses the directory navigation and ls commands to navigate and map your directory tree. You don't tell it anything about it's environment except you program it with the ability to navigate up and down directory hierarchies and do the ls (dir) command to see what's in each directory.

You could program it with a memory or not, just make it so it doesn't get too active and use up too much CPU. It will just wander around continuously acting like a tree-dwelling animal like a monkey or a squirrel. Now that I think of it, one could launch several of these programs and have them interact in some way, forming a little society of AI programs hopping about your directory tree. Perhaps they meet up and travel in packs, then go their separate ways from time to time.

Maybe these programs could do something useful such as notify you of new large files or changes in the directory structure. Or they could scan for viruses or do disk integrity checks.