AI learns to balance a stick

[JoeWright]

I've written a programme using Newton, Irrlicht, genetic algorithms and neural networks to learn how to balance a stick.

Details and download at http://www.topracket.com/Badmintology/R ... orAi01.php




Next is a bipedal walker hopefully
Joe

[bal]

I guess you didn't make your physical update call frame-independant... As on my PC the sticks goes to the left and to the right at an amazing speed . A simple time scaler would solve this (I can post example code if you want).

[JoeWright]

I didn't because learning takes quite a while so on faster machines I want the simulation to run faster.

You should find that when it gets to good solutions it is less manic. In fact, even on slow machines it should charge about a bit with some of the poorer solutions.

Generation 10 has some good solutions.

Regards
Joe

[Guest]

I've been studying genetic algorithms, I haven't gotten into neural nets yet though. What variables are used control the hand?

[JoeWright]

The neural network takes four inputs: position.x and velocity.x of the top of the stick, position.x and velocity.x of the slider (hand).

The NN gives one output - a force for the hand. This ranges between 0 and 1 but is converted to +/- 500 for a force going in the x axis.

Joe

[Midnight]

thanks to you the robots in the future will be able to serve us alcoholic drinks without spilling them after first spilling 5000.... THANK YOU SO MUCH!!! lol

just kidding very cool little program but it raises the question...what is this good for besides a few mins of entertainment for geeks like myself?

[p-Logic]

THATs cool!

Best thing I made with neuronal nets is a program converting decimal numbers into binaeris.

How many layers has the net?
Have you used different nets to find out which one works best?

The increasing time thing is brilliant!

Next is a bipedal walker hopefully

Good luck!

[JoeWright]

Midnight: the programme is a precursor to my walker (and future stuff) and will have applications in computer games, CG animation, biomechanics, motor learning and development, sport, robotics, etc...

p-Logic: Its a feedforward network with one hidden layer (plus one output layer of one node). I haven't tried different nets. I will for my walker though, especially recurrent nets, perhaps spiking NN.

This is my first NN ever and second GA ever (first was a sport league scheduler) so I'm quite pleased with its success.

Joe

[katoun]

Hey.
Nice thing.
Bay the way I was looking on the Internet for some resources about ODE an I have found something it might help you(it about AI creatures but it is an research,with source code of course):

http://www.erachampion.com/ai/

Good luck.

[JoeWright]

Thanks for that katoun. I was aware of that site but when I had last looked the source code wasn't there so its great to be reminded of it.

Joe

[firefly2442]

Wow. Very cool. I remember reading in a fuzzy logic book that someone did something similar like this using fuzzy logic to control it so that it didn't overcompensate. Once it "knows" the solution can it save it so that it doesn't have to relearn every time?

[JoeWright]

Yes, in fact my programme stores each generation as a file (e.g. generationxx.dat). If you rename one of the later generations as Generation0.dat and run the programme you'll see that one replayed instantly.

I'm currently trying to get my walker working. I'm finally happy with the physical model but the AI evolution is going poorly. Oh well, I'll keep trying until it works.

Joe

[niko]

Wow, interesting app. I never tried out neural networks, really seems to interesting. And cool how the AI keeps getting better and is quite perfect after about 4 minutes, I think I would have needed some hours more for this.

[Bot_Builder]

Neat - this is a classic GA and/or NN example. If you add wall limitations and a location input you can sometimes get advanced behaviours which will flip upside down and bang into the wall. Especailly if you add speed into the fitness algorithm. You'll get ones that will manage to hit the wall at just the right speed so that they can speed off in the other direction while maintaining balance.

I quite like working with neural nets and genetic algorithms like this. At the moment I'm working on making an OCR program that will use a GA to classify pixel data as letters. I'm also thinking about using them to detect and remove dust, but it might be possible to do this without them.

Its a cool program, but in my opinion it's too easy to solve the problem humanly with some simple equations. Throw speed into the fitness factor, and even segments and it'll give you some neat results hard to duplicate with humanly written algorithms.

For everyone else, I'd suggest looking into stuff like this. Particularly those of you making FPS/other games with AI. Instead of making some lame AI that just turns towards you, and fires while strafing, you could generate some AI by pitting them against eachother and taking the winner. They might be too good, however. Throw in some additional aiming inaccuracy to increase realism.

Seems to me it will be increasingly used in software development for effectiveness (even for simple problems totally novel solutions often are created) as well as ease (for complex problems writing a execution base and fitness algorithm is way easier).

[JoeWright]

I agree. I've started simple just because I wanted to get the fusion of ga, NN, irrlicht and newton working. I'm currently working on a walker (biggest problem is getting the physics model right).

Joe