Saturday, September 8, 2012

Complex Gesture Recognition using Kinect

Like I said earlier, I was working on some complex gesture recognition system that can recognize numbers. The whole system was built with Embedded systems in mind, so its just c++ and no dependencies. The system can recognize any pre-trained gestures with very good accuracy.

We developed this system at Muckati (

Team: Anjan, Harishankar, Sasikumar

Saturday, July 21, 2012

IrrNaCl - my port of Irrlicht 3d engine to Google NaCl

I've just completed porting my favorite 3d rendering engine to Google Chrome's Native Client. It was a tough learning curve for me from both sides. I was new on both sides, NaCl and Irrlicht. However, as both were on my favorite programming language c++, I felt happy in doing it.

The source code is available here

Tuesday, June 19, 2012

Gesture recognition using Kinect on Embedded system

For those who are still following my depth sensor and skeleton recognition research. I've successfully designed a system that can track users hands (and other parts) with my own program. My initial intention was to build a very minimal micro computer with linux kernal running on it, to do gesture recognition. So, I was not happy in adding MS SDK for kinect or OpenNI as my intention was to build a commercial device.

Now, my program is able to filter Remove Background from Depth data with very minimal CPU usage. The system is still able to run with 30 FPS on a very basic CPU available today. Now, after detecting pointing finger (or hand) I am trying to add a self-learning system to recognize custom gestures.

Hopefully, I'll soon add a video of gesture recognition using a simple a micro computer setup.

Saturday, January 7, 2012

3D Scanner 2.0 - Mac App update

For all those who had trouble in the old version, I guess I've fixed most of it by now. Now we have the next big update here, just uploaded it and waiting for Apple to approve the update. Though, you don't see any extra features in this version, the main intension of this version was to improve the quality of OpenGL rendering, background filtering, allowing user to control the thresholds that were previously hardcoded.

3D Scanner - Mac App Store Link

If you feel this update does not deserve a whole new version number, you need to wait until the next updates that will fill advanced features. Also, keep posting your feature requests, so we know what you expect. :)

Wednesday, January 4, 2012

3d Scanner 1.3 update

I just uploaded an update for "3D Scanner" Mac app. The new update is just a 32-bit support, so old Mac machines are not left off. I received few emails with issues in running the app on their pretty old machines, that I doubt should be because of the 32-bit processor. So I took a hard path to build the open source usb driver code from source compiled it along with my program. Now most Mac OSX machine with OSX version greater than 10.5 should work fine. But it is still restricted to intel based machines.

The next update is not too far, am working on noise removal, which should help the program a lot. Also my next assignment will be 3D video recording in md2 or other 3d formats. :)

Friday, December 30, 2011

Google AI Challenge Ants 2011 - Post Mortem

As a 3 week hard work and sleepless nights, I took the 127th rank in Google AI Challenge Ants 2011 and 1st in my country India. I thought the experience was worth sharing in this blog. In-between the competition, I had some health issues that kept me out of reach of computer for a week. Of course health was worse after the competition started. However, I managed to make it normal with a week long vacation with family.

Initial Idea:
Initially when I first learnt about the competition, I thought it would be a simple problem to solve. The first thought that came to my mind was grouping ants together to fight enemies. Because, the aim of the competition sounded more like, just keeping your ants live. So I thought I could build a group of ants making a + sign with just 4 ants each.

But eventually, when I started working on the game, I learnt, it affected exploring and food gathering a lot. And decided to concentrate on food gathering and exploration first.

Exploration and Food Gathering:
I tried a different way of using scent maps for food gathering and exploration. Totally, I fill 3 different scent maps depending on priority. The first map is for food and exploration. This map will carry a scent until it finds the first ant, then the scent will stop spreading on all sides. This will ensure one ant for one target policy. This way of spreading is unique for food and exploration map. I tried this idea to avoid many ants moving towards the nearest target which is usual for scent maps. To the map, I add maximum integer to the source cell. Food and unseen cells are my initial source seeds. This will make it easier to explore as fast as you can. But there is a problem with this technique, the ants will not re-explore visited areas to search for new food. So I started seeding the cells those were not visible for the past 6 turns. Why 6 !?!? coz adding the digits of 42 gives 6. In simple words, I do not have a answer why I chose 6 but it worked out better than other numbers. 

To decide on the number I could have tried a simple genetic algorithm, but I didn't have enough time to setup local game servers for that. I'll keep such tasks for next contest. However, I tried letting same bots with different numbers (4 to 12) to each other to decide on this number. 

Finally, I can see my bot exploring and gathering food as fast as any top bots, irrespective of any type of maps. The scent maps helped a lot, especially after choosing a slow programming language like python.  At least 5 times, I considered poring my code to different language during this contest. 

Best Example for Exploration and Food Gathering: Game Link

Combat and escape:
After settling with a decent exploring and gathering code, I decide to choose the most efficient way of fighting with other ants. I first considered random sampling method explained by user a1k0n in the forum. However, it didn't impress me a lot. So I decided trying my own simple battle resolution method using 2 radii calculations. However, after implementing my method I learnt, I might have to calculate 3 radii for battle to resolve efficiently. And my code time-out even with 2 radii, so continuing this might be disastrous with the language I chose. So I decided to take another route by Memetix, in which all battle resolutions were pre-calculated. Now, my ants do not die in 1 vs 1 fights. 

Best Example for Combat: Game Link

3 scent maps
Like I said before, I used 3 scent maps according to priority. All ants first check the 1st maps that can is called hill scent map and is seeded with just enemy hills, if found. The scent travels to a distance of
The reason again being (42 => 4 + 2 = 6). The next priority maps is the food map that travels until it finds the nearest ant. The third map is just a worst case map, that is filled with enemy hills, food, and unseen cells. This map makes sure no ant is left idle, in any case. 


With a decent exploration, food gathering and combat resolution code, it was easy to reach top 500. But my aim was to get into the top 100 list which might need a lot more heuristic improvements to my bot. So I considered trying different strategies to boost the performance. One very important improvement I did was blocking the path for the enemies to enter my territory. All that I did to bring this is, seeded enemy ants and all ants including ally will block scents, so until the opening in the maze is closed the enemy ants will generate scent. And this scent is sent to the 3rd map, if the ant has no scent in 1st and 2nd maps.

Overall, the performance was much better at this point. However, my bot performed very poor in close neighbor multi hill mazes. So I decided to tune my bot to do some sacrifices near my hill within a certain radius. The radius, I chose was 14 (coz 42 was divisible by 14). This simple change in my combat code that does sacrifice ants in 1v1 combats saved my hills from being visible very early. So I my ants had enough time to breath and collect food.

Best Game for hill sacrifice: Game Link

Also, my bot sacrifices some of its hills to concentrate on exploring and food gathering. I didn't make it intensionally, but it was a bug in the other strategy but it was worth keeping it. However, some final change made the performance worse on other maps, which I didn't care much as I had no enough time to think about tuning further. I left my bot with that and sat back to watch it play.

During this competition, I learnt a lot of new topics that would help in future. Some of the topics were very surprising to me. I felt those topics were very much for scientific experiments and not for gaming. However, this contest changed my thought about such topics.

  • A* Shortest Path
  • BFS Shortest Path
  • Collaborative Diffusion
  • Scent/Distance Maps
  • Heuristic Decision Making
  • Random Sampling
  • MiniMax
  • Genetic Algorithm
  • Genetic Programming

and a lot more. Hope I'll keep those new interesting topics for my next contest. Lastly, I would like to thank the contest organizers, TCP serve hosts, Tool designer and co-participants.

Saturday, October 29, 2011

Hacking iPhone 4 into a Digital Microscope

I was always eager to zoom in to things to see if I can find microbes. This time I took a step forward and opened up one of my old DVD drives to remove the lens inside. I successfully removed lens from 2 old optical disk drives and both had different magnifying lenses. With one of the lens I could see the RGB component LEDs from every pixels of my iPhone 4 retina display.

Click to see full sized picture

Then I have now fixed it behind my iPhone 4 camera to shoot some pictures. See the following video that we compiled during the weekend.

I am planning to increase the magnification of the setup by adding 2 lenses in front of the camera to see if I can zoom into microbes level. My first target is to shoot a moving Amoeba.