Ninja Academy
Cast spells in VR by holding hand poses in sequence. Fire, earth, ice. No controllers.
The Curiosity
This one is an exploration of hand tracking: whether hands on their own could be a real interface to a computer, rather than a novelty bolted onto one.
The honest source is Minority Report. The gestures in that film — the reaching, the sorting, the throwing of things across a wall — were what made me think, I want to build that. Two decades later the hardware was sitting on my desk. So: build it.
The Build
The first thing built was not the game. It was a diagnostic scene for reading a hand: every finger, every joint, laid out so I could watch the numbers move while I moved my hand.
A pose comes down to three quantities per finger. How far it is spread from its neighbours, which is abduction. How far the first joint is bent. And how much the whole finger is curled. Get those three and you can describe a hand precisely enough for a computer to recognise it.
The game sits in two layers on top of that. The lower layer detects a held pose. The upper layer watches for poses arriving in a particular order, and when the sequence completes, the spell fires. The idea comes straight from the hand seals in Naruto.
Four hand poses; hold three in the right order and something happens. A training yard with a ninja, a chicken and a samurai to practise on. The first test, in August 2023, was just the spell; by November the hand interactions and the pose sequence were working together.
The Result
The software works, and the demo shows it working. It stayed a prototype anyway, for reasons that had nothing to do with the code.
What I Learned
Playing it was frustrating. Holding three gestures in the right order is a lot to carry, and I know exactly how much because I could not do it either. I designed every pose and every sequence in this game and I still fumbled them, for hours. Once I had finally learned them, it turned fun.
That is the finding. The technology is there and it works. But hands as an interface is close to inventing a sign language with a computer, and a language has to be learned before it is worth anything. That is a ladder, and most people will not climb it for a game they have only just met.
Some people will. There are people on YouTube performing these seals beautifully, because they are devoted to the series the seals come from, and that devotion is what pays for the practice. A new game cannot ask for that on its own. Nobody owes a stranger's prototype several hours before the fun starts.