Cykod Web Development and Consulting Blog

The word "Lean" has come a long way from being primarily associated with ground beef and toned abs. For people in the startup community, lean has become synonymous with one thing: The Lean Startup.

Combining the central tenents of both the Agile and Customer Development philosophies - many of the ideas behind the Lean Startup focus around one core idea: minimizing time through the "Build, Measure, Learn" loop.

When we started talking about doing GamesForLanguage we really only knew one thing: we wanted to build an online language learning tool. This idea had been percolating for a loooong time, and we had actually bought the first domain for the site, Games4Language.com way back in 2004.

How we were going to do that and what exactly such a tool would consist of, we didn't presume to know. We also didn't even know if people wanted to learn languages by playing games. Now, we strongly suspected they did given the popularity of gamification currently going on in all walks of life, but that was just a hunch.   Based on a quick survey of friends and family, we decided to give the idea a shot. 

Prototyping, round 1:

Our first step was to put something very basic in front of people and see their reaction - the goal was to see if people would play through a quick scene or get bored and quit. Since one of the most important aspects was to be able to build something and get feedback quickly and incrementally, we made one overarching decision: each level would consist of a number of "mini-games" that would be independent of each other and could swapped around. This was an absolutely essential decision as it allowed us to put something together very quickly and test individual bits without the context of an entire system (one our original idea was a much larger-in-scope RPG type affair)

We sat down, hammered out some ideas for basic scenes and then threw together a quick-as-possible ugly-as-sin demo in an afternoon.

I threw that demo up on a easy to get to url and we spent the next few weeks harassing everyone we could to go through any play the game while we watched. At this point all we had was a static shell of a game along with some recorded voices and some Javascript tying everything together. We didn't have any real data, but were trying to get a feel for what people liked and didn't like by shoulder surfing. Starting with 2 "games," we kept adding and modifying until we had 8different screens and an online recorder where people could record and play back their voice (one of the early requests from players).

Here's where we were after two weeks of prototypes (warning it will play sound on you):

We learned a lot from just watching a handful of people play with the prototype. The most important lesson, which gave us hope for the project was that people genuinely liked the games. The flip side, however, was that anything that wasn't in the form of the game caused people to lose interest on much more quickly. 

The lack of interest in anything that wasn't a game presented a little bit of a problem. We had intended to alternate between "teaching screens" - those explaining words and phrases - and "game screens" where users would practice. Well, we went back to the drawing board and decided we would see if we could come up with a solution and now the only screen that's not a game of some sort is the dialog itself. Event that will likely become a game later on when players have more vocab under their belt.

Protoyping, round 2:

At this point we went to the (literal) drawing board and our designer put together a fantastic look-and-feel for the game. Design partially in place, we went back and shoulder-surfed some more. The results were very encouraging for the basic premise of the project: that people like using games to learn. Furthermore, we realized that exactly what constitutes a "game" more or less comes down someone deciding that what they are doing is a game. If it looks like and feels like a game, it must be game.

Check out the music game genre (Rock Band, Guitar Hero, etc), which consists of pressing plastic buttons at the correct time - and yet an entire industry grew up around it because the type of game flipped a mental switch in people's heads that said "hey I'm a Rockstar" and they decided they were having the time of their life, pressing buttons at the correct time as dots float down the screen.

The clearest example we got was our "Word Invaders" screen - which was nothing more then a word translation game where we gave you a rocket ship instead of a mouse cursor to pick the matching words. People love the game, even though it's the exact same mechanic as clicking on a word, just dressed up with some visuals and animation.

Building a base of knowledge:

Confident that we had something people would at least try - our next step was to switch from "Hunch-based"-knowledge to "Numbers-based"-knowledge. The advantage is that the latter can be done more easily remotely and you don't have to rely just on your gut. The disadvantage is that you need to get your numbers from somewhere and that takes some traffic and some technology.

Using a lot of the features we had already built into our Webiva CMS platform we put together a system designed to take in as much data as possible and make it as easy as possible to update the games based on that data.

The first thing we did was put together a quick infrastructure on the back end to input information for the games - instead of doing anything to fancy for inputting the games, we mostly used textarea's as big input fields and parsed the text in there to turn it into the words and phrases needed for the games. That (and using Webiva) saved us a lot of time building an Admin interface that let our non-technical partners build up the games.

The second thing we did was record every piece of user data we could. Yup that's right - everyone who visits the site is a guinea pig. We watch you learn and learn from your learning. Every time someone plays a game, or skips a game, or scores 100% on a game - we track it. This allows to very easily see where people are getting hung up and giving up. We also ask users to fill out a quick survey after playing - giving us some insight into how user recorded data correlates with the user's experience.

This gave us a lot of actionable data that we were able to quickly turn around and use to enhance the site. Some things we've learned: Starting people with a game rather than dropping them into straight Dialog boosts completion percentages dramatically. Swapping a very basic "Listen and click on the right word" game with the current game called "Balloon words" similarly gave us a big uptick. Making sure we don't repeat the same game twice in a row - also important.

Paying it backwards

In order to get data from peole playing the games, however, we needed people actully playing our games and there's only so many times we can force friends to play. Luckily there's an easy way to find beta testers even if you have no PR or press - you just have to pay for them for visit your site using PPC. Google Adwords is probably one of the greatest tools ever for a lean startup. It let's you find out:

1. How many people are interested in what you're doing (based on common search terms and keywords)
2. What words and phrases get people to care (using the detailed results on which keyword phrases people are clicking on)
3. What segments of what your doing (in our cases what languages) people care about the most.

We're paying $30/day and getting enough data that we can reach statistically valid conclusions when we try stuff out on site, at the same time learning about our target audience based on the performance of our ads.

A,B,C...

We've had good success so far optimizing the pieces of our site to boost site interaction. One of our next steps is to add in support for A/B testing the various games - all while tying back to some basic numbers like "% correct" will let us see exactly how to step the difficulty to maximize user learning and retention - information that most people teaching languages would never be able to get without explicit questioning and surveying (ain't the internet grand?)

That's how we're keeping it lean down at G4L - try our French and German Language Demos  - Spanish and Italian are on the way (this post inspired by the Lean Startup Bundle for the Lean Startup Challenge)

With all the new releasing coming out - I thought it was a good time for a presentation on WebGL - below are the slides I presented to the Boston HTML5 Game Development Meetup on March 16. I also wrote a post on the state of 3D in Browsers on BostInnovation: 3D in the Browser: It's Go Time.