Word Lens - a real babel fish

Word Lens is the stuff of science fiction. You point your mobile phone camera at a sign in a language you can't read and the display shows it to you in a language you can. This is exactly what Word Lens does.

At the moment the app is only for the iPhone and it only translates between Spanish and English and vice-versa. It works without having to be connected to the net i.e. the app is self contained. What is amazing is that as well as doing the translation it also performs character recognition well enough to locate the written text in the image and extract the words. It then submits these words to a dictionary/phrase book and replaces the words within the realtime image of the sign with internally generated ones. That is, it is not only doing translation it is implementing it as augmented reality. 

The video demo (see below) makes it look really impressive but according to users' comments it isn't quite as foolproof as it appears to be. However, the general verdict seems to be that it is good enough for you to understand without any great effort or danger or getting it wrong - it would be another matter if it translated warning signs into something less urgent for example.

If you think about the subject domain then in fact the task of translation isn't quite as tough as free translation. After all you could in principle build a database of common signage text and it wouldn't be huge if stored as a tree structure. In many ways it is the realtime augmented reality that is more impressive as it delivers a fairly technical feat, i.e. translating text, in an attractive and easy to use form.

The app is free to download from the App store but you only get the augmented reality part of system which simply identifies and reverses words to show you that it's working. To see the translations in action you have to buy the dictionary for around $5.

o