By Brian Hackett

Until now, Connectrix PDAs and, indeed, all other PDAs have been unable to recognize natural handwriting. The Graffiti system of recognition relies on users entering individual letters in a highly stylized fashion. For example, each 'k' must be written like '<,' and none of the 'i's or 'j's are dotted. To people unused to writing using a PDA, this is unintuitive, and entering a few sentences can become a laborious process at first. The Scribble system is a true handwriting recognition tool. It lets you write anywhere on the PDA's screen, and it will interpret and digitize the text within seconds. This increased flexibility makes Scribble invaluable for note taking, or any situation involving a lot of writing. Many of the advances made by Scribbler are made possible by the newest generation of more powerful PDAs, but what really makes Scribble work so much better than Graffiti? To answer this, we asked some researchers at Connectrix about the fundamental differences between the two systems.

Graffiti

Graffiti was developed in 1993 by Palm, Inc. as a simple recognition system that makes minimal demands on the PDA's resources. PDAs which use Graffiti have a small area at the bottom of the screen for entering letters. Letters have one stroke each – one continuous movement of the stylus – and are entered one at a time by repeatedly making the strokes in this area. The Graffiti software works by looking for patterns in the entered stroke: an 'i' is just a vertical line, while a 'j' is curled at the bottom. When you enter a letter into the PDA using Graffiti, it compares your letter to the many 'prototype' letters stored by the PDA. The prototype which most resembles your character is selected and printed on the screen.

The process of determining which prototype your letter resembles is called 'fuzzy logic.' Each prototype is graded based on how many of its basic features – corners, lines, or curves – are included in your letter. The prototype with the most features in common with your letter is selected; not all the features need to be present, hence the name 'fuzzy logic.' Graffiti is very good at recognizing single letters entered in a specific fashion, but with more powerful PDAs we can now build systems which recognize entire handwritten words. Scribble is the first of these systems, giving unparalleled flexibility in writing text using your PDA.

Scribble

There are several problems with Graffiti that make it impractical for use on natural handwriting. It only processes one stroke at a time, preventing it from understanding words where a single stroke is used for multiple letters, as with cursive handwriting, or single letters where multiple strokes are used for the same letter, as with many english letters like 'k' and 'r' (recent versions of Graffiti have corrected the second problem). Its fuzzy logic approach limits it to only a small set of prototypes: if you want to enter a capital letter, you have to enter a special stroke first, akin to pressing the shift key on a keyboard.

Scribble recognizes handwriting in a very different fashion, and this allows it to process entire words written by anyone in either cursive or print. It uses what is called a 'neural network,' a piece of software which has been trained to find words in print and cursive handwriting. Neural networks have been used successfully by Artificial Intelligence researchers for decades, and are very good at finding patterns in different types of data. They have been used to identify objects, recognize faces, and even drive cars around on highways (with someone in the driver's seat at all times, of course, ready to take over if necessary). These types of tasks are very easy for us to do, since our eyes have adapted to automatically organize our visual information. Neural networks emulate this ability by approximating the behavior of the neurons which make up our visual systems. One of the most important things our eyes do is allow us to quickly read letters and words, as long as they're in a particular area of our visual field (To see this in action, try reading a line of text with your eyes focused a few inches away on the page. You should be able to see the words, but you won't be able to read them). It is only natural, then, to build a system which uses a neural network to read and digitize handwriting.

The neural network used by Scribble simulates the behavior of several thousand neurons, and the strength of the connection between each pair of neurons determines how good the network is at recognizing different words. After setting up the network to guess randomly after seeing a word, it was trained by being fed hundreds of thousands of writing samples. For each sample, the network makes its best guess about what word was written. If it chooses wrong, the neuron connections are calibrated, changed so that similar words will be recognized correctly in the future. After seeing the same words hundreds of times, the network becomes very good at distinguishing them. The neural network in Scribble can easily identify thousands of common words, and it is very good at recognizing ones it has never seen before. It can even learn from its particular user, gradually getting better and better at reading their handwriting quickly and accurately. Scribble is the first handwriting recognition system to work on a PDA, and presents a dramatic improvement in the ease of use of Connectrix PDAs.