Notetaking tool for blind students levels the classroom

A notetaking tool dubbed the "Smartpen" is being developed to help blind students attend college engineering classes on an equal basis as sighted students.

A "smart" pen with an embedded microprocessor lets students feel notes written on sheets that raise up when stroked while students listen to annotated audio. Spawned by the inventor of the Leapfrog Leap-pad for kids, Livescribe Inc., the Smartpen will be available to college students in 2008.

The Smartpen enables students to annotate their notes with real-time audio, then later click on notes to hear a lecture. Adapting the Smartpen for use by the blind, by combining it with the Sewell Raised Line Drawing Kit, is the mission of Vanderbilt University and the Smith-Kettlewell Eye Research Institute under a grant from the U.S. National Science Foundation.

Optimized for engineering
"Engineering and science classes, which depend heavily on diagrams, graphs, charts and other figures, ordinarily put students with visual disabilities at a significant disadvantage," said Andrew VanSchaack, Livescribe's senior science adviser and a professor at Vanderbilt. "We plan to use Livescribe's Smartpen and Sewell's Raised Line Drawing Kit to make it easier for blind students to attend these classes."

The three-year project includes a phase that will explore current technology for augmenting tactile information with audio. A second phase inlcudes quantitative tests of the most effective ways of using the Smartpen with the drawing kit. In the third phase, researchers will observe blind students in order to debug algorithms.

First conceived in the 1980s, augmenting braille-like graphics with audio annotation for the blind has been implemented in many versions. The Smith-Kettlewell Eye Research Institute already has a project underway to use the Talking Tactile Tablet as a platform for tactile maps. Tactile mapping is either vacuum molded in plastic or printed on an embossing printer, then placed on the touch tablet. The blind can then feel the map and push it to trigger audio telling about a location.

Since the tablet is tethered to a PC, the blind can only use these maps at home to plan a trip. By developing a Smartpen-based version, a portable navigator for the blind could be printed in notebooks of tactile maps with audio descriptions triggered with the battery powered Smartpen while standing on the street corner.

"The Talking Tactile Tablet works well, but the Smartpen offers portability and is easier to use. We plan to layer all kinds of information that will be very useful to blind people trying to navigate in the world," said Joshua Miele, a research associate at the Smith-Kettlewell Eye Research Institute."

Java inside
The Smartpen uses a microprocessor running Java. Livescribe offers a software development kit. The Smartpen has a microphone for recording audio, a speaker for playback and a camera to read the gray pattern of dots on the paper. Software developers print their software creations on Livescribe dot paper, then activate their algorithms by clicking the Smartpen on their printed graphical symbols.

"Another huge advantage of the Smartpen for the blind is that they can explore graphics with their hands and query them by clicking at the same time," said Miele, who is blind.

The Smartpen can hold up to 100hrs of audio and plays back through its own speaker or through earphones. Software for particular applications is uploaded to the pen through a USB port. The Smartpen will retail for $200. The Sewell Raised Line Drawing Kit is available now for $40, with 70-sheet refills priced at $10.

- R. Colin Johnson
EE Times