New program lets devices double as pen
Forget sticky notes and smudged ink on the back of your hand. A group of Duke researchers have engineered a technology that lets users write notes to themselves in the air.
Eight months ago, Romit Choudhury, assistant professsor of electrical and computer engineering and computer science, and his team developed software that uses a phone's accelerometer to detect and record the movement of a person's hand. If a person writes letters, the software can decipher them into typed text that can then be sent to the user's e-mail account. The phone becomes a pen that can write notes in the air, he said.
Choudhury calls this invention the PhonePoint Pen, a play on the term "ballpoint pen."
"I used to keep forgetting things," he said. "I thought, wouldn't it be nice to have a pen with an accelerometer?"
Three years after Choudhury finished graduate school, he still cannot find a pen with an accelerometer, but several phones have them. And thanks to the accelerometer, Choudhury has helped develop a software that allows written letters to be recorded-no more sticky notes.
An accelerometer is a sensor that measures acceleration, which can then be used to calculate the distance something has traveled. It is the device responsible for switching the image from portrait to landscape view when some phones are turned horizontally.
Although many phones today have keyboards and many users use text messages, Choudhury said there is still a need for the technology to save and transmit handwritten notes.
"We want to get away from people having to text on keyboards," he said. "I hear lots of complaints about typing on a small phone."
Senior Sandip Agrawal worked with Choudhury to develop the PhonePoint Pen. Agrawal said he knows there are some people who can type on phone keyboards faster than he can type on his computer, but texting requires two hands-unlike writing with the PhonePoint Pen.
The device has not been perfected, and for its creators the current "pen" is only a glimpse of the vision they have for its future. Choudhury said even in its current state, the device has received a lot of attention from the medical community. He said the tool is great for people with speech and finger-mobility disabilities.
"This might do great for people who cannot type, cannot move their fingers properly," said Ionut Constandache, a computer science graduate student who also worked on the PhonePoint Pen.
Developers faced several problems while manipulating the phone's accelerometer to function like a pen, Choudhury said. First, a person writing with the phone has no global frame of reference. Choudhury described the process of writing the letter "A" as an example.
Writing an "A" involves drawing an upside down "V" with a line across it. With no frame of reference, the accelerometer cannot detect where the line crossing the "A" is supposed to be placed. In effect, the device cannot decipher the difference between an A or a triangle, Choudhury said.
Another problem involved dotting "I"s and crossing "T"s, Choudhury said. The accelerometer could not detect when a user lifts the "pen" from the "paper." The software attempts to combat this by measuring acceleration across the X-, Y- and Z-axes in a 3-D plane. When the "pen" is removed from the imaginary paper, there is an impulse along the Z-axis that would not be recorded as a letter stroke.
An additional difficulty was the ability of the device to discern between natural hand rotations and linear movements intended to be letter strokes. Technology exists that can combat this-Nintendo Wii controllers contain a device called a gyroscope that differentiates between both types of motions, which Choudhury said could appear in a phone in the future.
The device is still in its developmental stages, and there have been some concerns about the size users must write letters in the air, Choudhury said.
"[People are worried that] they might look like fools with their hands waving in the air and people might laugh at them," Constandache said. "Most people are excited-they like the idea, they think it's cool."
Choudhury said his team are working to shrink the size of letters that can be recognized.
Agrawal, who won the first Hoffmann Krippner Award for Excellence in Student Engineering for the PhonePoint Pen, said he would definitely use the device.
"Maybe not at what its current state is," he noted. "Right now it doesn't look that good, but if we can get to the point where you hold the phone like a pen."



