ablet PC and Mobile PC development is very popular today, and will only get more popular and important in the future. For Mobile PC developers, there are significant changes coming in the near future, some in the Windows XP timeframe, others in Windows Vista. This article provides an overview over what's on the drawing board or already available in technology preview builds.
Since they were first released, Tablet PCs have gained significantly in popularity; as time goes on, they will move more and more into the mainstream. Currently, Tablet PCs run on a specialized version of Microsoft Windows XP. In the future, all Windows operating systems will support Tablet PC fundamentals, such as digital Ink.
Mobile PC development has been important for a long time. In fact, in May of 2005, notebook sales surpassed desktop PC sales in the US for the first time ever (for a full month), and they have not given up their lead since then (nor are they predicted to). These devices are used in an ever-growing variety of scenarios, and expectations from features, such as extended battery life, are on the rise.
The Tablet PC and Mobile PC scenarios are of strategic importance to Microsoft, and as a result, many exciting new things are scheduled to release with Windows Vista and with new versions of the Tablet PC SDK, currently available as a CTP (Community Technology Preview) and which also runs on Windows XP.
Software developers are usually most interested in what software can do, which SDKs are available, and how they can use these components in their own applications. But in Mobile PC and Tablet PC scenarios, the hardware is equally important, as it is part of what drives new scenarios and opportunities.
|In May 2005, for the first time, notebook sales surpassed sales of desktop computers.|
One of the obvious next steps is for devices to get both bigger and smaller in physical dimensions to supplement the currently available devices. On one end of the scale, this means that devices will have much larger displays with higher resolutions. On the other end of the scale, devices are getting smaller, moving toward the domain that is currently solely occupied by Pocket PCs.
A development that may be less obvious and less expected is the move toward touch displays. Conventional Tablet PCs use electromagnetic digitizers to sense the tip of the pen on the display. This has some advantages; for example, it lets hardware manufacturers mount the digitizer (the device that senses the pen) behind the LC display, providing a better and brighter display quality. But there are also disadvantages, such as that you always need a special pen that the EM digitizer can sense. If lost, replacing the pen can be expensive, and even worse, it's not possible to interact with the device without the special pen. On the other hand, it is possible to double-tap an icon with a finger(nail) on Pocket PCs (which typically use touch-sensitive digitizers).
Touch screen Tablet PCs provide an additional option. The reason we have not seen them before is that there are some technical hurdles that need to be overcome to provide touch-sensitive displays. The problem is that people rest their hands and arms on the large Tablet PC displays while writing. This is not a problem on smaller devices such as Pocket PCs. On larger displays, it is important to differentiate between a tap of a fingernail intended to select an icon (or interact in another meaningful way), and an accidental tap or palm pressure that occurs accidentally while writing with the pen. The software has to do a lot of intense processing to filter unwanted "noise" from meaningful interaction data.
Touch devices have the disadvantage that hovering fingers can not be detected the same way a hovering electromagnetic pen can be. Hovering (or any mouse movement that is not a click) is therefore not natively available on touch devices. Microsoft provides software that makes up for that shortcoming.
For developers, these new devices provide new opportunities, and they also create some new challenges. Higher screen resolutions in Windows Vista will rely on resolution-independent display technologies handled through vector graphics and exciting new techniques and APIs such as the Windows Presentation Foundation (WPF) formerly known as Avalon.
Smaller form factors also require some developer consideration. User interfaces that may have worked well on 1024x768 displays may not work well on smaller devices, so developers may have to account for that in their applications. Using techniques such as flow layout (a layout option available in WPF) are helpful in these scenarios. Using this approach, user interfaces can re-flow to accommodate smaller displays without the need for the dreaded horizontal scrollbar, which is known to not work well at all.
New Tablet PC SDK Features for Windows XP
The Tablet PC SDKand its ability to make very complex tasks such as handwriting recognition ridiculously easyhas always been nothing short of amazing. Microsoft keeps pushing the limits. A recent Community Technology Preview (CTP), made available in September 2005 targets the Windows XP-based Tablet PC operating system and provides some amazing new features, particularly in the area of Ink recognition.
|Tablet PCs will be available in new form factors, both larger and smaller than current devices.|
|Figure 1. Analyzing Ink: A sample application uses Ink Analysis to identify paragraphs, bulleted lists, and even graphical elements such as rectangles.|
The most exciting new feature is Ink Analysis
, which is Ink recognition on steroids. Although simple Ink recognition is based on a collection of strokes, Ink Analysis takes a more sophisticated approach, applying both Ink recognition and Ink parsing at the same time. These two aspects are worthy of further explanation.
Ink recognition (or handwriting recognition) refers to the task of recognizing handwriting stored in individual strokes of Ink, and returning it as strings of text. Handwriting recognition is usually tied to a specific spoken language and culture.
Ink parsing refers to two further distinct tasks: Ink classification and layout analysis. Ink classification refers to the division of Ink and strokes into meaningful entities, such as words, lines, and paragraphs. It is also possible to designate certain areas of a larger recognition area, such as a title area in a notepad-style Inking area.
Layout analysis determines the position of Ink and how it relates spatially and semantically. For instance, layout analysis can determine that certain parts of a document are circled and connected to a paragraph, making it an annotation. This allows for advanced features, such as keeping annotations with a certain part of the document, even if the document is re-formatted.
|Figure 2. Analyzing Ink: The ink content after analysis and recognition of text, shapes, and bullet points.|
Ink analysis also allows for shape recognition, a feature that was previously very difficult to implement. This feature can be used to recognize simple primitives, such as rectangles, circles, different types of triangles, trapezoids, and more (about a dozen, all in all). This ability is very useful as the basis for many non-text recognition models, such as the recognition of flow diagrams and the like. The provided default set of recognized shapes should handle many needs. Figure 1
and Figure 2
show examples of Ink interpreted through Ink analysis and exported to MS Word. And there is an opportunity for third parties wanting to extend that default set.
For some examples of what's possible with Ink analysis, see the "Ink Recognition and Ink Analysis
" article in this issue.
Another important new feature is the COM-based Real Time Stylus support. In previous releases of the SDK, the Real Time Stylus API was only available for managed developers. C++ developers, and other COM-based developers such as Visual Basic 6 developers, could only use interop to use the Real Time Stylus. This, of course, defeated one of the fundamental goals of the Real Time Stylus-performance-and has now changed with a native (COM-based) implementation of the Real Time Stylus API. (For more information on the Real Time Stylus, see the article "Get Control and Performance with the Real Time Stylus API.")