If you have followed along with all the examples in this article, you may have noticed that they all work fine, but all the rendered output is somewhat-shall we say-fragile. Although all the rendering works well, it is not persisted. If you draw on any of the forms, and then minimize and bring them back again, the drawings are gone.
That's because so far, you only drew on the form whenever the pen caused events, but you do not listen for any re-draw events that the Windows operating system may indicate. In particular, each window raises a Paint
event whenever its contents need to be re-drawn, and because you completely ignore that event on all the examples, they all end up with blank forms whenever they are refreshed.
To solve this problem, you need to add one very fundamental step. You need to store the information you receive from the pen, so you can use it to re-draw all the Ink whenever it needs to be. There are a number of ways to accomplish this task. For instance, you could simply take all the data points you received and store them in an array or collection. You could then use the Paint
event to iterate over all the collected points and draw them again.
You can also take advantage of Ink storage provided by the Tablet PC SDK. This has the advantage that you end up with a standard Ink structure in an Ink object, including all the related functionality, such as Ink serialization and Ink recognition. In fact, there are even standard objects that help us to render Ink that is stored in an Ink object.
Ink data stored within Ink objects is maintained in a collection of strokes. Rather than simply collecting all data points in a single large collection, Ink objects store a series of points on a stroke-by-stroke basis. This means that you need to determine when a stroke is started, which happens when the pen is put down on the digitizer. From that event on, you need to store all points (including the touch-down point of the StylusDown
event) in a temporary collection of points until the StylusUp
event occurs. You then take all the points and move them from the temporary collection into a new stroke on the Ink object. The tasks required to achieve this are straightforward and can be seen in Listing 8
The only part that requires extra explanation is the call to the CreateStroke()
method, in particular the second parameter. This parameter is a collection of properties associated with the stroke. The easiest way to provide this parameter is to use the default set of parameters associated with the Real Time Stylus.
At this point, you have collected the Ink but you have not displayed it anywhere. One easy way to render the collected Ink is to use a Renderer object, which is also provided by the Tablet PC SDK. You can simply call this helper object in the Paint
event of whatever control you want to draw on:
Protected Overrides Sub OnPaint(ByVal e As _
Dim rend As New Microsoft.Ink.Renderer
There are two problems that make this approach incomplete. For one, the Ink object's Strokes collection only contains each stroke's data once it has been completed. In other words, the information needed to render strokes is not available while the pen is still touching the display. It only becomes available when the pen is lifted up. This does not reflect a real-world writing experience accurately, because it means users can't see the Ink while writingit becomes visible only when the writing process has been completed.
The second problem is that the Paint
event will not fire unless Windows detects a good reason to refresh the display, which is only the case if the window gets minimized and restored, or is partially or fully obstructed and brought forward again. You could force the window into a refresh by invalidating its content. However, this is not a good option performance-wise.
These are clearly problems you need to solve. You have already solved the previous problem, which is Ink persistence and Ink re-rendering when the window or control refreshes. So this clearly is useful, and all you need now is a way to dynamically render the Ink while it is still being drawn. You could manually draw the same way you did in earlier examples, but there is an even simpler way. Microsoft provides a standard Real Time Stylus plug-in called the DynamicRenderer, which can be registered in addition to your custom plug-in. Here is the code that initializes both plug-ins:
Public Sub New()
'This call is req. by the Win Form Designer.
'We create the stylus
Me.rts = New RealTimeStylus(Me)
' We create the plug-in
Me.stylus = New InkStylus(Me, Me.rts)
' We create a default dynamic renderer
Me.myDynamicRenderer = New DynamicRenderer(Me)
' We enable everything
Me.myDynamicRenderer.Enabled = True
Me.rts.Enabled = True
This already works pretty well, but you can enhance the code a little further by making sure the dynamic renderer never interferes with paint events. You can do this by calling the dynamic renderer's Refresh()
method in the paint events. The following snippet shows the completed Paint
Protected Overrides Sub OnPaint(ByVal e As _
' Make sure the new Ink is refreshed properly
' Render Ink in the Ink object
shows all the code required by the form that uses this combination of plug-ins.
At this point, people could point out that you have spent a lot of effort on re-creating what the default InkCollector object would have provided (or in fact, even a little less). This is true for this example, but keep in mind that Real Time Stylus technology affords a lot more control than the default objects.
For instance, you could combine the last example with the input-area restriction shown earlier. It is also possible to apply logic to the data that gets collected. For instance, Tablet PCs collect a lot of data very quickly and at a high resolution. This is great in most scenarios, but the sheer amount of data can also be overkill. Using Real Time Stylus plug-ins, you can decide to collect data at a lower rate by dropping packets that are very similar to previously collected packets, or you can apply other filters.
All the examples so far operate synchronously. This means that data is collected by the digitizer and then it is passed to the plug-ins that process it; once that is completed, more data is collected and then it goes back to the plug-ins again. This loop is continued as long as more data is available.
It is the ultimate goal of the Real Time Stylus API to process data as efficiently as possible in real time. This is not always feasible or even required.
Consider the most recent example that collected Ink in an Ink object. Was it really necessary to store strokes in real time? No. As long as you are able to dynamically render new Ink in real time, it is not overly critical how quickly Ink gets stored in the Ink object, as long as it is available when the Paint event fires the next time. So a slight delay in Ink collection is acceptable, and any performance gain in the real-time rendering improves the user's experience drastically.
This is where asynchronous plug-ins come into play. They are almost identical to the plug-ins you have seen so far. The only difference is that they implement the IStylusAsyncPlugin interface rather than the IStylusSyncPlugin interface. Both interfaces are identical and only differ by name. All implementations you have created thus far work in identical fashion as asynchronous plug-ins if you simply search and replace the interface name.
The only other change is that the plug-in needs to be added to the AsyncPluginCollection collection, rather than the SyncPluginCollection you added it to before.
Give it a try and run the same example in asynchronous mode again. You will notice that it is functionally identical, but you will probably have a smoother experience, as all the processing power is available for Ink rendering, and the process of storing Ink is handled asynchronously when the system isn't busy. Note that this does not mean that the asynchronous plug-in will run minutes later. The delay will be minimal. But it means that the main processing thread is not blocked by non time-critical code.
In the Next Version of the SDK
In the current generation of Tablet PC operating systems, the Real Time Stylus API is strictly a managed (.NET) API. Unmanaged (native) code developers can only use the Real Time Stylus API through interop, which defeats the purpose of a high performance API and thus negates many of the advantages provided by the Real Time Stylus. In the next version of the SDK (currently available as the September 2005 Community Technology Preview for MSDN subscribers) as well as in Windows Vista, the Real Time Stylus API will also be available to COM-based environments (such as native C++) without the need to interop.
Other Real Time Stylus Settings
The Real Time Stylus API provides many features that go beyond the scope of this article. For instance, you can programmatically influence what information is provided as the package data. This sort of information is set on the RealTimeStylus object, rather than on individual plug-ins. When those settings are changed, all plug-ins receive the same modified data. The reason is that data packages are generated only once and then passed to all plug-ins. Creating individual data packages for each plug-in would not result in the performance you expect from the Real Time Stylus API.
The Real Time Stylus API is a great addition to the Tablet PC SDK. It provides more raw power, control, and flexibility than any of the default Ink collection controls and classes for a relatively small increase in complexity. This doesn't mean that the Real Time Stylus API should be used as a replacement for other options in all scenarios, but whenever you desire this degree of freedom, the RTS API is a great way to go!