odern computer hardware and powerful graphics cards, now found in even ‘budget’ systems, support a new level of rapid rendering of more high-level descriptions of objects and dynamic response to interaction with them. A variant of XML, XAML, has emerged as the Microsoft syntax for describing the form and visual state of controls and user interfaces, and the links to executable code that let users interact with them in an application.

The leaps-off-the-page benefit of vector-based controls is visually accurate scaling on a wide range of display devices. Graphic designers and developers may be able to interact in new ways that increase productivity when they can rapidly exchange interface design in the software development cycle.

At a hardware level, the Graphics Processing Units (GPU) found in today’s video cards will handle vector-graphics processing now handled by the CPU in a way similar to the way games now offload texture mapping and 3d rendering into DirectX enabled video hardware. Although XAML simplifies describing the appearance of vector-based controls, it will remain just as difficult to implement the code-behind, the algorithms, the complex interactivity, linkage to external data sources that modern control consumers expect. But, the new technology is a huge win for developers because it potentially eliminates handling tedious low-level UI drawing and painting tasks that constrain productivity.

Early applications and development tools for creating vector-based controls and user interfaces are in developers’ hands now: Microsoft’s Windows Presentation Foundation (WPF) and WinFX technologies are in Beta 2, and Adobe’s SVG initiative has been available (so far with modest adoption) for a while. Also, in my opinion, you can consider the widely adopted Flash technology to be a somewhat vector-based solution paired with a superb rendering engine.

As with all new technologies there are performance issues. In WPF/WinFX, the need to alternate rendering (between GDI+ in main memory on one hand, and DirectX in video-card GPU memory on the other) means application developers need to monitor how their complex controls or UI’s may perform, particularly in the run-up period to Vista. For more information, monitor the WPF MS forum, searching for keywords such as “performance” and “DirectX.” A developer friend of mine suggests that there are clipping problems in the first version of WinFX that can result in software rendering “slower than GDI+.” He also suggests that complex visual effects, like gradients, must be rendered into textures inside the GPU, and that the transfer of these textures may become a bottleneck.

He comments, “It is not a true immediate mode API. So you cannot really optimize memory usage or buffering. Nor can you change the rendering at a low level, nor can you use a context dependent rendering optimization, or any kind of incremental change optimization.”

Tim Cahill, on the team responsible for WPF performance issues, blogs regularly on these issues, and I strongly suggest you read the entry WPF Graphics Performance Q & A, in which he summarizes current performance issues and how they may affect different types of software applications, directly addressing concerns raised in the comments above.

Attention Graphically-impaired Software Developers!
Graphic designers today enjoy very highly-evolved vector drawing tools, like Illustrator, and also enjoy the integration of vectors into what were formerly raster-painting programs, like PhotoShop’s paths and masks. Until now, Microsoft’s developer tools for creating custom controls or user interfaces have required developers to deal explicitly with the low-level tasks of creating brushes, calculating clipping regions, and painting rectangles to get the really dazzling graphics effects that end-users of controls relish?even expect.

Very high-level graphics effects such as per-pixel alpha transparency are so difficult to achieve that it’s taken a generation of graphic hackers (morphed into graphic designers known loosely as ‘skinners’) working with technology such as StarDock’s WindowBlinds to show how it can be done. And (dare I say?) a generation of game-designers using DirectX to show how not to do it.

All that’s about to change. Attention graphically-impaired developers! You will now luxuriate in luscious gradients dancing to your commands on the humblest of tiny controls; you will have glass edges per-pixel anti-aliased to the alpha of your choice. So goes the latest song.

The ubiquitous COM-based controls (TreeView, ListView, etc.) have caused developers from VB through .NET 2.0 to struggle with the same gnarly low-level problems, such as “How do you suppress that horizontal scrollbar from hell in the TreeView control?” Customizing and mastering these controls required developers to delve deep into Windows internals, learning about API calls, windows hooks, and massaging parameter types. In my humble opinion, the best WinHook will be the last of its kind, mounted in the Museum of Unnatural Programming to be gawked at by visiting schoolchildren.

Will advances in hardware capabilities, new vector-based tools for making controls and UIs, new higher-level ways of describing them, and new software IDEs for manipulating, designing, coding, and debugging them change the game for designers as well as programmers? Take your designer friend for a latte, and stay tuned.

Getting Our Share of the MindShare
The inevitable merger of what until now have been parallel but interacting universes of desktop applications and Web pages is part of the “great game” for developer and designer mindshare that’s driving the developer tools’ capabilities of Microsoft, Adobe, and a host of other companies to new technical heights, and their public relations campaigns to new depths. I see the current “state of the tools” as a kind of summer vacation on the Island of Dr. Moreau: companies enthusiastically launch mutant mixtures of HTML, CSS, JavaScript, SQL, ASP, Flash, etc. along with powerful back-end tools that require mastering their own specific complex APIs?all of which may have short and non-reproductive half-lives that leave programmers with valuable insight into the underlying DNA of-it-all, but that will most probably fail to reach a maturity level where “mere mortals” can use them as effective broad-spectrum tools.

I’d suggest the current state-of-the-art vector drawing tools embody an “international resource,” a repertoire of ideas, icons, and functionality, to which millions of people are socialized. Most of those millions may be designers, graphic artists, draftspersons, decorators, and business users rather than programmers?but if more of their talents for graphic design can be brought into the software development cycle, the companies who bring them in are going to be the companies that win.

Expression is Microsoft’s major development IDE for designing vector-based controls. Cider is Microsoft’s toolset for vector-development integrated into Visual Studio 2005. These are first-generation tools in beta, but show great promise for development. And do check out the links to bigger and smaller companies innovating in this arena such as MobiForm, Xamlon, and Marc Clifton’s myXAML.

But there’s no reason to limit such capabilities to Microsoft. Why shouldn’t you be able to translate a user interface design done in Illustrator into XAML, or cut and paste a Photoshop gradient into Expression? For Illustrator, Michael Swanson is already working on it. And Grant Hinkson of Infragistics has an exporter from Adobe FireWorks to XAML.

The key issue is for designers and developers to be able to see results quickly and tweak visual elements interactively without complex re-compiles. A friend within Microsoft believes Expression will improve their potential collaboration:

“The ability for designers to hack on working examples of controls is a godsend. Their core competency may not be coding, but anyone who’s dabbled in Flash programming will be comfortable trying out ways to tweak control appearance and interactivity in the design environments. The end goal of course is for designers to deliver fully working, finished UI elements to dev to plug in or just code review. A dream perhaps, but many designers have the skills to pull it off with the right environment. Working out of the same environment also goes directly to overall project productivity, and introduces designers into the world of source control, build breaks, and dev workflow. It gets designers “in the game,” not on the sidelines looking in.”

It’s a great dream to have a rapid-turnaround production/prototyping environment where good designers work effectively with programmers to make great controls, but I remind myself of the cautionary maxims of my friend Frank Hileman, who runs VG.net, a company developing vector graphic software for .NET developers:

“Developers generally don’t know how to build beautiful user interfaces. Graphical designers generally don’t know how to build useful user interfaces. True vector controls are much harder than most people think, requiring the skills of both groups.”

New Lamps with Old Oil, or Old Lamps with New Oil?
Back to the glorious Frontiers of Redmond where Young Freedom Fighters are at the barricades designing Expression: Let’s “zoom in” for a close look at the initial visual design tools offered in Expression…of course we zoom with tender merciful heart because we know we are in Beta Hospital, and the baby is still breathing with a respirator.

Take a look at the tool depicted in Figure 1, and then compare it with similar tools for gradients in programs such as Illustrator, Photoshop, Canvas, FreeHand, or your favorite, more obscure drawing and painting program.

Figure 1. Expression’s Appearance Palette: Expression introduces another new interface for setting colors and transparency.

Check out the other design tools in Expression, and then evaluate whether you could persuade your designer friends to drop their familiar tools and start using these to prototype user interfaces or controls while you went to work programming interactivity, tweaking the algorithms and data structures, or connecting to a Web service.

I’m optimistic about the next generation of programmers and designers being less constrained by traditional roles, more ready to co-operate around new tools, and I thoroughly disagree with Alan Cooper’s ideas about the “segregation” of programmers from user-interface designers while definitely agreeing with his general critique of the poor quality of design of many major computer applications used every day by millions of people.

But it’s not going to be an easy transition?as current PR efforts would have you believe. It’s going to take very hard work, and, in my opinion, will require the same factors that contributed to the success of Flash?’lifestyle’ marketing, evangelism, enabling a rich third-party scene of code-sharing and mutual education.

The underlying technologies are still in flux. Beyond beta problems, the very diversity of forms for common tasks, such as picking a color, across many OS’s and across applications may make it harder to rationalize the cost of that diversity into a new software development paradigm.

I believe the first generation of these tools will be complex, buggy, spawn a new set of talking-heads and great-explainers, muddy the waters, and lead to some very cool and valuable innovations. And the succeeding generations of the tools, if designers and developers can work together, will be superb.

I imagine descriptions of user-interfaces in XAML flowing around the Web or networks as transparently as e-mail?and I can imagine that because I remember back to 1985, when Sun and Adobe almost released a radical technology called “NeWS,” where multiple streams of PostScript describing vectors, fonts, and extended to support hit-detection and event propagation would sail around networks, letting your computer display several windows being fed from different sources!

Yes, that was in the Ancient Days (before the coming of the IPOD). Now excuse me while I go and rock a while on the porch of the “Old Programmers’ Home” and listen to my MP3’s.