The Rise of the Software Architect in an SOA World

Recently there's been a lot of talk about software architecture, particularly with the emergence of Service-Oriented Architecture (SOA). In this article, I will be describing the expanding role of the software architect. I'll use a building metaphor to describe SOA, since an architect in the real world is responsible for designing the blueprints atop which buildings are made.

This article will clarify why the architect is an essential player in SOA and why people who call themselves software architects will increasingly be in demand as enterprises increasingly adopt SOA. I will indicate the role of the architect indirectly by talking about how SOAs are designed, built, and how they succeed or fail. Throughout this process, the architect acts as a guide, planner, and overseer of the "big picture" of SOA.

SOA has caused consternation, confusion, and some excitement among Enterprise Software practitioners. Some of the confusion is caused by the diversity of implementation technologies, and even by definitions of what an SOA is.

For the purpose of this article, we will use the Wikipedia definition, which describes SOA as:

"...A software architectural concept that defines the use of services to support the requirements of software users. In an SOA environment, nodes on a network make resources available to other participants in the network as independent services that the participants access in a standardized way. Most definitions of SOA identify the use of Web services (i.e. using SOAP or REST) in its implementation."

This definition is very minimalist and does not include many of the typical best practices associated with SOA—but for our purposes, a minimal definition allows us to highlight the role of the architect in choosing the best path through the morass of standards, methodologies, and technologies available. Although Web services technologies are not required for the use of SOA, this article will use Web services examples in order to advance our discussion of SOA.

SOA is typically lauded for lowering the cost of software through reuse and for enabling a more agile process-focused way to address changing business requirements. It is also recommended for helping align the interests of information technology (IT) and business users. Achieving these goals will require governance, an intentional methodology, and selection of essential standards and technologies.

The three primary goals of SOA described above are reuse, agility, and business/IT alignment. Other goals for SOA have been described, but we will leave it at these three—it's the architect's job to set additional goals, if any. Let's see how the architect is instrumental in achieving all of these goals.

Reuse
In order to achieve reuse, the first essential step is to ensure the existence of standards. The second step is to enforce the standards. The enforcement of standards is one facet of what many refer to as SOA Governance. Standards are the foundation of reuse, as they enable services that have no prior knowledge of one another to consume one another. Obviously having standards is of no value if service implementers ignore them, so processes need to be put in place to help. One essential area to standardize is the service registry repository function. The registry component of the registry repository contains a catalog of services, which enables service discovery (and therefore reuse). The repository component of the registry repository contains documentation of policies and other information to support SOA Governance applications.

Agility
Agility is achieved in SOA through the ability to encode IT processes at a higher level of abstraction than the raw services. This means orchestration or workflow services or declarative configurable metadata systems such as policies and contracts. These elements collectively are referred to as "composite applications." This can be laid out in three layers, as in the three C's of SOA programming as described by this article—configuration, composition, and customization This area is one that is very sensitive to requirements and standards—many people have criticized such standards as BPEL as being insufficient for achieving business agile process definition, and there is a lack of standards around SOA contracts and declarative programming as well.

Aligning Business and IT
A key way to align business and IT is to have a methodology that connects business to architecture, such as the Intentional SOA methodology. This establishes the upfront goals of the SOA and strives to realize these goals. Without a prior agreement about the goals of the project, enterprise scale SOA will be both difficult to fund and difficult to realize. The architect serves as "the missing link" between business and IT. By maintaining the "big picture," an architect can help ensure that these disparate organizations can learn to work together. Another element of methodology that may help is a notation system. A notation system that can be understood by both business and IT users can help to ensure alignment of objectives. One such notation is the OASIS SOA Adoption Blueprints notation, recently donated by Cap Gemini.

As is becoming clear, all three of these larger SOA goals are dependent on architects. Although the end-to-end SOA lifecycle includes many constituencies such as developers, system administrators, IT Operations people, customers, business users, security experts, and others—the architect is the only role explicitly dedicated to the "big picture". Because of this, the architect is becoming increasingly indispensable to enterprise software success.

It seems that there are two fairly distinct types of approaches to SOA. I call them "S" people and "A" people. "S" people tend to focus on services. "S" people are very important, and SOA is built entirely of services—so it's key to have "S" people on the job.

That said, not all services are created equal. Some services such as security, identity, logging, notification, registry, and others are "metaservices," which are designed to be reused across all other services. Architects are responsible for establishing a strong framework (I kind of hate that word, but there you go) of metaservices for the "S" people. In addition to establishing the metaservices framework (also called "Service Infrastructure"), architects hold the responsibility for the standards and overall SOA "game plan."

A solid SOA blueprint will be an important product of an architect's work. An SOA blueprint begins with business use cases, much like the blueprint for a building is dependent on the uses for the building. Will the building be a home? An office building? A skyscraper? The use cases and requirements are the starting point for developing your blueprint.

OASIS has a committee focused on this problem called the SOA Adoption Blueprints Technical Committee (I am chair of this committee). You can download sample blueprints, notation, and methodology from this committee and if you're an enterprise architect, it may be valuable for you to consider joining this, or some other, community of best practice. By working together with other architects, you can ensure that your blueprint embodies the best practices and patterns available to the industry. You will also want to make sure that your blueprint avoids any SOA antipatterns which are well known among SOA architects.

The "ground floor" of your SOA depends on the style of programming you intend to deploy. These pilots include ways that are slower like WS-Heavy, and other styles, which enable programmers to get off the ground faster.

WS-Heavy: WS-I Basic Profile
A number of standards are needed to achieve a proper SOA. For a Web service-based SOA, there's no set of standards more fundamental than the WS-I Basic Profile, which includes XML, SOAP, WSDL, and UDDI. This has been referred to as the WS-Heavy approach, which tends to extend beyond the Basic Profile to include standards such as WSRP, ebXML, WS-Security, WS-Policy, and many others.

REST (Representational State Transfer)
REST is not a standard, but a way to build Web services applications. Critics of SOAP suggest that SOAP is unusable without significant tool support. The REST concept was invented by Roy Fielding in his doctoral dissertation. While not a Web services standard, developers are interested in REST. Like SOAP, REST provides a mechanism for communicating between service providers and consumers. All REST developers need is a browser and a Web programming environment—in some cases a scripting language such as Perl, PHP, or Python. REST is a lightweight way of providing and consuming Web services. The requester in the REST paradigm sends a URL in which all of the requesting parameters are encoded. The response is an XML document that corresponds to this request. This asymmetry initially appears to be client-server and browser oriented, but there is nothing preventing a server from using URL encoding to request a message from another server.

The REST paradigm is a robust architectural philosophy rather than a specific standard and can be applied partially or completely. For example, traditional REST suggests the use of HTTP GET as transport, when in fact REST paradigm URLs and URIs can be responded to over other transport forms if needed (for example, over guaranteed delivery middleware). REST is much more loosely defined and opportunistic than a SOAP-based approach. It does not rely on heavy tooling, because the calling format is the URL, the debugging tool is the browser, and the parsers are often scripting languages such as Perl- or PHP-oriented.

AJAX
Ajax is an acronym for Asynchronous JavaScript and XML. Ajax is something of a programming style made of constituent parts. Ajax incorporates:

• Presentation using XHTML and CSS
• Interaction using DOM
• Data interchange and manipulation using XML and XSLT
• Asynchronous data retrieval using XMLHttpRequest
• With JavaScript binding everything together

AJAX is described well in this article by Adaptive Path.

Web 2.0
A combination of "Social Software", Blogs, RSS, Ajax, and other technologies have been referred to in aggregate as "Web 2.0."

However, when I see a full release number, I expect a product to be rebuilt or rearchitected in a way that enables it to go to a higher level. The package of goodies assembled in Web 2.0 seems more like Web 1.3—a pretty good "dot rev." In fact, 1.3 is a point where the product is pretty stable and there's a pretty decent ecosystem of builders on top. In fact, Microsoft's own Software as a Service plans seem to have a pretty robust implication in terms of platform.

I think if you want to talk serious platform for V2, the Web platform has forked. There's the semantic Web, which seems to have slipped in its shipping schedule, which could be Web 2.0. Then there's this "SOA" thing, which has a lot of industrial-strength plumbing in it. That's a new platform and probably deserves the title of Web 2.0, not a bunch of clever mash-ups, JavaScript hacks, and REST apps.

The "second story" of the SOA "building" involves the stage where you have already built the "ground floor" and want to do more. This includes some basic services and some basic architecture. This means that you will have basic services that consume architectural metaservices such as security, identity, and the rest. These basic services will be sitting in a registry repository of some kind.

At this point, you will have services consuming others. But what about building the services that consume these services?

This can pose an interesting challenge—since this is potentially a way of creating a spaghetti factory of services consuming each other without any semblance of order. Some of the problems which can come with this include performance issues, maintainability issues, complex cascading outages, and other serious concerns.

In order to reach this "height," one important concept would be a way to establish versions of services. A dependency mapping system enables a clear understanding of which services are built on which others, and can be useful. A robust versioning system allows for services to be upgraded and changed without ruining services that rely on them. Service level agreements (SLAs) allow performance to be assured. SLAs are important infrastructure because as services depend on other services, performance can degrade to unacceptable levels. By maintaining enforceable SLAs an SOA can ensure that the performance of the system meets the requirements of the application.

One way to manage versions and service levels is by adding an intermediary such as a Web services management system. Intermediaries can route Web services traffic to load balance and assure performance and can direct consuming services to appropriate versions of existing services.

As SOA moves to enterprise scale, many of the "quick-fix" solutions for starting pilot projects are as bad for your enterprise SOA as having no pilot in the first place. Examples include trying to build a multi-layered SOA without regard for service management and appropriate governance capabilities. This would be like treating a quickly assembled shanty town of makeshift cardboard and tin lean-to structures as the foundation for your skyscraper. Examples of these weak foundations would include: SOA programming model without governance, UDDI registry without repository, SOAP messages without secure and reliable message infrastructure, WSDL descriptions of services without SOA contracts and SLAs, and service networks without service management.

One of the principles covered in Geoffrey Moore's classic text "Crossing the Chasm" is the idea that as technologies are adopted by the early majority and enter the mainstream—the very skills and methods that won over the early adopters are a hindrance to moving to the next level. This is a very important principle, as the earliest stages of SOA adoption tend to be overpopulated by "S" thinkers and developer focus rather than business focused or "A" thinkers.

Who's responsible for ensuring that an enterprise SOA can "Cross the Chasm"? You guessed it—the architect.

The architect plays a key role in enabling the SOA "building" to achieve its proper height. By creating appropriate blueprints, aligning key players on the business and IT sides, and selecting the right standards, programming models, and technologies, an architect can ensure that the long range goals of the organization can be met by SOA.

People who describe themselves as "software architects" will play the central role in the evolution of SOA. This will help ensure long-term job satisfaction and job security, as a good architect can make the difference between success and failure of these ambitious enterprise scale projects.

Email Article Print Article Download PDF Digg del.icio.us Newvine furl StumbleUpon BlinkList Newsvine Magnolia Facebook Tailrank Slashdot Technorati Google Bookmarks Yahoo Favorites Windows Live Ask