evelopers usually agree that it does not make sense to write all code character-by-character. After coding some features, they usually start to find similarities in the code, patterns that seem to be repeatable. Because nobody wants to write routine code, the idea of code automation?generating the repeatable portions automatically?arises both naturally and repeatedly. The obvious benefits of generators, such as productivity, quality, complexity hiding, and having more time to solve real domain problems, further boost the desire to seek automation.

Varied opinions exist concerning what kinds of code one can generate, and with what level of quality. For example, automation to produce static declarative definitions from common designs such as interfaces or database schemas has been a reality for ages, so multiple off-the-shelf generators are available. However, the situation is different when it comes to generating behavioral, functional code. I would like to emphasize that by generation I mean automatically producing code that is complete from the modelers’ perspective. This means that the code is complete, executable and of production quality from the modeler’s perspective; in other words, after generation, the code needs no manual rewriting or additions. Although it is possible to edit the generated code, you should avoid doing so. Editing generated code is (or should be) analogous to manually editing machine code after C compilation?typically unnecessary. I’m a strong believer that you should treat generated code simply as an intermediate by-product. That has been the recipe of success for compilers, and code generators can achieve the same objective.

Many developers have bad experiences with third party generators, because the generator vendor has fixed the method of code production. Despite the existence of multiple ways to write code for a certain behavior, the vendor has chosen just one of them. The vendor’s chosen way is not always likely to be ideal for your specific contingency, taking into account the target language generated, the programming model used, memory use, etc. Third-party generators often don’t know enough about an organization’s specific requirements to generate ideal code, so it is not surprising that many have found generated code unsatisfactory. Because modifying the generated code is usually not a realistic option, organizations end up throwing away generated code. The value of the generated code is then limited to prototyping and requirements gathering.

Because of these disappointing experiences, developers show little confidence in generated code. This lack of confidence, however, changes radically when developers are asked if they trust generators they have made themselves. Not having to give up control of the code generation process, from design to output format, to a faceless tool vendor makes a big difference in the acceptance of generated code. This is exactly the promise of Domain-Specific Modeling (DSM). With DSM, an experienced developer can define the automation process and output for the rest of the developers in that team.

This automation works because the generator (and specification language) is made to satisfy the demands of a narrow application domain?usually inside one company. It should be clear that you can’t have a general purpose generator or specification language in model-based code generation?even if it would be nice to have a standard. Perhaps the most notable attempt is the Object Management Group’s (OMG’s) Model-Driven Architecture (MDA), which uses UML as the specification language. Unfortunately, 10 years of UML use have proven its code generation possibilities to be modest, and the role of UML models has stayed at the sketching level.

It’s useful to look at some real-life cases where models have gone beyond sketches to generate full code. After having shown that full code generation can work, I’ll address some guidelines for generator design that I have found useful.

While there’s insufficient room in this article to cover everything, the example cases should suffice to show that code generation isn’t just for one particular niche. The cases cover various generator approaches and various target languages, including XML, Python, C and Assembler. Each example produces fully working code and contains both the application logic and behavior, not just the easier-to-create static structures.

Example 1: Generating XML
Defining generators to produce XML is often quite a straightforward process, especially if the modeling language maps well onto the XML schema. This is in fact normally the case, since both are designed to be a good way of describing the same domain. Where XML schemas have had to sacrifice understandability to cope with the limitations of XML, the modeling language can do things in a more natural way. In this case, the generator will do a little extra legwork to produce the verbosity and duplication required in XML.

This concrete example uses the Call Processing Language (CPL). CPL describes and controls Internet telephony services, in particular, specifying how a system routs calls. The structure of the CPL language maps closely to its behavior, so any CPL service developer can easily understand and validate the service from these graphical models (see sample in Figure 1). Figure 1 also helps show the benefit of models; even a non-CPL developer can largely understand the model, whereas deciphering the same model in XML would be considerably harder.

	?
Figure 1. Redirecting Calls: A sample call redirecting service expressed in CPL.

The modeling language includes concepts such as proxy, location, and signaling actions, which are essential for specifying IP telephony servers. The generation process is very clear-cut, because the XML already defines the language concepts, and the property values of the modeling constructs are attributes of the XML elements. The task of the generator is to follow the nodes via their connections and create the corresponding CPL document structure in XML. The following code shows the generator output when producing XML from the specification illustrated in Figure 1.

   01     02     02     03     04     05       06         07       08     09     10       11      
   12           13             14               15               16               17             18           19      
   20         21            22         23       24     25     

The generator starts by going through all the subactions of the service specification. This example contains only one subaction, the voicemail box at the right bottom corner of the model, for which the generator produces lines 4?8.

   04     05       06         07       08  

This “voicemail” subaction defines a location element (line 5) as well as a redirect element (line 6), which activates the redirection automatically.

After producing subactions the generator starts to specify the main call processing actions. It goes through the service specification from a service start (the yellow “CPL” circle in Figure 1). The generator crawls the connections from the CPL circle through the “Incoming” relationship to the Address-switch object. It produces the properties of the Address-switch node as element attributes in the generated output (lines 10?11).

   10       11      

The generator continues to follow the main flow path arrow to the next object and produces the location definition (line 12).

   12        

The path continues and the proxy handling is generated on lines 13?17, first the timeout attribute (line 13) followed by three alternate connections from the proxy element.

   13             14               15               16               17          

Finally, the generator produces lines 20?22 for the cases where the call origin has an address other than example.com.

   20         21            22      

The generated code forms a complete service whose validity has already been checked at the design stage. Because the modeling language contains the rules of the domain, service creators can create only valid and well-formed design models. The modeling language can also help service creators with consistency checks and guidelines, for example by informing them about missing information (such as the lack of a call redirect specification). These rules are highly domain-specific and thus be handled only with a domain-specific language.

Example 2: Generating Assembler for 8-bit Microcontrollers
You can apply the same model-navigation approach in other cases, too. Here’s a slightly more demanding case?generating 8-bit code for an embedded device. This particular device has a voice-menu system that enables remote control of many home-automation features, such as turning a light on or off, setting the room temperature, air conditioning control, etc. The system is programmable via an 8-bit microcontroller using an assembler-like programming language. In this case, code generation must take into account issues such as code size and memory usage that were not relevant when producing the XML in the previous example.

Besides such basic operations as accessing memory addresses, calculation, comparison and jump, the developer must also handle a few operations specific to voice menus, such as reading menu items aloud. The domain-specific language includes these concepts directly in the modeling language. The design model in Figure 2 illustrates an example of such a language.

	?
Figure 2. Setting Parameters: The figure shows a design for setting one home automation parameter in a voice menu system.

Here’s a sample of the code generated from the model in Figure 2.

   01   Speak 0x01 (Pause for 0.5 sec)   02   Speak 0x02 (the...)   03   Speak 0x03 (...current...)   04   Speak 0x04 (...lifestyle...)   05   Speak 0x05 (...is...)   06   GetLifeStyle     07   Speaks Lifestyle     08   Speak 0x06 (Pause 0.3 sec)   09   Speak 0x07 (select...)   10   Speak 0x08 (...another...)   11   Speak 0x04 (...lifestyle...)   12   FillMemB 00   13 :3_844   14   Add to MemB 01   15   Speak 0x09 (for...)   16   Speaks Lifestyle     17   Speak 0x10 (...press...)   18   Speak number MemB     19   Is MemB >= 0F   20   IFNot   21   Goto 3_844

The fundamental code-generating solution relies on a flow-like execution of actions specified in the model. The metamodel for the language is relatively simple. Each elementary operation type (speaking, memory access, and comparison) has been implemented as an individual modeling concept, while the flow of execution and conditional jumps appear as relationships between these concepts. Each modeling concept also carries information about related design attributes such as the command tag, possible parameters, or conditions. As the assembly language here is domain-specific, with one assembly mnemonic reflecting one basic voice menu operation, the mapping from modeling concepts to mnemonics is solid and simple.

The generator follows the flow of operation via relationships, transforming the information contained in each design element into output code. For example, the generator uses the first bubble on the top left corner of diagram to generate the set of speech commands in lines 1?5..

Because the spoken audio samples consume a lot of memory, each word is stored only once and reused when constructing complete sentences. Thus, the spoken messages in menu elements consist of a sequence of individual words or short phrases, along with the memory addresses for the corresponding audio samples. When the code generator meets such a construct, it simply iterates over the collection and outputs the speak command for each word followed by the address of the sample. When the same audio appears multiple times the generator outputs the same memory address?for example, lines 4 and 11 use the same memory address.

Variations of the speak command can read out variable values (see lines 7 and 16) or numeric arguments (see line 18). In addition to speech commands, there are also operations for getting (line 6) and setting (line 12) values of variables.

You can see a slightly more complex part of the generation process in lines 12?21, which reads out the selection numbers for all predefined lifestyle settings.

   12   FillMemB 00   13 :3_844   14   Add to MemB 01   15   Speak 0x09 (for...)   16   Speaks Lifestyle     17   Speak 0x10 (...press...)   18   Speak number MemB     19   Is MemB >= 0F   20   IFNot   21   Goto 3_844

First, it initializes the selection variable memB (lines 12 and 14). Then it retrieves and speaks the information about a predefined lifestyle (lines 15?17, simplified here to save space) and the selection option (line 18). Lines 19?21 represent the conditional jump. If the selection variable memB is less than 15 (0F hex), the code jumps back to line 13 and continues with the next predefined lifestyle, repeating the loop until memB equals 15.

In this case the key to the successful code generation is the modeling language. While there are a few domain-specific commands included in the target language itself, there is no real framework on the platform side to make the generation easier?and as you can see, the generation process itself does not include any “magic.” The model level covers the structural and behavioral essence of the system, leaving no need for the generator to tackle complex variation or implementation issues related to the low-level language used.

Example 3: Generating Python for Mobile Phones
Navigating models based on connections is just one possible way of translating models to code. To illustrate other ways generators can work here’s a case that generates SmartPhone applications by producing function definitions for individual model elements. Here the underlying phone platform, Symbian/S60 and its Python framework, provides a set of APIs and expects a specific programming model for the user interface. To enable model-based generation, a specification language and generator follows the programming model and APIs. Figure 3 shows a sample design.

The modeling concepts in Figure 3 are based directly on the services and widgets that S60 phones offer for application development. Modelers describe the behavioral logic of the application mostly based on the widgets’ behavior and on the actions provided by the actual product. If you are familiar with some phone applications, like phone book or calendar, you can probably understand what the application does by studying the model.

	?
Figure 3. Conference Application: The figure shows the design for a conference application intended to run on a Symbian/S60 phone.

From the design in Figure 3, the generator produces function-based code that the target device or an emulator can compile and execute. The generator itself is structured into modules, one generator module for each modeling concept. For example, one generator module takes care of lists, another handles confirmation dialogs etc. Because several concepts require generation of similar code, such as flow of control to the next concept, parts of the generator definitions are made into subroutines used in several places. The generator includes some framework code for dispatching and for multi-view management (different tabs in the pane). Here’s some sample Python code generated from the design in Figure 3.

   01  import appuifw   02  import messaging   03    04  # This app provides conference registration by SMS.   ...     33  def List3_5396():   34  # List Check Credit card Invoice    35     global Payment   36     choices3_5396 = [u"Check", u"Credit card", u"Invoice"]   37     Payment = appuifw.selection_list(choices3_5396)   38     if Payment == None:   39        return Query3_1481   40     else:   41        return SendSMS3_677   ..   85  def SendSMS3_677():   86  # Sending SMS Conference_registration   87  # Use of global variables   88     global PersonName   89     global Payment   90     string = u"Conference_registration "   91           +unicode(str(PersonName))+",  "   92           +unicode(str(Payment))   93     messaging.sms_send("4912345678", string)   94     return Note3_2227   ...   101  def Stop3_983():   102  # This applications stops here   103     return appuifw.app.set_exit   ...   107  f = Note3_2543   108  while True:   109     f = f()

The generator starts by outputting module import statements (lines 1?2) based on the services used. First, the overall application UI framework, and then?because the model contains SMS actions (yellow envelope symbols)?the messaging module. The comment following the import statements is simply taken from the documentation entry specified in the design. Next, the code defines each service and widget as its own function. Rather than producing functions in an arbitrary order, the generator produces functions arranged by type, for example, all list functions followed by SMS/text message functions etc.

Lines 33?41 describe the code for the payment method selection that uses a list widget. After defining the function name and comment, the code declares a global Payment variable. Line 36 shows the list values as Unicode in a local variable, and line 37 calls the List widget provided by the framework. The code handles sending SMS messages (lines 85?94) in a similar way to the List widget. Line 93 calls the imported SMS module’s sms_send function. The generator takes the parameters to the function (recipient number, message keyword and content) from the model, and handles forming the right message syntax. These messages are clearly defined, always using the same pattern.

The end of each function includes code to call the next function based on user input. For sending an SMS, the generator simply follows the application flow (line 94), but for list selections, the situation is little more complex. Depending on the user’s selection from the list, different alternatives can exist. While in this case the code always takes a single path forward regardless of the value chosen from the list, there is the implicit possibility of a cancel operation (pressing the Cancel or Back button). The generator automatically creates operation-cancelling code to return execution to the previous widget (lines 38?39), or if a choice was made, moves on to send the SMS (lines 40?41). Having the DSM implicitly handle Cancel or exception actions is just one extra way to make describing applications simpler. In other words, the modeler doesn’t need to do anything in most cases, only specifying behavior that differs from the default (such as the diagonal Cancel relationship to the end state from the first menu).

In the final function, the generator creates application exit code based on the application’s end state (see lines 101?103). Finally, a dispatcher starts the application by calling the first function (line 107). This function, like all the others, returns the next function to call, and lines 108?109 handle calling the next function after each function has finished. Handling the calls this way, rather than having each function itself call the next function, provides a kind of tail recursion to reduce stack depth when moving from one function to the next.

This case highlights an interesting benefit of DSM; the modeling language is totally independent of the generated programming language. As Symbian phones also support C++ and Java, you can change the generator without changing the designs to generate an identically behaving application in a different target language.

Example 4: Generating C for Digital Wristwatch Applications
The last example illustrates C code generation from state machines. There are several ways to implement a state machine code. For a more detailed description of possible solutions see Miro Samek’s?book Practical Statecharts in C/C++. The best solution for a given situation depends of course on such things as the target language and platform, requirements for the size and efficiency of the code, etc.

This digital wristwatch example’s architecture consists of a set of applications for displaying the current time, stopwatch, alarm etc. The modeling language focuses on capturing the static elements of the watch application, such as buttons, time units, icons, etc., and on capturing the functionality using a kind of state machine. It extends the traditional state transition diagram semantics with domain-specific concepts and constraints. For example, each state has a link to a display function that presents the time, state transitions are triggered only by pressing one button once, arithmetic operations are limited to those relevant for operating on time units, etc. Figure 4 is an example application model for showing and editing the current time, and related operations. Listing 1 contains the corresponding generated C code, slightly abridged to save space.

	?
Figure 4. Watch Application Design: The figure shows the design for a watch application that can display and edit the current time.

Defining the mapping process from model to code is reasonably painless. The generator goes through the design and creates enumerations (enums) that will act as unique labels for states and buttons (lines 1?2). These are then initialized in lines 4?5. Next the generator outputs a boilerplate runWatch() function, common to all applications. For each input event, runWatch() invokes the main behavioral part of the application, handleEvent() (lines 16?66). In handleEvent(), what to do and where to go to next depends on the preceding state, and what the input event was. The generator thus reads the state transition diagram and implements it using a simple nested switch statement. The generator produces this by iterating over the states in the model; for each state it iterates over the transitions leaving that state.

A transition can also trigger actions. The example model needs to support only basic time units for arithmetic operations. Setting time variables is a simple assignment; but rolling a digit pair up or down is actually somewhat complicated?different time units roll around to zero at different numbers. Because many applications need such a function, you’d probably move that code out of the per-application code into a framework component if you were manually coding such applications. While you could generate it inline each time it is needed, it seems better to keep the overall code small and conceptually neater by making it into a function; then, when it needs such a function, the code generator needs only to produce a call for it (line 35).

Though being more complex than the voice menu example, this case still does not require much work on the framework side. Some framework code or components are needed to provide some low-level services for behavioral part of the implementation (e.g. the implementation of the roll function) but for the structural part, we rely entirely on the C language (e.g. using a plain switch-case to implement the state machine). As in the previous example, the model captures the essence of the system, which reduces the complexity of the code generator.

Defining a Generator
Building a generator is about defining how to map model concepts to code or other output. In the simplest case, each modeling symbol produces certain fixed code that includes the values entered into the symbol as arguments by the modeler. To go a bit further a generator can also take into account relationships with other model elements or other model information, such as sub-models, models made with other languages, or pre-existing library code. The way the generator takes information from models and translates that into code depends on what the generated output should look like. The examples of model serialization you’ve seen here that navigate model connections, generate function calls from flow models and generate state machines using switch cases are just a few examples of what you can do.

The generator definition depends on the availability of a good reference implementation; in other words, you need to know what you are developing before you can automate it! The reference implementation for DSM should be available as a pair: the design data (a model), and produced output (code to be generated). Based on my experience, the generator definition process is usually test-case driven; you work backwards, starting with a reference implementation of the code to be generated. A good model has multiple alternative implementations; therefore, only locking down the details of the output allows you to know what exactly you want out of the generator. For example, in the XML generation example, the schema pretty much defines the whole scope for the generator output. In other situations, you might take the test case from an existing application or feature, or write it from scratch just for the needs of generation definition.

The best practice is to ask the most experienced programmers to come up with the reference code. It’s important to ask them to write in the same style as they would like to teach other developers; otherwise the code may include too many special tricks for particular cases, rather than good standard code that can be generalized to all applications in that domain. Generating that code makes a good impression. Even if you later abandon the generator for some reason, you still end up with standardizable and generalizable expert code. Using code from experienced developers also speeds up the generator creation simply it requires less discussion about different coding practices and standards. The experts should rule here.

Having reference code proves that generated code can look familiar, follow the required programming mode, include appropriate comments, and follow the local standards for code style. I even know of cases of generating legislation and compliance information into the code to prove it satisfies customer requirements. Although later on, modelers generally don’t need to look at the generated code, good-looking code creates confidence in the generative approach. Just like when buying a car?you want to see that there is a motor, but later on you don’t want to bother to look under the hood. The structure of the generated code actually has a bigger impact on the person who takes care of the generator afterwards; generated code is easier to read and test cases made earlier can be applied when the generator is modified.

Having a reference implementation as a basis for the generator, or for just part of it, lets you ensure that the generator produces the expected result; otherwise, you change the generator. You may also find that it is useful to alter the modeling language or create some framework code to keep the code generator simple and enable better code generation. For instance, you should try to avoid cases where the code generator needs to check that the input it gets is correct. The modeling language rules should normally cover this already. Generally speaking, it is best to get the modeling language right early, because later, when there are many existing models, there may be some restrictions on what kinds of modeling language changes you can make. However, even at that stage it is still possible to make changes to the generators. The simpler your code generators are to start with, the simpler it will be to make changes to them later. Keeping the generators simple means you have to do less work to update them in the early days, when the modeling language is still evolving.

You can apply reference implementations as test cases during all phases of generator definition. You usually start from few typical structures to be generated, and then extend the generator bit by bit. For example, in the XML case you can start by choosing just a portion of the schema, and then gradually extend the generator to handle the entire schema. Similarly, during maintenance and enhancement of the generator, getting a test case beforehand makes generator definition simpler.

Good Generator Properties

In my opinion, a good generator produces complete code. This has been the cornerstone for automation and raising abstraction in the past. We would not be happy if after writing C and compiling we needed to modify and rework the created assembly language or machine code. For the same reason I personally have difficulties in understanding how OMG’s MDA could work. The idea sounds attractive?making a model that gets transformed to another set of models, which are then modified and transformed to still more models, and ultimately, to code. However, this approach leads to the same results as wizards: lots of code (and models) that you didn’t write yourself, but that you are expected to maintain. Such wizards can sometimes be helpful, and they do offer increased productivity at the start, but over time creating mass of unfamiliar code that needs maintaining tarnishes the picture considerably. The MDA idea gets even worse when you consider round-tripping; would you like to update the manually made changes to the code and lower-level models back to all the higher-level models? Or, after you make a change to the top-level model, to successfully integrate your hand-made changes at the lower levels with new code caused by the top-level change? And if you wouldn’t, would you trust a tool to get it right? That was not the success pattern we saw when assemblers were replaced with compilers; nobody tries to maintain their C code in both C and assembler.

I would also advocate that model-based generators should target code directly instead of producing intermediate models that need to be extended during the development process. Naturally, you may need to generate some supporting intermediate models, but the process usually breaks if the generated models need to be changed. So, if you face the situation of model-to-model transformations, do the following: look to see what kind of information gets added by modelers to the lower-level generated models and then extend the higher-level language to capture it. As a side benefit, you usually find a way to record the information into significantly fewer data elements in the higher-level language. Finally, merge the transformations into one. If you can do it, you make life easier for everybody. It is easy for the modelers because there’s only one language, one model, and no round trip that might desynchronize the model and the result. It is also easier for the language developer because there’s only one language, and a single one-way non-updating generator. Experience shows that a single-model approach that targets code works?and scales in larger teams too.

Code generation is not the only place for automation. The power of models increases even further if you can also generate things such as configuration data, test cases, simulation material, documentation, automated build processes, and so forth from the same source. Having a single source and multiple targets can be very beneficial, because when making modifications, developers need to make a change in only one place, and the tool takes care of the rest.

Not all code can be generated from models. Some parts will still be hand-written, and moved to domain framework components. However, when looking at things from the designer’s perspective, models can be used to generate complete working code. The four cases presented here form concrete demonstrations of this from different application domains and from different generation requirements. The design rules of the examples depended on the application domain and the modeling languages were defined to give first-class support for specification work, making code generation, optimization, early error detection and correct reuse easier to achieve. On the generation side, the code produced is functional, readable, and efficient?ideally looking like code hand-written by the experienced developer who defined the generator.

If you look at the examples above, it would be hard to imagine how one single language or generator could have produced the correct code in each different case. General-purpose modeling languages such as UML are well suited for documentation, but not as well suited for generation. Code generation requires that details are correct too.

Recently, open and customizable technologies have emerged that allow developers to change both the design languages and/or code generators to meet different requirements of software development. Hence, experienced developers in a company can adapt the design languages and generators to a specific domain, and then model actual products using those domain-specific languages, which generate code directly from the models. Finally, it bears mentioning that for an expert, building model-based code generation is not only an interesting challenge?it is also a lot of fun!