Browse DevX
Sign up for e-mail newsletters from DevX


Effective Build Management: Don't Build a House of Cards : Page 2

By exploring the latest techniques for automating build management you'll get a better understanding of the fundamentals and benefits of effective build processes as well as how they fit into an overall configuration management strategy.




Building the Right Environment to Support AI, Machine Learning and Deep Learning

Simplifying the Process
In-house build systems are usually based on a standard when they are originally designed, but as time passes and technology and requirements change, these systems grow to include multiple scripts and makefiles, file transfer mechanisms, scheduling mechanisms, etc. These home-grown systems are generally poorly documented and frequently hinge on a single developer—the resident build "wizard"—to make sense of them. When modules of code are added or reorganized or technology changes, this individual is the only one with the knowledge of the build system and the necessary makefile/scripting language skill set to manually list dependencies, edit rules and configuration files, and so forth. Of course, when a build has grown to include thousands of lines of script across many individual script files, this can turn into a severe bottleneck in the development process.

For the benefit of those who aren't build wizards, I'll quickly run through the anatomy of a build script. Build scripts contain all knowledge about involved targets and dependencies, as well as rules to build and link the code (with various flags and settings and likely conditional content for each platform). Therefore, a typical makefile (even for a simple program) might look like Listing 1.

As you can imagine, hand-editing these scripts can be time-consuming and error-prone. Because enterprise-class build systems typically feature graphical front ends for build management, they provide a much simpler way of identifying dependencies, choosing rules and options, scheduling builds, and viewing progressoften without any scripts. This eliminates the need for "scripting and makefile hacking," which not only improves the entire process but frees the "wizard" for the more important work of developing code.

Populating Build Areas
Many home-grown build systems rely on manual population of the filesystem with the assets needed for the build. At best, these systems attempt automation using ftp or a similar script-driven mechanism. Populating build areas in this way is very error prone and is hard to maintain as the system grows.

Exacerbating this problem is the dreaded "last minute fix." It is very tempting for developers to try to include their latest and greatest code in a build, and product managers are notorious for slipping in that last, absolutely-must-have feature or fix. This is fine—if it is done in a controlled fashion. But often last minute features are added after the code is retrieved and the build is started. You can add the code and restart the build but how can you be 100 percent sure that the new code was built? As with any manual step, this process is very prone to user error.

You may be thinking that a potential work-around would be to ensure stability by performing a clean build, but this can impact performance. Alternately, the system could be designed to rely on timestamps, but these can be very unreliable due to time zone differences, etc.

It's also important to keep in mind that most projects are not just built once. At various stages in a project's lifecycle there are a number of different types of builds that may take place.

  • Individual developers might build their latest changes in order to catch compile errors and debug problems.
  • Unit test builds verify that a single unit or component of the project functions correctly.
  • Integration/system test builds ensure that all components hang together and the entire system functions as expected.
  • Production builds deploy the program to a production environment.

These stages are by no means fixed. They may differ between projects and organizations, but it's usually some variation on the theme of develop, test, and release. There is clearly a logical progression of code and deliverables between build stages, and the quality of the code matures as it moves up the various stages.

These build stages typically use different flags or options, and build steps and rules will change (for instance, perhaps there is no recompilation between system test and production). In many instances, each build is done on separate physical machines. Traditionally, this process is managed with separate makefiles/scripts for each stage, but the source can be shared among the stages to consolidate activities.

An enterprise-class solution, however, would be able to maintain these areas and (preferably) use search paths so that only the code needed for the current stage exists in the build area. It does this by looking for the code first in the current area and then working its way up the stages. For example, assume you have the build stages and versions of files shown in Table 1.

Build Stage

Files and version numbers













Table 1. Build Stages

With a well-managed enterprise system, a build executed at the RELEASE stage would compile Main.cs-1, Tree.cs-1 and Banner.cs-1. If executed at the SYSTEM TEST stage it would compile Main.cs-2, Tree.cs-2 and Banner.cs-1. And finally at the UNIT TEST stage it would compile Main.cs-3, Tree.cs-2, and Banner.cs-1.

This also nicely demonstrates the importance of selecting a build system that integrates with your configuration management (CM) tool. This way, the CM tool will automatically populate the correct build areas as the code progresses through its lifecycle, which dramatically reduces the risk of incorrect versions of source code appearing in a build.

Comment and Contribute






(Maximum characters: 1200). You have 1200 characters left.



Thanks for your registration, follow us on our social networks to keep up-to-date