2EE developers working in a standard lifecycle model often face challenges in managing deployment descriptors for the different lifecycle stages. In larger J2EE development environments where IT governance is an issue, developers follow a strict process of promoting their J2EE applications through a standard lifecycle model from development to testing and finally to production. The challenge is deployment descriptors are unique to the web server to which they are deployed. If the development, test and production web servers are configured differently, then developers must create unique .wars and .ears for each web server used in the lifecycle.
There’s a sensible reason why Web servers are often configured differently: unique database names for each stage of the lifecycle. Something as simple as a unique database name for each environment requires the J2EE developer to rebuild his J2EE components using a different deployment descriptor for each web server in the lifecycle. This means that instead of building a single .war or .ear that is moved across the lifecycle, the developers must build three versions: one for development, one to be executed in testing, and one for production release. And we all know that rebuilding an object after it has been tested is risky; how do you guarantee that only the deployment descriptors were changed between the testing and production versions of the .wars an .ears?
I have learned through experience the importance of carefully managing these files from the development to production lifecycle. If you release a “development” version of the deployment descriptor onto a production server, chances are that your evening will be consumed with the tedious task of sorting out why the application ran in development, but died in testing?or worse, died in production.
If you don’t quite understand why these files could cause such an immediate production headache, consider the following: A deployment descriptor can be understood by comparing an .ear file to a .zip file. Imagine that you have a .zip file containing 1,000 files. You want to copy that .zip file to five different locations, but first you must change some of the text in 10 of the zipped-up files, and the changes are different for each location where you need a copy. This task requires recreating the .zip file by uniquely configuring the 10 text files and re-zipping them five times. Then you copy each .zip file to the correct location according to the changes made. Basically, this is the exact process required for deploying a J2EE application to an enterprise environment where multiple server configurations are required.
|
This problem is not entirely the fault of the J2EE specifications, but largely the result of increased use of third-party modules and interconnectivity of enterprise applications and services. A single application running on a production server may require parameters that specify multiple database connections, log file locations, message queues, security information, CICS regions, and other server-specific information. Put that same application on a second production server for a different business unit and all of these values could be different. It is not unusual to see applications where as many as 100 values change. If you’re doing parallel development with multiple production, test, and/or development servers, the problem compounds rapidly. When this is the case, for obvious reasons, you want to know that the correct version of the .ear or .war is being deployed to the correct server.
Solve the Problem at the Build Level
It is important to keep in mind that the use of multiple deployment descriptors is not a deployment or coding problem. It is a “build” problem. I hear far too often companies looking for a “deployment” solution because they continually make the mistake of deploying incorrect .wars and .ears to web servers. Automating your deployment process will simply allow you to more quickly deploy the wrong .war or .ear. It is in the creation of the .wars and .ears during the “build” process that must be addressed.
When building .wars and .ears from within an IDE, the IDE allows you to configure a deployment descriptor for a single web server. Through the IDE you enter all of the unique web server configuration data and the IDE does the rest. This process works well for development, but is inefficient when attempting to support multiple .wars and .ears for test and production. This is because developers must stop working on the development level of code, and rebuild an older version of the .war or .ear, which can make it difficult to ensure that only the test or production level of code was used to create the corresponding version of the .war or .ear.
Because IDEs do not handle the creation of multiple deployment descriptors for multiple configurations, when it comes time to promote the application to the next stage of the lifecycle, developers must consider more efficient, less error-prone ways to get the job done.
There are basically two options. The easiest option is to consider the use of an enterprise build management tool that can handle the creation of multiple .wars and .ears. The market for enterprise build management tools is just beginning to emerge. Don’t rule out the use of a commercial build tool just because you believe you can write scripts to get the job done. Commercial build tools offer many benefits such as footprinting, impact analysis, remote build scheduling and monitoring. These features are not so easy to script. By using a commercial tool, you solve your J2EE build headaches and gain many new features that can assist you in your development and release process. If a commercial tool is not an option, then you will need to develop your own build process using scripts.
| Editor’s Note: The author is President and CTO of Catalyst Systems, a vendor of commercial build tools. We have selected this article for publication because we believe it to have objective technical merit. No specific editorial product endorsement is implied. |
Establishing an improved build process for your J2EE applications first requires a close look at exactly how your .ears and .wars are assembled. Planning is essential at this stage. If you choose to write a solution in-house, you should be prepared to perform Perl and Ant/XML scripting, and of course documentation and training for all developers who will be involved in the final assembly process. Also, you can expect to deal with the pains of trial and error before coming up with the correct solution that works for your organization.
Expert Preparation
Regardless of what road you take, following is a list of recommendations that you should consider:
Minimize Parameters Used
A common direction many J2EE developers take for addressing configuration issues is to use parameters. This may seem very practical at first, but as you move down this road, you will understand why it’s a flawed solution. When the developers are asked to take a working application and parameterize properties and values in files it means that if a developer has coded a particular property in a file for a log file location with a value of C:appsdevefoobarcomm.log, then the developer will create a parameterized version of the same file where the value has a parameter like @COMM_LOG_LOC@. The strategy then is to replace the parameter with values meaningful to the different target run-time environments?for example, C:appsqaefoobarcomm.log for a test server and C:appsprodefoobarcomm.log for a production server.
Using parameters in this way is a risky, manual process where you must eyeball and parameterize every relevant runtime value in a set of files. If one of these values is missed, the error would not be caught until runtime, after the .ear or .war files have been built and deployed. I have found at larger J2EE development sites that values that should be parameterized inevitably will get overlooked now and then, causing that embarrassing production “down” problem?all because of a very small parameter. Your end users will not understand; they only see that your application is unstable.
The burden for developers in tracking parameters is not trivial. They must know which properties need to be changed for each environment, but also which properties in which files have been parameterized. While there are situations where it may be necessary, parameterizing property values should be minimized and used for the smallest subset of files and parameters possible.
Identify Files that Often Need Updating
Fortunately, there are the usual suspects that will need to be updated for each environment during your build process. Look for the following file types in your application:
- XML files?Many tools are available for manipulating XML. XPath lets you target elements and attributes to change without the need for parameterizing the XML by adding tokens. There are excellent Ant tasks (XMLTask) and Perl modules (XML:XPath) that provide this functionality (see Listing 1).
- Properties files?These common files are defined by the java.util.Properties class and are basically simple lists of name-value pairs in a file. Ant is great for reading properties files, but has limited facility for regenerating them. The standard Ant tasks prefer the files to be parameterized ahead of time. You can also write a simple Java program for more sophistication. If you are comfortable with Perl, it can be as easy as processing a simple hash, or sophistication can be added using Perl’s powerful regular expressions.
- Other text files?As much as we try for standardization, some radical open source group or upstart company is bound to throw in some oddly formatted files you can’t do without. You may have to parameterize these files so that Ant can filter them or use Perl regular expressions.
- “Gotcha files”?There may be files that simply can’t be processed and must be used as is. These can be binary or license files required by third-party components. For Ant, the typical way to handle them is to set a property value indicating the location of the file to use for the current instance that you are building. With a commercial build tool, this problem may be easy to solve. For example using Openmake you simply set the search path to point to the file to use.
Track your Parameters
Managing parameters is really about documenting and tracking the configuration of each server and the infrastructure. I recommend you track this information in a spreadsheet. Using a spreadsheet does two things. First, it provides a central location for all of the configuration information that can be managed by different people. Second, the information from most spreadsheet programs can be exported as either XML or CSV files, which can be used as part of an automated process. The XML file can be processed using XML Path Language (XPath) using the Ant XSLT task, and the CSV format lends itself more to processing by Perl. See Table 1 for an example of spreadsheet column names, and see Table 2 for an example of the spreadsheet used to manage run-time values.
Table 1. Spreadsheet Columns for Managing Run-time Values
The spreadsheet columns are useful for managing the application and run-time information that may change.
| Spreadsheet Column | Cell Content |
| Environment | Identifies the type of environment such as development, testing, or production |
| Instance | Indicates a particular server when more than one may be in the same environment |
| Archive | The name of the JAR, WAR, or EAR file to be changed |
| File | The name of the file that contains values that change between environments and instances |
| Name/XPath | Locates and identifies the value to change |
| Value | Sets to the parameter value that is to be used for the environment and instance indicated in that row |
Table 2. Spreadsheet Rows for Managing Run-time Values
In the spreadsheet for managing run-time values, columns are identified as Env. for environment, Inst. for instance, Archive for the name of the JAR, WAR, or EAR file to operate on, and File for the file that contains a value that changes, which is embedded in the Archive file. The Token/XPath column provides the way to locate or identify the value to change, according to the type of file, and Value is the appropriate value to be used for the Environment and Instance indicated in that row.
| Env. | Inst. | Archive | File | Token/XPath | Value |
| Int | 1 | EComm.ear/ ECommEJB.jar | META-INF/ ejb-jar.xml | /ejb-jar/ enterprise-beans/ session[@id=’ApplicationData’]/ env-entry[@id=’EnvEntry_2′]/ env-entry-value | http://devlb:8040 |
| QA | 1 | EComm.ear/ ECommEJB.jar | META-INF/ ejb-jar.xml | /ejb-jar/ enterprise-beans/ session[@id=’ApplicationData’]/ env-entry[@id=’EnvEntry_2′]/ env-entry-value | http://testlb:8040 |
| … | … | … | … | … | … |
| Prod | 2 | EComm.ear/ Ecomm.war | WEB-INF/ server-config.wsdd | //service[@name=’Ecomm’]/ parameter[@name=’beanJndiName’]/ @value | PREPEND=”$STAGE$/” IFDEF=”$STAGE$” |
| Prod | 2 | EComm.ear/ | log4j.properties | @LOG_LOC@ | E:applogsprodecomm log4jlog4j.log |
In larger companies, using two spreadsheets is usually prudent. One spreadsheet is used for application-specific values determined by developers, and the other spreadsheet contains runtime infrastructure values. Security, change control procedures, or simply the size of an organization may require that a production control team separate from developers maintain the infrastructure spreadsheet. A Microsoft Access-type database can also be used, but you lose the advantage of file-based version control. If you have properties files all over the place, run?don’t walk?to your favorite spreadsheet tool.
Use the Development .war or .ear with the Spreadsheet
In most cases, the last .war or .ear that was created in the IDE by the developer is the version that should go to production. Your build process can use this .war or .ear to extract the source code. If you are using a version management tool, I recommend the developers check-in their last .war or .ear. The build process can than use this .war or .ear to retrieve the source code. This way you know you are simply rebuilding what the developer gave you, with modifications to the deployment descriptor files only. You then use the data from the spreadsheet to modify the deployment descriptor files. If you are looking at commercial solutions, look for one that is open enough to allow you to use a centrally managed parameter spreadsheet in the build.
If you are developing an in-house solution, you will need to write a script that will un-jar the .ear or .war, taking out the files you need to modify, update them based on the spreadsheet data, and put them back by reassembling the .ear or .war file. You can extract individual files from the .war or .ear files with a command-line JAR program or the Ant Unjar task. Remember, you can’t extract a file inside of an archive that is inside another archive. For example, you can’t directly extract a web.xml file inside a .war file that is archived in an .ear file. First, you will extract all of the contents of the .ear file and then operate on each archive in turn.
I’ve provided sample code to show you how to update your deployment descriptors using Perl and Ant xmltask replace and xmltask insert. Listing 1, update_deployment.pl, contains the Perl code and calls to the Ant xmltask replace and insert tasks used to perform the updating of your deployment descriptors. Listing 2 is the CSV input file, which would be similar to your Spreadsheet export. Listing 3 is a sample of a deployment descriptor that will be updated by running update_deployment.pl . Listing 4 is the results of the deployment descriptor after executing the update_deployment.pl script. This example code does not include extracting or re-archiving the deployment descriptor files from the .war or .ear. I’ve given you the hard code, extracting using jar or unzip is the easy step. You can download a zip file containing all four listings from the link at the bottom of this page.
Run One Build that Creates Many .wars and .ears
Now that you’ve extracted and modified the files, the last step is to put them back in the .ear or .war. Be super-organized about which files are source files and which are modified files. It is important to clearly separate files that are used as source and files that are modified for re-archiving. After re-archiving the new .ear or .war you are ready to repeat the process for next one. Include in your build process the creation of all .wars and .ears for your lifecycle. The build occurs once, but creates many versions.
You will need to create separate directories representing the different stages of the lifecycle to store your .wars and .ears. These files can be turned over to the testing and production release teams without having to re-execute the build for each stage of the lifecycle.
Creating Your Plan of Attack
Managing the creation of multiple .wars and .ears to support a lifecycle model offers new challenges for J2EE developers. The challenge is not in the coding or release of the .wars and .ears, it is in the build of the .wars and .ears. If you find yourself working strictly within an IDE to build your multiple .wars and .ears, it is time to consider creating a build process that works outside of the IDE. The process must be able to create all needed versions of the .wars and .ears using unique deployment descriptor files in one complete build.
Commercial build tools are available that can automate your J2EE build process including the creation of multiple deployment descriptors. In some cases a commercial solution is best as it can be easily repeated for any J2EE application, not just custom-written for a single J2EE application. If a commercial tool is not an option, consider a scripted solution that builds all .wars and .ears needed for the lifecycle in a single build pass. This ensures that the same source is used to create each unique .war and .ear.
If you follow the guidelines I have provided and review your server architecture early, you will save yourself from the embarrassing mistake of deploying a perfectly good .ear or .war to production, when it was configured for development. These recommendations have come from real-life success stories. They are intended to help eliminate some of the pitfalls that are common in J2EE development. Keep in mind that as upper management begins to look more closely at how development is managed, they will require a process that is more transparent and repeatable. Whether it is done by an enterprise build management tool or by an in-house custom script, moving beyond the IDE is critical in securing a transparent and repeatable build process for deploying .wars and .ears to production.
