Browse DevX
Sign up for e-mail newsletters from DevX


Change-proof Your Flat-file Processing with XML

If you build applications that use text files as input, try this technique for processing them into XML—and simultaneously insulate your applications from changes to the content and structure of the incoming files.




Building the Right Environment to Support AI, Machine Learning and Deep Learning

very new enterprise system built today has to integrate with and support existing systems. Since the earliest days of programming, passing data between systems via flat files has been a common operation—and developers get to write the code to transfer the data from one format to another. As unexciting as the topic may be, flat file processing is a required feature of any new applications developed for the U.S. government.

As XML becomes ubiquitous in modern systems, it's increasingly common for developers to take the flat files exported by older applications, and use them to construct single or multiple XML documents. The basic task of parsing an incoming record and creating XML nodes from that is simple; the real pain starts at the maintenance level. Any change to the flat file format involves tedious code changes to alter the construction of the XML file(s).

The basic task of parsing an incoming record and creating XML nodes from that is simple; the real pain starts at the maintenance level.
But changes to flat file formats are part and parcel of flat file processing. Ideally you would construct a flat-file-to-XML application that—if designed carefully—is immune to most flat file format changes, and can, by simply altering a few rules stored externally to the application itself, adapt the XML output to match the changes.

Here's an example. Suppose you need to capture patient appointment data for each doctor in a hospital software system. The software system exports patient data in a fixed flat-file format. What you need to do is update patient visits for existing patients and create a new entry for new patients. But the existing flat-file system doesn't differentiate between new and existing patients. Here's an example patient record:

// Sample flat file Patient appointment data: // Note: The following code would be one // line in the incoming flat file. P JAVA DEVELOPER73774777719740310 20874Programming stress disorders

Table 1 shows the field type and size information needed to parse the file and separate the fields.

Table 1: The description and length of each field in incoming patient flat-file records.


Length (Number of chars)

Record Type     


Patient First Name      


Patient Last Name       


Patient SSN     


Patient DOB     


Doctor ID Number        


Reason for visit        


The first automation step is to parse the incoming flat file and construct XML. Because you want to accomplish this generically, making the application adapt to changing flat file formats, you need to create parsing rules stored externally to your program. You can store the rules however you like, but it's convenient to store them in XML files or in a database table.

Comment and Contribute






(Maximum characters: 1200). You have 1200 characters left.



Thanks for your registration, follow us on our social networks to keep up-to-date