uring the past decade, I’ve witnessed remarkable change in the software industry’s overall approach to persistence. Though measuring Plain Old Java Object (POJO) solutions in terms of market share is tough, I’ve noticed a recent resurgence of JDBC programming. Some developers now roll their own JDBC solutions, while others look to increasingly robust JDBC frameworks. These frameworks allow some abstraction and provide better control over SQL, which can be important for performance, and automated frameworks can support a popular design pattern called data access object (DAO).

I think this trend is in part a response to the existing persistence alternatives. In the past, persistence frameworks often burned developers for many reasons, including the following:

• EJB container-managed persistence (CMP) still has serious technical limitations.
• Object-oriented databases never accumulated enough market share to take root.
• Many persistence frameworks could not provide adequate performance.
• Some proprietary frameworks failed, leaving their customers in the lurch.

Developers have long memories, and they’ve frozen persistence frameworks out in terms of investment. JDBC is simply safe for them, because right now there’s no dominant persistence product and no dominant persistence standard. The big freeze is still in effect, but persistence experts are beginning to see hope for a thaw and I’m seeing signs that the industry is ready to move again.

For example, an open source alternative, Hibernate, is prompting interest in transparent persistence. It’s a strong solution for medium-sized and smaller applications, especially when the database schema is directly under the control of the development team. Still, many larger customers will not consider an open source alternative, especially a nonstandard one, and Hibernate does not yet have the enterprise extensions or mapping flexibility of the best Java Data Object (JDO) solutions.

As an independent consultant and instructor who specializes in persistence strategies, I’ve seen my customers and students begin to consider JDO again. Their justification makes sense. Without a dominant proprietary solution, and with EJB in disarray, many are looking to the next best standard. It’s an incredibly important point in history for JDO. In this article, I explain how the industry has gotten to this pivotal point and offer my guidelines for the future success of JDO as a persistence solution.

EJB Crashes and Burns
Six years ago, only a few customers used persistence frameworks at all, and far fewer used them well. The top Java vendors and insiders pushed EJB hard, and the industry bought it. EJB introduced container managed persistence (CMP) as one of Java’s first full-service persistence layers. No one could really make CMP work, but few could resist the EJB hype, which was at a fever pitch. Many developers in larger development shops simply adopted EJB because it was hot, and they believed that CMP would fatten their resumes.

In its early stages, CMP was all but unusable. It could not manage relationships, and forced entities to use expensive remote interfaces with pass-by-value semantics. Still, modern-day knights of application development mounted up, drew their keyboards, and tilted at the windmills. In the end, rather than scrapping CMP and starting over, they implemented their own persistence: bean-managed persistence (BMP). I can’t imagine an uglier programming model. We had the worst of two worlds. BMP forced developers to code their own persistence, yet they still had to conform to EJB’s confusing and awkward interface.

Many of the largest J2EE shops were in limbo. They had committed a cardinal sin: They believed their vendors. Contrary to its hype, EJB complicated programming, seldom worked, and scaled poorly. Eventually, most developers cracked and conceded that they were in over their heads. Amidst the wailing and gnashing of teeth, WebGain joined the fray with its TopLink object-relational (OR) mapper.

TopLink Thrives
To this point, no persistence framework had yet experienced mass-market success. WebGain had been around for some time but only as a bit player. The vendor significantly grew its reputation and market share by providing TopLink, which could be used as a snap-in replacement for EJB CMP, enabling seamless persistence for that defective model. Along with providing rich OR mapping, TopLink supported the richness and complexity of relational databases. It also had a powerful caching model.

All this came at a cost, of course. TopLink was moderately complex and frighteningly expensive?especially when you factored in the cost of its consultants. But most customers were in so much pain that they paid the price. And it worked. TopLink had its warts, but if you could afford it, it made your application fast, reliable, and flexible. For the first time, Java developers experienced true insulation between the persistent model and the database schema. Still, most persistence experts in the Java space insisted that there was a better way. Enter Java Data Objects.

JDO Flounders
JSR 00012 (the Java Specification Request for JDO) requested transparent persistence for Java in 1999. If you used early versions of EJB for persistence, then you know exactly how frustrated developers were with it. At the time, leadership representing TopLink and other persistence solutions did not want a standard for OR mapping. They fought JSR 00012 with the kind of venom and fervor once reserved for Microsoft.

To get the JSR through, JDO proponents eventually produced a spec without OR details, which was exactly what OR vendors and RDBMS vendors wanted. After all, the market leader always wants to be proprietary. RDBMS vendors did not want an interface that opened their customers to OODBMS vendors. Likewise, OR mapper vendors would not support an interface that opened their customers up to their competition. While a few vendors were happy, most would-be customers weren’t.

The 1.0 version of the JDO specification finally made it out of committee in 2002, after three years of uncertainty and industry bickering?that’s three calendar years, not Web years. Unfortunately, the specification did not cover OR mapping, which was precisely what JDO customers wanted. The market did not respond well to the incomplete spec, the lack of big-time support by major vendors, or bytecode enhancement. Further, this young standard was trying to get established during a time when the economy was in shambles. Key players such as TopLink only partially adopted the standard, and even then, only early versions of it. Larger database and app server vendors held off entirely. Still, a core of strong JDO implementations began to emerge and evolve.

EJB Stagnates
Meanwhile, on the EJB front, customer failures began to stack up and the press began to sound off against the specification. Critics claimed that the framework was too big, too hard, and incomplete. In swooped EJB 2.0. The new CMP specification fixed the most glaring problems by adding local interfaces and allowing relationship management. A few customers began to have some moderate success with EJB CMP, but the EJB backlash was picking up steam. I watched with interest, because I myself was writing a book that criticized key parts of the EJB specification. I needed that market to buy my book, but EJB faded too quickly.

Key consultants and academics in the Java community began to understand that the EJB framework was far too invasive: EJB components depended on the specification and could not run outside of a container. EJB CMP developers could not take advantage of basic Java features such as inheritance or threading. EJB applications also were tough to build and harder to tune for performance. More importantly, the framework did not support the ideals of easy testing and simplicity that developers desired. By early 2003, the community was coming to the conclusion that EJB, in its current incarnation, did not stack up to the hype that the big J2EE manufacturers aggressively pushed.

TopLink Falters
The struggling economy finally took its toll on WebGain. The company collapsed, but not before selling TopLink to Oracle. Once that happened, new customers with non-Oracle databases or application servers felt uncomfortable making TopLink investments, and the market share numbers reflected their reluctance. Ironically, now TopLink was the underdog. It needed to support a major open standard like JDO or it would be too risky of an investment for its customers. Nothing else has stepped into the void, yet.

The Time Is Now for JDO
So here we are. EJB CMP is all but dead, since the major vendors are not supporting it with nearly as much fervor as before. No proprietary solution reigns supreme. Only JDO supports transparent persistence. Only Hibernate seems to be gathering momentum in the open source community, beyond simple POJO frameworks. That leaves a significant vacuum in this market. With the JDO 2.0 specification under way, the timing is right for JDO to seize this opportunity.

Here’s what it must do to take advantage:

1. Finalize the 2.0 specification quickly. If it takes the JDO expert group as long to get 2.0 out as it did 1.0, it’s dead. Right now, the 2.0 spec is scheduled to be delivered this year. It mustn’t slip.
2. Address OR mapping. The primary obstacles to OR mapping in a JDO specification are no longer in force. Specifically, no proprietary OR mapping solution is dominant, and relational database vendors don’t face any real competition. JDO 2.0 will include OR mapping support.
3. Embrace SQL. JDO needs a broadly accepted standard that addresses the needs of its community. JDOQL can still be supported for non-relational frameworks, but it won’t ever be a mainstream language. Hibernate shows that an SQL dialect can succeed in a persistence framework. Right now, SQL is a possibility as a pluggable query language. SQL (or a close derivative) should be included in the overall standard.
4. Separate implementation details from the specification. Don’t give customers another reason to reject JDO. Bytecode enhancement doesn’t need to be a problem. Let customers choose JDO for its benefits and then choose bytecode enhancement where they feel it makes the most sense. The JDO 2.0 spec will no longer mandate bytecode enhancement. The JDO expert group will gain the biggest benefit from allowing frameworks like Hibernate and TopLink to participate in the standard.
5. Don’t push binary compatibility. For most of JDO’s potential customer base, it’s not an issue. Few other J2EE standards enforce binary compatibility. Binary compatibility gets in the way of other important aspects of the spec. It looks like JDO 2.0 will make binary compatibility an optional feature, which existing vendors will continue to support.
6. Get big-name support. From the very beginning, lack of a big-ticket vendor has been a critical limitation for JDO. Oracle or IBM would be ideal. SAP also has a JDO implementation, and it is increasing its involvement with the JCP. Right now, TopLink and Hibernate appear to be strongly considering support of the JDO 2.0 standard. Either or both would go a long way toward relieving customer concerns that JDO is a small-time player.

Like most Java developers, I feel that J2EE does need a standard for transparent persistence, and I think that JDO 2.0 has the best chance to be a reasonable alternative to entity beans. Others agree. It’s starting to show in the press, and beyond:

• I understand that one of the flagship JDO vendors, SolarMetric, is experiencing outstanding success, with two consecutive quarters of growth.
• The JBoss group is offering a JDO implementation.
• The Sun ONE application server now supports the standard.
• Membership at the JDOCentral Web site is increasing dramatically.

With an emphasis on better, lighter solutions, other vendors could well follow suit. The economy is once again growing, creating a great atmosphere for a new successful standard. Developers are hungry for persistence frameworks again. My “Java Persistence Strategies” class at the No Fluff, Just Stuff Java Symposiums is experiencing tremendous growth in enrollment, and people are talking about JDO again. If JDO doesn’t step into this vacuum, someone else will.