RSS Feed
Download our iPhone app
Browse DevX
Sign up for e-mail newsletters from DevX


Implement Document Storage and Search on Google Java App Engine

Learn how to use the Java version of the Google App Engine by implementing search and document storage on a Java web application running on the platform.

hy should a Java web developer accept Google's recent invitation to use the Java version of its App Engine platform? A few of the most compelling reasons are:
  • Google provides free hosting to web applications that receive fewer than 50 million page visits per month. You can convert App Engine into a paid-for service when you go over the free use quota.
  • Google provides scalability with no additional development effort when you work within the limitations of the App Engine platform (both the Java and Python versions).
  • The Admin web interface allows you to inspect runtime errors in the error logs, browse your application's datastores, check the application's performance (for sample, the statistics on request-processing times), and generally monitor your deployed applications. Google's Admin web application compares favorably with Amazon's excellent EC2 Admin web console.
  • You can move your applications to your own server and run them there using the App Engine SDK. You will lose the scalability of a hosted application, however.
  • Because you use standard APIs (with limitations) to develop Java App Engine applications, you can move to other deployment platforms with relatively little work. Unfortunately, the opposite is not as easy: if you code to a large subset of J2EE APIs, rely on a relational database, etc., then porting to the App Engine may require a lot of effort.

Those who have written web applications with the J2EE software stack might find the limitations of App Engine off-putting at first, but the reward is reduced server costs. If you cannot live within these constraints and need greater scalability, then you might want to consider Amazon's EC2 services (I use both App Engine and EC2).

This article teaches Java developers how to use Google App Engine. It demonstrates how to implement search and document storage on a Java web application running on the platform. Along the way, the article explores useful techniques and application ideas that augment what is already in the Java App Engine documentation.

What You Need
Eclipse or IntelliJ IDEA
If you don't have an App Engine account, then sign up here.*
Download and install the App Engine SDK for local development.
Install the Eclipse plugin or IntelliJ IDEA plugin for Java App Engine.

*You can experiment with App Engine for a while without an account and just run your experiments on your laptop using the App Engine SDK.

Figure 1. Roadmap of Files in the Sample Project: Here are the sample project files as an Eclipse Java App Engine project.

Roadmap for the Sample Web Application

Many Java developers use Lucene (or some framework based on Lucene) to implement search. However, using a Lucene in-memory index model with App Engine doesn't make sense in a production environment. The sample application in this article provides an alternative search implementation.

App Engine's persistent datastore is very efficient, but it does not use a relational model, and it will not work with Object Relational Mapping (ORM) frameworks such as Hibernate. However, App Engine does provide support for some standard persistence APIs such as Java Data Object (JDO), Java Persistence API (JPA), and JCache. The sample application uses JDO to implement data persistence.

The application is deployed and running here. Anyone using this deployed sample can delete all the persistent data and start over, so information that you add may not be available the next time you look at it.

Author's Note: This application demonstrates the use of JDO and one approach to implementing search on top of JDO. It is not a complete web application with support for multiple users, user accounts, etc.

Figure 1 shows the sample project files as an Eclipse Java App Engine project. The sections to follow go into some detail about the model classes in the package com.kbsportal.model and the persistence utility class PMF in the package com.kbsportal.persistence. Because the Java utility classes in the package com.kbsportal.util are simple Java classes with nothing specific to the Java App Engine, they will not be discussed at all. You can take a quick look at the source code to learn more about them and about the JSP files (in the directory war/WEB-INF). A few of the embedded Java code snippets in the JSP files will be discussed.

Using JDO for Data Persistence

JDO is an older Java API for persisting Java objects. Originally, JDO required developers to write and maintain XML files that specifically mapped data attributes in Java classes to persistent storage. Google uses the DataNucleus tools to automate this process. You need only supply annotations in your Java model classes and the DataNucleus tools automatically maintain the correct mappings for you. Using either the Eclipse or the IntelliJ IDEA App Engine plugin support, the DataNucleus tools automatically run in the background for you whenever you edit a persisted model class.

Warning: Don't get caught up using JDO and App Engine only to run into compatibility problems with your existing persistent datastores when you modify your model classes. When you develop locally with Eclipse, simply delete the file in WEBAPP/war/WEB-INF/appengine-generated/local_db.bin. If you have a deployed web application and change the model classes, then you sometimes will need to delete existing indexes for the application using the App Engine console for the deployed application.

The following sections describe implementing two persisted classes and discuss JDO-specific code and techniques as needed.

Close Icon
Thanks for your registration, follow us on our social networks to keep up-to-date