Tracking Events Using a Semi-Structured Repository

ave you ever used a multi-user system and found yourself wondering, “How did this get changed?” or, “Who created this and why?” Helping your users track the system changes and the reasons for these changes is an important part of running an efficient business. However, these questions are not easily answered by many multi-user systems.

Questions do not only come from the users of a system. Just as frequently, the developers of a system need to understand what was happening in the system at the time of an unexpected event. Developers sometimes find it difficult to determine what actually happened solely by looking at issue reports.

Having a good integrated tracking system can save both the users’ time and the developers’ time by answering questions quickly.

Adding event tracking to an already developed system does not require significant changes. On the contrary, as this article demonstrates, you can track system events without much up-front effort and achieve immediate benefits through the ability to browse past events and query them to answer questions.

Logging Exceptions
Every developer, at some point, has wished that more time was spent collecting the state of the system during an exception. Relying on users to take a screenshot or copy an exception message often does not yield enough reliable information, particularly when you need to resolve multiple concurrent issues. Typically, what you find for exception reporting in many Java applications is a try-catch block that looks something like this:

try {  // do something here} catch (SQLException e) {  logger.error(e.toString(), e);}

The logging statement might contain a message explaining what the code above was trying to do or just be e.printStackTrace();. Either way, there is often not enough information in this log entry or printout (if the printout is even readable) to determine the initial cause of the problem.

The lack of information in a stack trace is sometimes addressed by loading the exception message with additional information. The message property can only hold a limited amount of information and quickly loses the purpose of a readable message. Another solution is to add properties to an exception, such as an error code, or state information. The exception may also contain a cause or a list of exceptions. However, this information does not always make it into the log entry or print out. A common example is JDBC’s batch exception. Some JDBC drivers use the SQLException’s nextException property to link exceptions together, but these critical properties are not always reported.

There is a better way to preserve the exception in its entirety and easily locate it later: persisting the exception and all its properties into a local repository for later review.

Some implementations use a database to persist exception information, but this is difficult to maintain because as new exception properties or types are added, you also need to modify an independently maintained database schema. Instead, using a semi-structured repository can significantly reduce the overhead of maintaining a persistent JavaBean pool.

Rather then printing the exception to a flat file, you can persist the exception to a repository as shown below.

try {  // do something here} catch (Exception e) {  LogManager.log(e);}public class LogManager {  private static EntityManagerFactory factory = createEntityManagerFactory("rdf");  public static void log(Throwable t) {    EntityManager manager = factory.createEntityManager();    manager.merge(t);    manager.close();  }}

However, you must inform the manager what information needs to be persisted in the exception and how it should be extracted. To do this with the OpenRDF Elmo library, you can create an interface that contains all the exception properties and annotate it with unique identifiers for the class and properties.

  package org.example.logging.concepts;  @rdf("http://example.org/rdf/logging#Throwable")  public interface IThrowable {    @rdf("http://example.org/rdf/logging#cause")    IThrowable getCause();    void setCause(IThrowable cause);    @rdf("http://example.org/rdf/logging#message")    String getMessage();    void setMessage(String message);    @rdf("http://example.org/rdf/logging#stackTrace")    List getStackTraceElements();    void setStackTraceElements(List list);    @rdf("http://example.org/rdf/logging#date")    Date getDate();    void setDate(Date date);  }

Then you need to provide a method to copy the properties from the Exception into the managed interface.

  package org.example.logging.support;  public abstract class ThrowableMerger implements IThrowable, Mergeable, Entity {    public void merge(Object source) {      if (source instanceof Throwable) {        Throwable t = (Throwable) source;        setMessage(t.getMessage());        ElmoManager manager = getElmoManager();        StackTraceElement[] stack = t.getStackTrace();        setStackTraceElements((List) manager.merge(Arrays.asList(stack)));        setCause((IThrowable) manager.merge(t.getCause()));        setDate(new Date());      }    }  }

You also need to do the same to the child classes, if applicable (see Listing 1).For the code in Listing 1 to work correctly you need to include a persistence.xml file in the class-path like the one in Listing 2.

You will also need to include the file org.openrdf.elmo.roles in the classpath to the registered merger classes and the interfaces.

• # META-INF/org.openrdf.elmo.roles
• org.example.concepts.IThrowable
• org.example.concepts.IStackTraceElement
• org.example.concepts.ISQLException
• org.example.support.ThrowableMerger
• org.example.support.StackTraceElementMerger
• org.example.support.SQLExceptionMerger
• org.example.exceptions.MyException
• java.lang.Throwable = http://example.org/rdf/logging#Throwable
• java.lang.StackTraceElement = http://example.org/rdf/logging#StackTraceElement
• java.sql.SQLException = http://example.org/rdf/logging#SQLException

You can include other properties of other classes in a similar way. For classes that you can modify, you can add annotations onto the getter methods instead of creating a merger class. If you are not interested in reading the persisted repository from Java (just using the browser and scripting languages) you can also forgo the interface and place all the annotations on the exception class.

package org.example.exceptions;@rdf("http://example.org/rdf/logging#MyException")public class MyException extends Exception {  private int code;  public MyException(int code, String message) {    super(message);    this.code = code; } @rdf("http://example.org/rdf/logging#code") public int getCode() {    return code; }}

Now that your exceptions are being persisted in a repository, take a look at some of the tools available to inspect them.

Inspecting the Repository
A convenient way to query and inspect the repository is through an interactive shell. In Java 6 for Windows, Linux, and Solaris, there exists a tool called jrunscript, which is a JavaScript interactive shell. If you are not using Java 6 on one of these platforms, you can download Mozilla’s Rhino interactive JavaScript shell or use an interactive shell for another language, such as Groovy or JRuby. However, the syntax varies between shells.

Be sure to set the CLASSPATH environment variable correctly to include all the jars from the application.

Now start the interactive shell using the command:

jrunscript

Just like when writing Java code, you need to import the packages you plan on using. This is done using the following commands:

• importPackage(java.io)
• importPackage(java.util)
• importPackage(javax.xml.namespace)
• importPackage(org.openrdf.elmo)
• importPackage(org.openrdf.elmo.sesame)
• importPackage(org.openrdf.rio)
• importPackage(org.openrdf.rio.rdfxml)

You can also execute these in the command line by using the ‘-e’ argument followed by the command and then ‘-f -‘ to jrunscript.

Run the following commands to open a connection to the repository:js> factory = new SesameManagerFactory(new ElmoModule(),“devx”,“rdf”))
js> manager = factory.createElmoManager()

OpenRDF Elmo uses the newly standardized SPARQL query language.

To execute a query, run the commands from Listing 3.

Although this shell is useful for arbitrary inspection, it is difficult to get the bigger picture. However, you can also export the data into RDF/XML (a portable format). With the repository exported you can browse the contents more interactively in your browser.

js> manager.connection['export'](new RDFXMLWriter(new FileWriter("export.rdf")))

Browsing the Repository
It can be difficult to browse larger repositories, but if your repository is small and portable (<1MB) you can view the file using MIT's Simile web service called Babel. Otherwise you may want to set up an HTTP repository and view the results through the openrdf-workbench, included in the OpenRDF Sesame SDK.

To view the repository in the browser using an RDF/XML file, open your browser to SIMILE Babel and choose to convert from RDF/XML to Exhibit JSON. Browse for the RDF/XML file you just created, then click the “Upload and Preview” button to see the result. This allows you to search and browse the contents of the file using Exhibit.

Figure 1. Preview in Exhibit: Exhibit allows you to view the content, sort by any field, and filter the results by a field.

By using Exhibit you can view the contents, sort by any field, and filter the results by a field. This gives you a quick view into the activities of the system. Exhibit is also very customizable and can be used to create a catered view of the data. For more information about Exhibit, see the web site. Figure 1 shows an example of a Preview screen in Exhibit.

Tracking Events
Multi-user applications often do not properly track system events. This is sometimes caused by the complexity involved in persisting them. In any system there are many types of events (sometimes hundreds of types). When using a database to persist events, the overhead for maintaining a database schema with so many event types, and new event types being created with every new feature, is high. However, by using a semi-structured repository, the overhead for persisting events is greatly reduced.

To persist events in OpenRDF Elmo you need to create an interface for the event objects. You can use the same technique for persisting events as used previously for persisting exceptions. Create an event type interface as shown in Listing 4.

You can also create an event implementation class for this interface and merge it into the manager as shown previously, or use the delegate manager’s designate method, shown here:

private static EntityManagerFactory factory = createEntityManagerFactory("rdf"); // singletonEntityManager entityManager = factory.createEntityManager();ElmoManager manager = (ElmoManager) entityManager.getDelegate();ModifiedEvent event = manager.designate(ModifiedEvent.class);event.setUser(user);event.getModified().add(target);event.setDate(new Date());manager.close(); // event can no longer be used

The user and modified targets will also need to have interfaces, with annotations, listed in the elmo.roles file for them to be persisted. Objects that are referenced by multiple objects or events, such as user and modified targets, should have a getQName method. This method must return a qualified name that will be used to uniquely identify it in the repository.

With events now stored in the repository, you can use the repository to provide a history of the system. For example, to find out how an object was modified, you can query the repository.

js> sparql = "PREFIX evt:  SELECT ?event WHERE { ?event evt:modified ?target }"js> query = manager.createQuery(sparql)js> query.setQName("target", new QName("http://example.org/rdf/orders/", "209"));js> event = query.singleResultjs> println("order 209 was modified by" + event.user);js> println("order 209 was modified on" + event.date);js> println("order 209 was modified along with" + event.modified);

Tracking creation events is a similar by creating an interface and persisting it like in Listing 5.This event has an extra property reason that stores the user’s inputted goal at the time of creation. This gives the user a chance to provide the reason why they are making a chance. This information can then be later retrieved using a query and provides necessary information to assess the correctness of the operation.

js> sparql = "PREFIX evt:  SELECT ?event WHERE { ?event evt:created ?target }"js> query = manager.createQuery(sparql)js> query.setQName("target", new QName("http://example.org/rdf/orders/", "209"));js> event = query.singleResultjs> println("Order 209 was created by" + event.user + "on" + event.date + "for" + event.reason );

Conclusion
Semi-structured repositories can provide more flexibility and less maintenance than traditional persistence layers. This type of repository is ideal for structures that are continually changing, have diverse properties, or are managed by independent teams. You can use the OpenRDF Elmo library to persist a wide variety of objects without necessarily modifying the original source. Without a lot of setup, you have seen how to create a repository, use it in your own software, and access it through a convenient search and browser interface.