RSS Feed
Download our iPhone app
Browse DevX
Sign up for e-mail newsletters from DevX


NetKernel: Moving Beyond Java’s Concurrency

Despite Java's support for multi-threading and concurrency constructs, developing applications that fare well on modern, multi-CPU hardware can be difficult. Alternate environments like NetKernel might be an easier path to harnessing extra processing elements.

s multi-core and multi-CPU systems become more prevalent, the opportunity to perform several tasks at once is now a reality. Unfortunately, the way most systems are designed it is not as easy as having another thread take a task on. The programming language you use needs to ask the execution environment to schedule work on system resources. The constructs of a language influence how easy it is to take advantage of higher-order concurrency functions.

Historically, you could hand things off to a "helper" through multiple threads. As the CPU waited for network or file I/O activity to complete on one thread, it could execute another thread to get some work done and switch back when the first one was ready. This certainly helped applications feel more responsive, but the computer was still really only doing one thing at a time.

Prior to the Java language and JVM, only sophisticated developers could wrap their heads around these concurrent-programming models and the opportunity for reuse never reached its full potential. Because the constructs were not directly part of the language, you needed to choose a threading library and stick with it. Any libraries built on top of those threading libraries were often incompatible with other approaches. POSIX standardization helped, but it remained complicated and out of reach for most software engineers.

Java elevated the concurrency game by introducing relatively easy, relatively cross-platform threading mechanisms at the language and JVM levels. It was exciting that Java supported threads, the Runnable interface and monitors. Some early JVMs supported native threads while others only offered "green threads," which were behind-the-scenes JVM magic that approximated native thread concurrency. With these basic tools, it became simple to write fairly portable multi-threaded code:

Thread t1 = new Thread() {
    public void run() {
        System.out.println("Hello from Thread 1!");

Thread t2 = new Thread() {
    public void run() {
        System.out.println("Hello from Thread 2!");

The good news is that this was easy to do. The bad news is that this profligate use of Thread instances was expensive and did not scale well. Simply creating a ton of Threads would bring an application to its knees swapping contexts between all of them. Developers were encouraged to avoid creating new Thread objects and instead use Runnable instances scheduled on some kind of a Thread pool. There was not any standard ThreadPool instance for several years, but many effective ones were developed and proliferated in the meantime.

Runnable r1 = new Runnable() {
    public void run() {
        System.out.println("It's good to be an r1 Runnable!");

Runnable r2 = new Runnable() {
    public void run() {
        System.out.println("It's good to be an r2 Runnable!");

// Create a ThreadPool with 3 threads waiting for something to do
ThreadPool tp = new ThreadPool(3); 
This model, in which a pool of threads handles client requests, scaled reasonably well and ultimately formed the server infrastructure base for many organizations. A pool of threads would handle client requests. Most servlet engines worked under the hood like this, but even thick Swing clients appeared more responsive by off-loading longer running tasks like querying a database or issuing an RMI request to another thread (failure to do so resulted in unresponsive and unpopular Swing applications!).

It seemed as if the Java platform, like Prometheus before it, had stolen Multi-Threaded "fire" from the Gods of Concurrency and made it available for all Developerkind. The downside of giving people fire to play with is that it can burn! This model usually required the use of listeners for notification when a scheduled task was done. This decoupled the normal flow of a program, which usually makes things harder to follow. Also, after developers saw what they could do, they wanted to eke out ever more performance. Rather than straight task queuing, they wanted to have priority queues. Doug Lea famously introduced rich and exotic concurrency structures like Syncs, Barriers, Rendezvouses and the like to address these and other issues. This was all very exciting and sophisticated developers built powerful software, but average developers frequently still ran into problems.

The state-based nature of Object-Oriented programming is difficult to get right when multiple threads might attempt to update or retrieve this state. Multi-threaded errors are among the most difficult to debug because they often appear random and hard to reproduce. The Swing team understood this and decided to make Swing a single-threaded API. While some folks complained about this limitation, the decision was completely about making it simple for developers to create new components. Given the popularity of "Java Concurrency in Practice" over 10 years after Java was released, it is clear that many programmers still struggle with getting it right.

When you run into production code that looks like this, something is not right:

synchronized(new Object() {}) {
    // ... unsynchronized synchronized block here
    // DO NOT EMULATE THIS CODE! Why won't this do what 
    // intended?
Imagine moving away from low-level state management and into higher-order functionality like issuing web service calls, querying databases or transforming XML. These workflows impose new burdens on how processes are interrelated, how often a service can be invoked, etc. These processes often change based on business rules, so you do not want to have to rely on language-level constructs for coordinating all of these activities. In order to fully benefit from ThreadPools, Executor frameworks and similar constructs, you need to write your code around interfaces such as Runnable and understand the various contexts under which you might use the code. Sometimes you may wish to block on a step, sometimes you might not. Sometimes you might want to orchestrate asynchronous calls to multiple data sources and then transform the results into HTML. If not everything is separable into unrelated, executable blocks of code, you might find it difficult to scale to take full advantage of extra CPUs. Even if you somehow designed your applications to work that way, your code might be deployed into an environment like a modern application server that controls what threads can and cannot do.

Despite Java's rich and powerful platform and language-level features, using them correctly and effectively is harder than it needs to be for many applications. These issues are pushing developers to consider alternate languages like Erlang, Scala, and Clojure. These functional and hybrid languages introduce new concepts like Software Transactional Memory and programming styles that minimize the amount of state maintained and needing to be protected by concurrency locks. Scala and Clojure both run on the JVM so it is possible to reuse a fair amount of your Java code, but Erlang requires new tools and runtime environments.

Rather than simply changing languages, if you are willing to look to a next-generation runtime environment, you might find that it is easier than you think to write software that takes full advantage of a system's cores or CPUs while reusing much of what you have already written. The trick is to shift from an object-oriented mindset to one that focuses on information resources. Objects are still used as an implementation technology, but your code dependencies become resilient to change and remain concurrency-friendly!

Introducing NetKernel
1060 Research Ltd. has created a URI-based microkernel environment called NetKernel to help solve some of these problems, take advantage of multiple CPUs seamlessly, and build logically-connected, layered applications. NetKernel has a dual license allowing both open source and commercial development. This article focuses on how NetKernel simplifies building high-concurrency systems without much effort. The full scope of NetKernel's resource-oriented approach is beyond the scope of this article, but you are strongly encouraged to read more about it.

NetKernel's magic starts with its well-designed microkernel architecture. It is an efficient environment that can do "real" work in a 10MB VM (many production NetKernel applications operate effectively in 64MB or less!). Out of the box, NetKernel only uses a handful of threads although this can be tuned if your requirements demand more. Requests come in through a transport (usually HTTP, but others are possible) and are scheduled asynchronously on one of the microkernel threads. This is a key point to remember. Everything is ultimately scheduled the same way: asynchronously. For convenience sake, synchronous patterns overlay this asynchronous backend. This is a key aspect to NetKernel's ability to scale. Every incoming request is handed off to a different kernel thread whether you intend it to or not. You can control this behavior as you will see below, but this environment already is compelling from an operational, CPU utilization perspective. The same application will probably take advantage of extra CPUs automatically and you do not have to do anything!

Everything in NetKernel is URI-addressable. Files, specific functionality, everything! Internally, NetKernel uses a URI scheme for referencing behavior called active URIs. There are also data URIs for referring to things like files on disk. If you want to retrieve a file from disk, you just ask for its URI. If you want to invoke some XSLT processing, you schedule a request for the functionality that responds to XSLT requests (active:xslt). Behind the scenes, there is a module that advertises this URI. It will be asked to perform the transformation with specified inputs. While you do not need to care what is actually used to do the transformation, you may be interested to know that it is Saxon, the faithful Java XML transformation library. You can often wrap up existing code with a logical URI. Dependent code can invoke it without having to directly create or talk to your object instances. This way, if and when you decide to swap the implementation technology, clients do not have to care. Sure, you can approximate this flexibility with Java interfaces, but it is much cleaner and easier with just a logical name. This is one of the harder shifts for Java developers to make when looking at resource-oriented environments, but once you do, it is hard to go back to the old way of thinking.

When you ask NetKernel to run something, you create a subrequest and ask for it to be executed. Although it ultimately runs on a microkernel thread behind the scenes, from the client perspective, it blocks until the activity is done. This is how most developers want code to behave unless they specify otherwise. NetKernel itself is written in Java, but uses the resource-oriented abstraction to invoke behavior. You are free to implement modules or write client code in just about any major language that runs on the JVM (e.g., Java, JavaScript, Groovy, python, Ruby, BeanShell and even Scala and Clojure are becoming options!).

See Sidebar 1 for information on running the examples and see Sidebar 2 to read about BeanShell.

Close Icon
Thanks for your registration, follow us on our social networks to keep up-to-date