Wednesday, December 16, 2009

Lift from a Wicket Developer's Perspective

I've been messing around with Scala again lately. After learning the basics of the Scala language, I decided to have a look at the lift web framework - "the simply functional web framework". In this highly subjective post I will outline, why I'll stick with Wicket.

My Lift Experience


To be clear, I'm not an experienced Scala developer. I did my first steps in Scala a year ago or so, and I just started learning Lift by having a look at the getting started guide. But this short first impression was enough for me to decide, that I'm not going any further with lift. Without going into much detail, here's why:

  • Lift is a template driven framework, whereas Wicket is component based. In Lift you compose a page of (X)HTML templates, called snippets. In Wicket you compose a page of components, i.e. not through (X)HTML but through an object graph/hierarchy. Lift's template approach allows for reusable templates, but I like the Wicket's object oriented, hierarchical approach much more.

  • There's not much documentation on Lift. There is the getting started guide, but the examples did not work immediately, because the API had changed. This troubled me a lot. Also, there's not much documentation in the source code.

  • A positive aspect of Lift is that it comes with an integrated OR-Mapping API, which looks quite interesting. Also there's a nice query API that can be compared to Hibernate's criteria API, and at first glance it looks even nicer. I heard that lift also runs on Google App Engine (?). I wonder, though, how hard it was to get this integrated OR-Mapping/Query API to run on GAE.

  • The model/entity classes have too many responsibilities. They contain validation logic and OR-mapping logic. This is okay so far. But there's also conversion logic in there, e.g. from String to the specific data type, and there's also code for HTML and even JavaScript generation. The MappedBoolean class for example, which is used to represent a boolean property of an entity, also has methods like toForm, which yields a (X)HTML checkbox element, and asJsExp, which yields the corresponding JavaScript expression of the value. I'd rather seperate these responsibilities into more than one class.

  • In general, the seperation of presentation content and logic is not quite as good as stated in the getting started guide. Here is an example snippet from the getting started guide:

    Without getting into the details, this method contains presentation logic in the form of HTML span elements and a checkbox input element generated by the ajaxText call. Actually, these aren't really HTML element strings in the code but this is Scala's XML support. But that does not make it better, right? You'll never - well, almost never - see such things in Wicket.

  • Lift is a Scala web framework, Wicket is Java. And although Scala in general is a great language and superior to Java in many aspects, I'm still not quite sure whether it will become my first language of choice ever. I still have some issues with it.

  • Related to this is the tooling aspect. The IDEs (Eclipse, IDEA) still lack full support for the Scala language. Maven support for Scala/Lift is quite good, though.



So, this is was my first impression of Lift. Don't get this wrong, Lift has some interesting aspects and in some ways it's much better than many other Java frameworks. But if I had to choose a web framework for a green field project right now, I'd certainly go with Wicket.





Friday, November 13, 2009

Concurrency is not such a big Deal ...

... when it comes to web development.

In the current debate about the future of Java (Java the language, of course) and which alternative language would be an appropriate successor of Java (Scala, Clojure, ...), much of the discussion focuses on the question, which language provides the better concurrency model in order to utilize multi-core processors in the future. Actually, I'm not an expert in concurrency, and concurrency clearly is really hard to get right (at least in Java), but usually in web applications - and this is quite a huge part of the Java applications out there - concurrency is not such a big deal, even despite the fact, that web applications are one of the most concurrently running kind of applications.

Concurrency in typical web applications
Surely, most web developers know exactly how to tackle concurrency in web applications. It's not that hard. But for the fun it, let's have a closer look. In a web application client requests are handled by the app server concurrently, each in its own thread, possibly distributed among cpu cores. So generally, there are two situations, where threads run simultaneously:

  • Two or more requests of different clients

  • Two or more requests of the same client

These threads may possibly access (read and write) shared state. And concurrency is hard mainly because of mutable state, accessed by different threads simultaneously. But let's put it another way then:

  • Concurrency isn't hard for immutable state and

  • Concurrency isn't hard for state that is not accessed by different threads simultaneously (mutable or not)

Now let's have a look at what types of state there are in a typical web application and how it is accessed by different threads:

State of the current request

This is for example data needed to generate the response of the request, status of the request, and so on. In most cases, this kind of state is only available to a single thread, i.e. not accessed simultaneously. This is guaranteed by all application servers for request state held in the app server. And this is guaranteed by all web frameworks (also with front controller pattern) for request state held there. The developer has to take care not to store request state where it can be accessed by other request threads. This is neither hard to understand nor hard to accomplish.

Application state held in the application server or in the web framework

Examples are ServletContext or web framework settings. This is mostly static state that will be initialized at startup but won't be mutated in the lifetime of the application. Otherwise, the application server/web framework will take care that this kind of state is accessed in a thread-safe way.

Conversational state

HTTP is stateless. There is no conversational state. There is not even a conversation. Unfortunately, this is not exactly true in most web applications, as most applications will store some kind of conversational state in the HTTP session. This type of state might be accessed simultaneously by multiple threads, but only by threads spawned by the same client. In most applications concurrent access to this state is rare and might be well supported by the web framework. Nonetheless, this is state the developer has to take care of.

Application state persisted to the database (business state)

In fact, this state is read and mutated highly concurrently by different threads. Fortunately, most persistence frameworks support the unit of work pattern: State is replicated from the database for each thread accessing it (as a copy). The thread modifies it and writes it back in a database transaction. Modifying state in this way (in a transaction) is an appropriate concurrency strategy. See also software transactional memory (STM), which is supported e.g. by Clojure.

"Classical" concurrently accessed state

For example heavy calculations done explicitly concurrently. In web applications, a request might spawn several other threads to do an expensive calculation. This is the "classical" case, where thread programming might surely become hard. This is definitively a situation where Scala or Closure or any other language with a better concurrency model might be appropriate.


Summary
In summary, if there is no state, there is no problem. Immutable state is also not a problem. State, that is only accessed by a single thread is no problem. And state that is modified in a unit of work/transaction is not a problem. And in web applications most state is of one of these kinds (and by the way, this is mostly because of the stateless, request based nature of HTTP). In this sense, Java is quite well prepared for the multi-core future in the field of web applications, although other JVM languages might have the better concurrency model. Nonetheless, there are some even better arguments than concurrency for some alternative languages, for example expressiveness, conciseness or productivity in general.


Update: I did a "prezi" of this post. What a fun tool! See here.





Monday, October 26, 2009

A Scala Console with JavaFX (Experimental)

It's been a while ago, when I started implementing a Scala console with JavaFX. And orginally I've never thought about publishing it, because it was (and still is) an experimental, personal project. But today I started playing around with it again, and I decided to push it to github and write a short post on it. So, if you are interested, you are welcome to read the source code, try out the application and give some feedback.

The JavaFX part
For the JavaFX part of the app, which is of course the UI part, I used Netbeans. It simply consists of a textarea for the input, another textarea for the output and a toolbar to execute the input or clear the input.


Admittedly, the UI could be much nicer, especially with JavaFX. But as said, this was just an experiment and I'm not quite a UI guy.

One thing that struck me, was that JavaFX did not have a textarea component. At least at the time I was building the application. I can't say, if there is one today. Fortunately, I found a blog post on the web (but I can't find it anymore), where the author describes how to wrap a Swing textarea into a JavaFX component. This worked quite nice. Adding some JavaFX binding and some event handling functions to the buttons, and the UI part was working. Doing the design part with JavaFX was real fun.

The Scala Part
I implemented the interpreter part in Scala as a separate project in Eclipse. I also used Maven with maven-scala-plugin. I had some issues with the Eclipse Scala plugin, but the Maven plugin worked quite nice. Scala comes with an Interpreter class to interprete Scala code. It would be really simple to execute a whole script at once with the interpreter, but I wanted the application to interprete the code line by line, just as the the Scala console does that comes bundled with the Scala distribution. So, I used Scala's case class pattern matching do accomplish this, see the source code for more details.

Bringing it together
There's nothing special to bring the two parts together as everything runs on the JVM. I bundled the Scala part into a jar with Maven and added it together with the Scala jars to the JavaFX project. Instantiating the Scala classes from JavaFX is nothing more than calling new.

Resources
[1] Project (Source code and WebStart) at github




Saturday, October 24, 2009

GroovyBot - Developing a Groovy Google Wave Robot

Google Wave is a communication and collaboration tool developed by Google, which is currently in preview and available only to a limited number of users. Last week, I was chosen to be one of those users, and as there's also a Java API available to extend the platform, the first thing that came to my mind was to build a Groovy robot.

This post gives a short introduction of how to build a Google Wave robot. The goal is to build a robot that takes user input as a Groovy script, evaluates it and displays the result back in the wave. To avoid any misunderstanding, the robot itself is not written in Groovy but in Java (shame on me ;-) and uses GroovyShell to evaluate Groovy code. For the impatient (with a Google Wave account) who want try the robot, it's address is groovybot@appspot.com. See the resources section for source code.

Introduction to Google Wave
Google Wave is a communication and collaboration platform, where people can participate in conversations. It's kind of like an e-mail conversation but in real-time like a chat, and much more interactive, with much more features. A conversation, consisting of one or more participants, is called a wave. A wave is a container for wavelets, which are threaded conversations, spawned from a wave. Wavelets are a container for messages, called blips. For more details, see the Google Wave documentation.

Google Wave Extensions
There are two types of Google Wave extensions - gadgets and robots. A gadget basically is a component containing some HTML and JavaScript, that can be included in a wave conversation. But as GroovyBot currently does not make use of gadgets (yet), I won't consider gadgets any further in this post. The other type of wave extension is robots, and as the name suggests, GroovyBot is of this type of extension. A robot is a certain kind of web application that can participate in a wave conversation just as a human participant. And just as a human participant, it can be added to a conversation and then react to certain kinds of events in the wave, e.g. when it's added to the wave or somebody submitted a message (a blip). In response, it can then modifiy the content of the conversation, e.g. by appending content or replacing content.

Getting Started
Currently robots have to be deployed to the Google App Engine. Fortunately, it's really easy to get started with Google App Engine. First you'll need to sign up for an accout. Then with the Eclipse plugin for the app engine create an application, just like shown in this post. This will create a simple Servlet web application, which can immediately be deployed on app engine. But we don't want a simple servlet application. Let's see how to build a wave robot.

Building a Robot
First of all, we need the Google Wave Java Client API, which consists of four jar files, and (for this particular robot) the Groovy jar groovy-all-1.6.5.jar. Place these in the war/lib directory and add them to the build path of your Eclipse project. Then we create the robot, which is simply a class extending AbstractRobotServlet. We'll name it GroovyBotServlet:

public class GroovyBotServlet
extends AbstractRobotServlet {

@Override
public void processEvents(
RobotMessageBundle bundle) {
// TODO
}

}

As said, a robot is basically a wave participant, which can react to certain events happening in the wave conversation. The event handling is done in the processEvents method. All information about the event and the context in which the event ocured, like the wavelet, the blip, the participants and so on, is contained in the RobotMessageBundle parameter. The robot servlet has to be mapped to the URL path /_wave/robot/jsonrpc in the web.xml.

The first thing we'd like to do is to show a simple message in the wave, when the robot is added as a participant. The wave API provides a convenience method called wasSelfAdded to check for this particular event:

public void processEvents(RobotMessageBundle bundle) {

if (bundle.wasSelfAdded()) {
final Blip blip = bundle.getWavelet().appendBlip();
final TextView textView = blip.getDocument();
textView.append("GroovyBot added!");
}

}

So, if the robot was just added to the wave, we get the wavelet in which the event occured, append a blip to it and add the message to the blip's document. The result looks like this:


This was easy, and to make the robot run a Groovy script and display the result back in the wave is not much harder.

Executing Groovy Scripts
GroovyBot should not execute all messages submitted to the wave as a Groovy script. Instead, if the user wants the content of a blip to be executed as a script, the blip has to start with the prefix '!groovy' followed by the code, e.g.


The event we want to react to is a BLIP_SUBMITTED event. We have to tell wave that our robot is interested in events of this type by specifying this in the file war/_wave/capabilities.xml:

<?xml version="1.0" encoding="utf-8"?>
<w:robot xmlns:w="http://wave.google.com/extensions/robots/1.0">
<w:capabilities>
<w:capability name="BLIP_SUBMITTED" content="true" />
</w:capabilities>
<w:version>0.1</w:version>
</w:robot>

Then we modify the processEvents method to handle these events:

public void processEvents(
final RobotMessageBundle bundle) {

for (final Event e : bundle.getEvents()) {

if (e.getType() == EventType.BLIP_SUBMITTED) {

// Get input
final Blip inputBlip = e.getBlip();
final TextView inputDocument =
inputBlip.getDocument();
final String inputText = inputDocument.getText();

if (inputText.startsWith("!groovy")) {
final String script = inputText
.substring("!groovy".length());

// Execute script
final GroovyShell shell = new GroovyShell();
final StringBuilder result =
new StringBuilder("Result: ");
try {
result.append(shell
.evaluate(script));
} catch (final Exception ex) {
result.append(e.toString());
}

// Create response
final Blip blip = bundle.getWavelet()
.appendBlip();
blip.getDocument().append(result.toString());
}
}
}

This code first checks if there is an event of type BLIP_SUBMITTED. If so, it gets the blips contents and checks for the prefix "!groovy". Then it removes the prefix, executes the remaining content as a Groovy script with GroovyShell and appends a blip with the result. The output will look like this:



Conclusion
That's basically all, to build a simple Groovy robot. The code above shows a simplified version of GroovyBot which is currently deployed, though. It could be improved in many ways, for example System.out could be captured and also shown in the output. Also, the result could be displayed in another way for a better user experience, for example as a child blip, an inline blip or even as a gadget. But I ran into some issues trying this, for example inline blips were not displayed, or child blips could not be removed and re-added. There are some known bugs in the client API, but all in all, I had a really good experience with wave and its client API. There are many things left, I'd like to do, for example syntax highlighting, using a gadget or making some predefined scripts available, e.g. some DSLs. If you've got some ideas, for example what would be a better user experience, you are welcome. You can start trying GroovyBot right now. It's address is groovybot@appspot.com. Why don't you start with this example, taken from the Practically Groovy series:

!groovy
def zip = "80020"
def addr = "http://weather.yahooapis.com/forecastrss?p=${zip}"
def rss = new XmlSlurper().parse(addr)
def results = rss.channel.item.title
results << "\n" + rss.channel.item.condition.@text
results << "\nTemp: " + rss.channel.item.condition.@temp
println results



Resources
[1] Google Wave
[2] GroovyBot at GitHub (Source Code)
[3] Google Wave Jave Robots Tutorial
[4] Google App Engine




Tuesday, August 18, 2009

Scala Tail Recursion

If you are into Scala, then you eventually may have heard about tail recursion or especially tail call optimization. This post sheds some light on what tail recursion is and on Scala's ability to optimize tail recursive functions.

Tail recursive functions
In Scala it is common to write in a functional style, and recursion is a technique frequently used in functional programming. A recursive function is a function that calls itself (directly or indirectly). Let's take for example the following factorial function:

def rfak(n: Int): Int = {
if (n == 1) n
else n * rfak(n-1)
}

This function calls itself recursively until its parameter n is down to one. The recursive calculation runs like this: rfak(3) = 3*(rfak(2)) = 3*(2*rfak(1)) = 3*(2*(1)) = 6.

Now, the function as shown above is a recursive function, but it is not tail recursive. In a tail recursive function the recursive call is the last action in the execution of the function's body. In the expression n * rfak(n-1), though, first the recursive call is made, and then (as the last action) the result of that call is multiplied by n. So the function is not tail recursive, because the recursive call is not the last action. But we can transform the implementation above into a tail recursive function:

def ifak(n: Int, acc: Int):Int = {
if (n == 1) acc
else ifak(n-1, n*acc)
}

We introduced a second parameter acc. n is like before, but acc (the accumulator) holds intermediate results of the calculation. In this case, in the else part, the recursive call is the last action in the execution of the body, so the function is tail recursive. The calculation process goes like this: ifak(3,1) = ifak(2,3) = ifak(1,6) = 6.

The benefit of tail recursion
So, why would one care if a recursive function is even tail recursive?

If you have a look at how the calculation goes on, again, you will notice, that in the non tail recursive way, the process has to "remember" intermediate results, and that (in consequence) the recursive calls have to be "stacked" until all recursive calls have been made. Then the final result can be calculated: rfak(3) = 3*(rfak(2)) = 3*(2*rfak(1)) = 3*(2*(1)) = 6.

This is not the case with tail recursive functions, as you can see in the process of the second calculation: ifak(3,1) = ifak(2,3) = ifak(1,6) = 6. There are no intermediate results (except for the accumulator argument) and the recursive calls are not stacked (This is why the process of a tail recursive function is also called iterative). In consequence, tail recursive functions are far less expensive regarding memory consumption, at least in theory.

Tail recursive functions in Java
Unfortunately, the JVM does not optimize tail calls itself. For example, if we had implemented the tail recursive factorial function in Java,

public static int ifak(int n, int acc) {
if (n == 1) return acc;
else return ifak(n - 1, acc * n);
}

then for each recursive call a new stack frame is built, which will slow down the calculation and will eventually lead to a StackOverflowError for reasonably large numbers of n. You can also watch this behavior in a debugger, or if you interrupt the calculation, for example by throwing an exception.

public static int ifak(int n, int acc) {
if (n == 1) return acc;
if (n == 10) throw new RuntimeException();
else return ifak(n - 1, acc * n);
}

Then the stacktrace will look like this, showing that the function is called again and again:

Exception in thread "main" java.lang.RuntimeException
at FakTestJava.ifak(FakTestJava.java:32)
at FakTestJava.ifak(FakTestJava.java:33)
at FakTestJava.ifak(FakTestJava.java:33)
at FakTestJava.ifak(FakTestJava.java:33)
at FakTestJava.ifak(FakTestJava.java:33)
at FakTestJava.ifak(FakTestJava.java:33)
at FakTestJava.ifak(FakTestJava.java:33)
at FakTestJava.ifak(FakTestJava.java:33)
at FakTestJava.ifak(FakTestJava.java:33)
at FakTestJava.ifak(FakTestJava.java:33)
at FakTestJava.ifak(FakTestJava.java:33)
at FakTestJava.ifak(FakTestJava.java:27)
at FakTestJava.main(FakTestJava.java:15)

The disassembled byte code instructions, generated by javac, proves this:

public static int ifak(int, int);
Code:
0: iload_0
1: iconst_1
2: if_icmpne 7
5: iload_1
6: ireturn
7: iload_0
8: iconst_1
9: isub
10: iload_1
11: iload_0
12: imul
13: invokestatic #6; //Method ifak:(II)I
16: ireturn

The recursive call is the second last instruction.

Tail call optimization in Scala
Now let's have a look at the byte codes the Scala compiler generated from the code above. Here's the tail recursive code again:

def ifak(n: Int, acc: Int):Int = {
if (n == 1) acc
else ifak(n-1, n*acc)
}

And here are the instructions that comprise the byte codes:

public int ifak(int, int);
Code:
0: iload_1
1: iconst_1
2: if_icmpne 7
5: iload_2
6: ireturn
7: iload_1
8: iconst_1
9: isub
10: iload_1
11: iload_2
12: imul
13: istore_2
14: istore_1
15: goto 0

As you can see, there are no recursive calls anymore. Instead there is a goto as the last instruction, which makes the process simply jump back to the beginning of the function. So, in this case the execution really runs in an iterative process, which is far more efficient than recursively calling the function again, while building new stack frames.

In fact, this is a feature of the Scala compiler called tail call optimization. It optimizes away the recursive call. This feature works only in simple cases as above, though. If the recursion is indirect, for example, Scala cannot optimize tail calls, because of the limited JVM instruction set.


That's essentially it, but if you are interested, let's go and fiddle some more with it. We could do the same as in the Java example above to prove, that there are no additional recursive calls. For example, if you'd throw a runtime exception in the process like this:

def ifak(n: Int, acc: Int):Int = {
if (n == 1) acc
if (n == 2) throw new RuntimeException()
else ifak(n-1, n*acc)
}

and call ifak(10,1) the stacktrace will also tell you that there are no additional stackframes built:

Exception in thread "main" java.lang.RuntimeException
at scalaapplication1.FakTest$.ifak(FakTest.scala:32)
at scalaapplication1.FakTest$.main(FakTest.scala:16)
at scalaapplication1.FakTest.main(FakTest.scala)

But you can also turn off the tail call optimization with a -g:notailcalls flag to the scala compiler. If you run the example again, then the stacktrace will look like this:

java.lang.RuntimeException
at scalaapplication1.FakTest$.ifak(FakTest.scala:32)
at scalaapplication1.FakTest$.ifak(FakTest.scala:33)
at scalaapplication1.FakTest$.ifak(FakTest.scala:33)
at scalaapplication1.FakTest$.ifak(FakTest.scala:33)
at scalaapplication1.FakTest$.ifak(FakTest.scala:33)
at scalaapplication1.FakTest$.ifak(FakTest.scala:33)
at scalaapplication1.FakTest$.ifak(FakTest.scala:33)
at scalaapplication1.FakTest$.ifak(FakTest.scala:33)
at scalaapplication1.FakTest$.ifak(FakTest.scala:33)
at scalaapplication1.FakTest$.main(FakTest.scala:16)
at scalaapplication1.FakTest.main(FakTest.scala)

Finally, let's have a look at how the code looks like when we decompile the class file using a Java decompiler like jd (http://java.decompiler.free.fr/). Here is the code once again:

def ifak(n: Int, acc: Int):Int = {
if (n == 1) acc
else ifak(n-1, n*acc)
}

And here is what jd decompiles from the class file:

public int ifak(int n, int acc) {
while (true) {
if (n == 1) return acc;
acc = n * acc;
n -= 1;
}
}

The tail recursive calls are transformed into an equivalent while loop!

After all, this should not be too surprising.

Resources
[1] Tail calls in the VM



Sunday, July 12, 2009

Wicket, Spring, JDO on Google App Engine - Sample Application

In my previous post on Wicket on Google App Engine I described the basics of how to set up a simple Wicket application that runs on GAE. This post takes the next step and describes how to set up a simple CRUD application. The source code of the sample application is available here.

Overview
The sample application is a simple contacts application, where phone numbers can be associated to persons. It uses Spring for dependency injection and transaction management, JDO for persistence, and Wicket for the presentation layer.

Basic GAE/Wicket Setup
As mentioned, the basic setup of a GAE/Wicket application is the topic this post. So, as described, put the required jars in WEB-INF/lib, set up the appengine-web.xml, set up the Wicket servlet filter in web.xml, set up a Wicket WebApplication class and a home page, set up development mode and turn off the resource modification watcher. After that, you should have a basic application up and running.

Modification Watching
Let's first tackle the one issue left with the basic setup. The resource modification watching is not working, because the Wicket implementation of modification watching uses threads, which is not allowed on app engine. As of version 1.4-RC6, Wicket allows for changing the implementation, so we can write our own that does not use threads and set it in our application's init() method. The sample application uses a custom WebRequestCycle to call the modification watcher on each request.

Note: If you are using an earlier version of Wicket (pre 1.4-RC6), there's a workaround. Simply put the custom modification watcher in the same location on the classpath as the Wicket one. You can do this, by setting up your own org.apache.wicket.util.watch.ModificationWatcher in your project, which replaces the Wicket implementation.

The sample application's code to setup the custom modification watcher looks like this.

public class WicketApplication extends WebApplication {

@Override
protected void init() {
getResourceSettings().setResourceWatcher(
new GaeModificationWatcher());
}

@Override
public RequestCycle newRequestCycle(final Request request,
final Response response) {
return new MyWebRequestCycle(this, (WebRequest) request,
(WebResponse) response);
}


with a custom WebRequestCycle, that calls the modification watcher on each request (in development mode):

class MyWebRequestCycle extends WebRequestCycle {

MyWebRequestCycle(final WebApplication application,
final WebRequest request, final Response response) {
super(application, request, response);
}

@Override
protected void onBeginRequest() {
if (getApplication().getConfigurationType() == Application.DEVELOPMENT) {
final GaeModificationWatcher resourceWatcher = (GaeModificationWatcher) getApplication()
.getResourceSettings().getResourceWatcher(true);
resourceWatcher.checkResources();
}

}
}


Spring / JDO transaction management
First things first. Let's configure Spring for dependency injection and transaction handling. Our goal is to set up a JDO persistence manager factory bean, so we can persist our entities to the datastore. First step is to put the spring jars (and others: cglib, commons-logging, ..) along with the wicket-ioc and wicket-spring jars into WEB-INF/lib and include them on the classpath in our Eclipse project. Then we set up web.xml to configure a ContextLoaderListener and use the Wicket Application.init() method to setup Wicket/Spring annotation-driven dependency injection:

@Override
protected void init() {
addComponentInstantiationListener(new SpringComponentInjector(this));
}


Then we need an application context xml configuration file. This should look like this:

<tx:annotation-driven />

<bean id="persistenceManagerFactory"
class="org.springframework.orm.jdo.LocalPersistenceManagerFactoryBean">
<property name="persistenceManagerFactoryName"
value="transactions-optional" />
</bean>

<bean id="transactionManager"
class="org.springframework.orm.jdo.JdoTransactionManager">
<property name="persistenceManagerFactory" ref="persistenceManagerFactory" />
</bean>


On GAE we cannot use component-scanning or annotation-driven configuration with <configuration:annotation-driven/>, since this introduces a dependency on javax.Naming, which is not on the GAE whitelist. So we have to configure our beans through XML. But we can use annotation-driven transaction configuration.

For persistence I chose JDO instead of JPA. GAE is backed by BigTable as a datastore, and since BigTable isn't a relational database, I thought JDO might be a better fit. But JPA might be a good choice, as well. For JDO we have to configure a LocalPersistenceManagerFactoryBean and a JDO transaction manager. The factory's name must match the name in the jdoconfig.xml file, which is normally created by the Eclipse plugin. The sample application also contains a simple persistence manager factory bean for JPA, that works on GAE.

After that we can create transactional services and DAOs as Spring beans. But first, let's set up a simple, persistable domain model.

Domain Model / Persistence
Our first persistable domain object class is a person pojo entity with some JDO annotations, the persistence meta data (similar to JPA):

@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class Person {

@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Long id;

@Persistent
private String firstName;

@Persistent
private String lastName;

@Persistent
private Date birthday;

public Person() {
super();
}

public Person(final Date birthday,
final String firstName,
final String lastName) {
super();
this.birthday = birthday;
this.firstName = firstName;
this.lastName = lastName;
}

public Long getId() {
return id;
}

// ... Getters and Setters


All persistent entities have to be annotated with @PersistenceCapable. The primary key is annotated with @PrimaryKey. On GAE/BigTable primary keys have special semantics in addition to just uniquely identifying an entity, see the GAE documentation for details. For this example, we'll keep things simple and just choose the primary key to be of type Long (we'll change that later for some reason). All persistent fields have to be annotated with @Persistent.

That's it, for this simple entity for now. Let's go for actually persisting it to the database.

Persisting entities
The sample application uses a PersonDAO to persist person entities. This DAO could inherit from Spring's JdoDaoSupport base class, but we can also do it without it. So, our DAO might look like this:

public class PersonDao /* extends JdoDaoSupport */
implements IPersonDao {

private PersistenceManagerFactory pmf;

@Override
public Person makePersistent(final Person person) {
return getPersistenceManager().makePersistent(
person);
}

private PersistenceManager getPersistenceManager() {
return PersistenceManagerFactoryUtils
.getPersistenceManager(pmf, true);
}

public void setPmf(final PersistenceManagerFactory pmf) {
this.pmf = pmf;
}

}

The PersistenceManagerFactory will be injected by Spring. If you look at the getPersistenceManagerFactory() method you'll notice, that it's not just pmf.getPersistenceManager(), instead it calls PersistenceManagerFactoryUtils to get a persistence manager. This way we get a persistence manager that participates in Spring's transaction handling, which cares for opening and closing the persistence manager and transaction handling on our behalf.

We configure this DAO by defining it as a Spring bean in the application context xml file and inject the PersistenceManagerFactory. We can then use the makePersistent(person) method to persist person instances.

At this point we could now easily implement a simple Wicket form to create new person instances. But I'll leave this task up to you (or have a look at the sample application).

Retrieving Entities
But let's have a closer look at JDO, for those not familiar with it (like me). What we'll probably want to do, for example, is to query a single person by ID or to find all persons, applying some paging parameters. Here are some sample methods from the Person DAO to get an impression of JDO:

public Person get(final Long id) {
final Person person = getPersistenceManager()
.getObjectById(Person.class, id);
return getPersistenceManager().detachCopy(person);
}

@Override
public int countPersons() {
final Query query = getPersistenceManager()
.newQuery(Person.class);
query.setResult("count(id)");
final Integer res = (Integer) query.execute();
return res;
}

@SuppressWarnings("unchecked")
public List<Person> findAllPersons() {
final Query query = getPersistenceManager()
.newQuery(Person.class);
query.setOrdering("lastName asc");
final List<Person> list = (List<Person>) query
.execute();
return Lists.newArrayList(getPersistenceManager()
.detachCopyAll(list));
}

@Override
@SuppressWarnings("unchecked")
public List<Person> findAllPersons(final int first,
final int count) {
final Query query = getPersistenceManager()
.newQuery(Person.class);

query.setOrdering("lastName asc");
query.setRange(first, first + count);

final List<Person> list = (List<Person>) query
.execute();
return Lists.newArrayList(getPersistenceManager()
.detachCopyAll(list));
}

For further details about querying with JDO, see the resources section below. Especially for JDO on GAE, there are quite some restrictions. I think, for example, it's not possible to do something like wildcard matching in queries.

One word on detaching: all the DAO methods above detach the retrieved entities before returning them to the caller by calling detachCopy or detachCopyAll. This way the entities are detached from the persistence manager that fetched them. These instances can then be modified by the web layer and passed back to another persistence manager instance in order to update their database representation. The sample application uses the OpenPersistenceManagerInView pattern instead, configured in web.xml. This way, a persistence manager instance is opened at the beginning of the request and closed afterwards, so the web layer can freely navigate the object graph.

Mapping relationships
Let's add a simple relationship to our domain model. Assume a person has a one-to-many relationship to phone numbers. In GAE such a relationship could be owned or unowned. In an owned relationship the entities on the one side are the parents of the associated entities, which are then called child entities in this relationship. Child entities in an owned relationship cannot exist without their parents. For more information on the different types of relationships and their transaction semantics, see the GAE documentation. The following relationship between persons and phone numbers is an owned, bi-directional one-to-many relationship.

@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class PhoneNumber {

@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key key;

@Persistent
private String type;

@Persistent
private String number;

@Persistent
private Person person;

//...


The phone number has a field person of type Person, annotated with @Persistent. This makes phone number a child entity of person. You'll notice the field key of type Key. This is a possible type for primary keys in the GAE. We cannot use Long in this case, because phone number is a child entity of person and as such its primary key has to contain the key of its parent (see the docs).

Note: For root entities, it's normally okay to use a key of type Long, but this did not work in my example, so I changed the key of Person to Key, too.


public class Person {

@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key key;

@Persistent
private String firstName;

@Persistent
private String lastName;

@Persistent
private Date birthday;

@Persistent(mappedBy = "person")
private List<PhoneNumber> phoneNumbers = Lists.newArrayList();

// ...


The relationship is made bi-directional by adding a list of phone numbers to the person class. The mappedBy parameter denotes the property of PhoneNumber that points to the person. With this, we can now add phone numbers to persons and these will be persisted to the database along with the owning person.

Note: Somehow there's an issue with removing phone numbers from the list by calling person.getPhoneNumbers().remove(phoneNumber), which throws an exception. But removing it by its index works, though. Let me say, that this was not the only time I had problems with the datastore. I have quite a feeling that there are still some open issues in this area.

UserService
To close this post, I'll shortly describe GAE's simple support for authentication. In a GAE application users may login with their Google account.

Checking, if a user is logged in, is easy:

final UserService userService = UserServiceFactory.getUserService();
boolean loggedIn = userService.isUserLoggedIn();


Accessing the current user's details is also easy:

User currentUser = userService.getCurrentUser();
String email = currentUser.getEmail();
String nickname = currentUser.getNickname();


And redirecting a user to a login/logout URL in Wicket can be done e.g. with a ExternalLink:

class LoginLink extends ExternalLink {

private static final long serialVersionUID = 1L;

LoginLink(final String id) {
super(id, getURL());
add(new Label("label", new LoginLabelModel()));
}

public static String getURL() {

final RequestCycle requestCycle = RequestCycle.get();

// Get the URL of this app to redirect to it
final WebRequest request = (WebRequest) requestCycle.getRequest();
final HttpServletRequest req = request.getHttpServletRequest();
final String appUrl = req.getRequestURL().toString();

// Create the login/logout page URL with with redirect tho this app
final String targetUrl = getUserService().isUserLoggedIn() ? loginUrl(appUrl)
: logoutUrl(appUrl);

return targetUrl;

}


Resources
[1] The sample application source code
[2] Apache JDO
[3] JDO Spec
[4] GAE datastore docs



Thursday, July 2, 2009

Java vs. Scala vs. Groovy - Performance Update

It's been quite a while since my post on the micro benchmark comparison between Java, Scala and Groovy. Since the Groovy developers tackled some performance issues with the Groovy 1.6 release, I thought, I'd give it another try. I used Groovy 1.6.3, Scala 2.7.5 and JDK 1.6.0 Update 14. I left the code that I used for the last comparison unchanged. I did not make much use of any specific language features, so it's quite a straightforward quicksort implementation, see the code below.

The Result
Before showing the result, just let me make clear, that I love the Groovy ;-) But seriously, after the anouncement, that the performance was one of the primary goals for the Groovy 1.6 release, I was a bit disappointed. We'll have to keep in mind though, that this is just a (poorly written) micro benchmark, that doesn't mean anything. And after all, it's all about the groovy language features anyway.

Nuff said, here is the result of sorting 100.000 integers:

1. Java: 45ms
2. Scala: 56ms
3. Groovy: 12800ms


And here's the code:

Java

public class QuicksortJava {

public static void swap(int[] a, int i, int j) {
int temp = a[i];
a[i] = a[j];
a[j] = temp;
}

public static void quicksort(int[] a, int L, int R) {
int m = a[(L + R) / 2];
int i = L;
int j = R;
while (i <= j) {
while (a[i] < m)
i++;
while (a[j] > m)
j--;
if (i <= j) {
swap(a, i, j);
i++;
j--;
}
}
if (L < j)
quicksort(a, L, j);
if (R > i)
quicksort(a, i, R);
}

public static void quicksort(int[] a) {
quicksort(a, 0, a.length - 1);
}

public static void main(String[] args) {

// Sample data
int[] a = new int[100000];
for (int i = 0; i < a.length; i++) {
a[i] = i * 3 / 2 + 1;
if (i % 3 == 0)
a[i] = -a[i];
}

long t1 = System.currentTimeMillis();
quicksort(a);
long t2 = System.currentTimeMillis();
System.out.println(t2 - t1);

}

}



Scala

package quicksort

object QuicksortScala {

def quicksort(xs: Array[Int]) {

def swap(i: Int, j: Int) {
val t = xs(i); xs(i) = xs(j); xs(j) = t
}

def sort1(l: Int, r: Int) {
val pivot = xs((l + r) / 2)
var i = l;
var j = r
while (i <= j) {
while (xs(i) < pivot) i += 1
while (xs(j) > pivot) j -= 1
if (i <= j) {
swap(i, j)
i += 1
j -= 1
}
}
if (l < j) sort1(l, j)
if (r > i) sort1(i, r)
}
sort1(0, xs.length - 1)
}

def main(args : Array[String]) {
var a : Array[Int] = new Array[Int](100000)
var i : Int = 0
for (e <- a) {
a(i) = i*3/2+1;
if (i%3==0) a(i) = -a(i);
i = i+1
}
val t1 = System.currentTimeMillis();
quicksort (a)
val t2 = System.currentTimeMillis();
println(t2-t1)
}
}


Groovy

def swap(a, i, j) {
temp = a[i]
a[i] = a[j]
a[j] = temp
}

def quicksort(a, L, R) {
m = a[((L+R)/2).intValue()]
i=L
j=R
while (i<=j) {
while (a[i]<m) i++
while (a[j]>m) j--
if (i<=j) {
swap(a, i, j)
i++
j--
}
}
if (L<j) quicksort(a,L,j)
if (R>i) quicksort(a,i,R)
}

def quicksort(a) {
quicksort(a, 0, a.length-1)
}

// Sample data
a = new int[100000]
a.eachWithIndex {x, i ->
a[i] = i*3/2+1
if (i%3==0) a[i] = -a[i]
}

t1 = System.currentTimeMillis()
quicksort(a)
t2 = System.currentTimeMillis()
println t2-t1


Note: There was a bug in Scala code of the first version of this post, which is now fixed. Scala now goes slightly better.