Wicket, Spring, JDO on Google App Engine - Sample Application

In my previous post on Wicket on Google App Engine I described the basics of how to set up a simple Wicket application that runs on GAE. This post takes the next step and describes how to set up a simple CRUD application. The source code of the sample application is available here.

Overview
The sample application is a simple contacts application, where phone numbers can be associated to persons. It uses Spring for dependency injection and transaction management, JDO for persistence, and Wicket for the presentation layer.

Basic GAE/Wicket Setup
As mentioned, the basic setup of a GAE/Wicket application is the topic this post. So, as described, put the required jars in WEB-INF/lib, set up the appengine-web.xml, set up the Wicket servlet filter in web.xml, set up a Wicket WebApplication class and a home page, set up development mode and turn off the resource modification watcher. After that, you should have a basic application up and running.

Modification Watching
Let's first tackle the one issue left with the basic setup. The resource modification watching is not working, because the Wicket implementation of modification watching uses threads, which is not allowed on app engine. As of version 1.4-RC6, Wicket allows for changing the implementation, so we can write our own that does not use threads and set it in our application's init() method. The sample application uses a custom WebRequestCycle to call the modification watcher on each request.

Note: If you are using an earlier version of Wicket (pre 1.4-RC6), there's a workaround. Simply put the custom modification watcher in the same location on the classpath as the Wicket one. You can do this, by setting up your own org.apache.wicket.util.watch.ModificationWatcher in your project, which replaces the Wicket implementation.

The sample application's code to setup the custom modification watcher looks like this.

public class WicketApplication extends WebApplication {

    @Override
    protected void init() {
        getResourceSettings().setResourceWatcher(
            new GaeModificationWatcher());
    }

    @Override
    public RequestCycle newRequestCycle(final Request request,
            final Response response) {
        return new MyWebRequestCycle(this, (WebRequest) request,
                (WebResponse) response);
    }
    

with a custom WebRequestCycle, that calls the modification watcher on each request (in development mode):

class MyWebRequestCycle extends WebRequestCycle {

    MyWebRequestCycle(final WebApplication application,
            final WebRequest request, final Response response) {
        super(application, request, response);
    }

    @Override
    protected void onBeginRequest() {
        if (getApplication().getConfigurationType() == Application.DEVELOPMENT) {
            final GaeModificationWatcher resourceWatcher = (GaeModificationWatcher) getApplication()
                    .getResourceSettings().getResourceWatcher(true);
            resourceWatcher.checkResources();
        }

    }
}
    

Spring / JDO transaction management
First things first. Let's configure Spring for dependency injection and transaction handling. Our goal is to set up a JDO persistence manager factory bean, so we can persist our entities to the datastore. First step is to put the spring jars (and others: cglib, commons-logging, ..) along with the wicket-ioc and wicket-spring jars into WEB-INF/lib and include them on the classpath in our Eclipse project. Then we set up web.xml to configure a ContextLoaderListener and use the Wicket Application.init() method to setup Wicket/Spring annotation-driven dependency injection:

@Override
protected void init() {
    addComponentInstantiationListener(new SpringComponentInjector(this));
}
    

Then we need an application context xml configuration file. This should look like this:

<tx:annotation-driven />
 
<bean id="persistenceManagerFactory"
 class="org.springframework.orm.jdo.LocalPersistenceManagerFactoryBean">
 <property name="persistenceManagerFactoryName"
  value="transactions-optional" />
</bean>

<bean id="transactionManager"
 class="org.springframework.orm.jdo.JdoTransactionManager">
 <property name="persistenceManagerFactory" ref="persistenceManagerFactory" />
</bean>
    

On GAE we cannot use component-scanning or annotation-driven configuration with <configuration:annotation-driven/>, since this introduces a dependency on javax.Naming, which is not on the GAE whitelist. So we have to configure our beans through XML. But we can use annotation-driven transaction configuration.

For persistence I chose JDO instead of JPA. GAE is backed by BigTable as a datastore, and since BigTable isn't a relational database, I thought JDO might be a better fit. But JPA might be a good choice, as well. For JDO we have to configure a LocalPersistenceManagerFactoryBean and a JDO transaction manager. The factory's name must match the name in the jdoconfig.xml file, which is normally created by the Eclipse plugin. The sample application also contains a simple persistence manager factory bean for JPA, that works on GAE.

After that we can create transactional services and DAOs as Spring beans. But first, let's set up a simple, persistable domain model.

Domain Model / Persistence
Our first persistable domain object class is a person pojo entity with some JDO annotations, the persistence meta data (similar to JPA):

@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class Person {

    @PrimaryKey
    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
    private Long id;

    @Persistent
    private String firstName;

    @Persistent
    private String lastName;

    @Persistent
    private Date birthday;

    public Person() {
        super();
    }

    public Person(final Date birthday,
            final String firstName,
            final String lastName) {
        super();
        this.birthday = birthday;
        this.firstName = firstName;
        this.lastName = lastName;
    }

    public Long getId() {
        return id;
    }
 
    // ... Getters and Setters

All persistent entities have to be annotated with @PersistenceCapable. The primary key is annotated with @PrimaryKey. On GAE/BigTable primary keys have special semantics in addition to just uniquely identifying an entity, see the GAE documentation for details. For this example, we'll keep things simple and just choose the primary key to be of type Long (we'll change that later for some reason). All persistent fields have to be annotated with @Persistent.

That's it, for this simple entity for now. Let's go for actually persisting it to the database.

Persisting entities
The sample application uses a PersonDAO to persist person entities. This DAO could inherit from Spring's JdoDaoSupport base class, but we can also do it without it. So, our DAO might look like this:

public class PersonDao /* extends JdoDaoSupport */
implements IPersonDao {

    private PersistenceManagerFactory pmf;

    @Override
    public Person makePersistent(final Person person) {
        return getPersistenceManager().makePersistent(
                person);
    }

    private PersistenceManager getPersistenceManager() {
        return PersistenceManagerFactoryUtils
                .getPersistenceManager(pmf, true);
    }

    public void setPmf(final PersistenceManagerFactory pmf) {
        this.pmf = pmf;
    }

}

The PersistenceManagerFactory will be injected by Spring. If you look at the getPersistenceManagerFactory() method you'll notice, that it's not just pmf.getPersistenceManager(), instead it calls PersistenceManagerFactoryUtils to get a persistence manager. This way we get a persistence manager that participates in Spring's transaction handling, which cares for opening and closing the persistence manager and transaction handling on our behalf.

We configure this DAO by defining it as a Spring bean in the application context xml file and inject the PersistenceManagerFactory. We can then use the makePersistent(person) method to persist person instances.

At this point we could now easily implement a simple Wicket form to create new person instances. But I'll leave this task up to you (or have a look at the sample application).

Retrieving Entities
But let's have a closer look at JDO, for those not familiar with it (like me). What we'll probably want to do, for example, is to query a single person by ID or to find all persons, applying some paging parameters. Here are some sample methods from the Person DAO to get an impression of JDO:

public Person get(final Long id) {
    final Person person = getPersistenceManager()
        .getObjectById(Person.class, id);
    return getPersistenceManager().detachCopy(person);
}

@Override
public int countPersons() {
    final Query query = getPersistenceManager()
            .newQuery(Person.class);
    query.setResult("count(id)");
    final Integer res = (Integer) query.execute();
    return res;
}

@SuppressWarnings("unchecked")
public List<Person> findAllPersons() {
    final Query query = getPersistenceManager()
        .newQuery(Person.class);
    query.setOrdering("lastName asc");
    final List<Person> list = (List<Person>) query
        .execute();
    return Lists.newArrayList(getPersistenceManager()
        .detachCopyAll(list));
}

@Override
@SuppressWarnings("unchecked")
public List<Person> findAllPersons(final int first,
        final int count) {
    final Query query = getPersistenceManager()
            .newQuery(Person.class);

    query.setOrdering("lastName asc");
    query.setRange(first, first + count);

    final List<Person> list = (List<Person>) query
            .execute();
    return Lists.newArrayList(getPersistenceManager()
            .detachCopyAll(list));
}

For further details about querying with JDO, see the resources section below. Especially for JDO on GAE, there are quite some restrictions. I think, for example, it's not possible to do something like wildcard matching in queries.

One word on detaching: all the DAO methods above detach the retrieved entities before returning them to the caller by calling detachCopy or detachCopyAll. This way the entities are detached from the persistence manager that fetched them. These instances can then be modified by the web layer and passed back to another persistence manager instance in order to update their database representation. The sample application uses the OpenPersistenceManagerInView pattern instead, configured in web.xml. This way, a persistence manager instance is opened at the beginning of the request and closed afterwards, so the web layer can freely navigate the object graph.

Mapping relationships
Let's add a simple relationship to our domain model. Assume a person has a one-to-many relationship to phone numbers. In GAE such a relationship could be owned or unowned. In an owned relationship the entities on the one side are the parents of the associated entities, which are then called child entities in this relationship. Child entities in an owned relationship cannot exist without their parents. For more information on the different types of relationships and their transaction semantics, see the GAE documentation. The following relationship between persons and phone numbers is an owned, bi-directional one-to-many relationship.

@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class PhoneNumber {

    @PrimaryKey
    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
    private Key key;

    @Persistent
    private String type;

    @Persistent
    private String number;

    @Persistent
    private Person person;
 
    //...

The phone number has a field person of type Person, annotated with @Persistent. This makes phone number a child entity of person. You'll notice the field key of type Key. This is a possible type for primary keys in the GAE. We cannot use Long in this case, because phone number is a child entity of person and as such its primary key has to contain the key of its parent (see the docs).

Note: For root entities, it's normally okay to use a key of type Long, but this did not work in my example, so I changed the key of Person to Key, too.

public class Person {

    @PrimaryKey
    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
    private Key key;

    @Persistent
    private String firstName;

    @Persistent
    private String lastName;

    @Persistent
    private Date birthday;

    @Persistent(mappedBy = "person")
    private List<PhoneNumber> phoneNumbers = Lists.newArrayList();

    // ...

The relationship is made bi-directional by adding a list of phone numbers to the person class. The mappedBy parameter denotes the property of PhoneNumber that points to the person. With this, we can now add phone numbers to persons and these will be persisted to the database along with the owning person.

Note: Somehow there's an issue with removing phone numbers from the list by calling person.getPhoneNumbers().remove(phoneNumber), which throws an exception. But removing it by its index works, though. Let me say, that this was not the only time I had problems with the datastore. I have quite a feeling that there are still some open issues in this area.

UserService
To close this post, I'll shortly describe GAE's simple support for authentication. In a GAE application users may login with their Google account.

Checking, if a user is logged in, is easy:

final UserService userService = UserServiceFactory.getUserService();
boolean loggedIn = userService.isUserLoggedIn();
 

Accessing the current user's details is also easy:

User currentUser = userService.getCurrentUser();
String email = currentUser.getEmail();
String nickname = currentUser.getNickname();
 

And redirecting a user to a login/logout URL in Wicket can be done e.g. with a ExternalLink:

class LoginLink extends ExternalLink {

    private static final long serialVersionUID = 1L;

    LoginLink(final String id) {
        super(id, getURL());
        add(new Label("label", new LoginLabelModel()));
    }

    public static String getURL() {

        final RequestCycle requestCycle = RequestCycle.get();

        // Get the URL of this app to redirect to it
        final WebRequest request = (WebRequest) requestCycle.getRequest();
        final HttpServletRequest req = request.getHttpServletRequest();
        final String appUrl = req.getRequestURL().toString();

        // Create the login/logout page URL with with redirect tho this app
        final String targetUrl = getUserService().isUserLoggedIn() ? loginUrl(appUrl)
                : logoutUrl(appUrl);

        return targetUrl;

    }
 

Resources
[1] The sample application source code
[2] Apache JDO
[3] JDO Spec
[4] GAE datastore docs

Java vs. Scala vs. Groovy - Performance Update

It's been quite a while since my post on the micro benchmark comparison between Java, Scala and Groovy. Since the Groovy developers tackled some performance issues with the Groovy 1.6 release, I thought, I'd give it another try. I used Groovy 1.6.3, Scala 2.7.5 and JDK 1.6.0 Update 14. I left the code that I used for the last comparison unchanged. I did not make much use of any specific language features, so it's quite a straightforward quicksort implementation, see the code below.

The Result
Before showing the result, just let me make clear, that I love the Groovy ;-) But seriously, after the anouncement, that the performance was one of the primary goals for the Groovy 1.6 release, I was a bit disappointed. We'll have to keep in mind though, that this is just a (poorly written) micro benchmark, that doesn't mean anything. And after all, it's all about the groovy language features anyway.

Nuff said, here is the result of sorting 100.000 integers:

1. Java: 45ms
2. Scala: 56ms
3. Groovy: 12800ms


And here's the code:

Java

public class QuicksortJava {

 public static void swap(int[] a, int i, int j) {
  int temp = a[i];
  a[i] = a[j];
  a[j] = temp;
 }

 public static void quicksort(int[] a, int L, int R) {
  int m = a[(L + R) / 2];
  int i = L;
  int j = R;
  while (i <= j) {
   while (a[i] < m)
    i++;
   while (a[j] > m)
    j--;
   if (i <= j) {
    swap(a, i, j);
    i++;
    j--;
   }
  }
  if (L < j)
   quicksort(a, L, j);
  if (R > i)
   quicksort(a, i, R);
 }

 public static void quicksort(int[] a) {
  quicksort(a, 0, a.length - 1);
 }

 public static void main(String[] args) {

  // Sample data
  int[] a = new int[100000];
  for (int i = 0; i < a.length; i++) {
   a[i] = i * 3 / 2 + 1;
   if (i % 3 == 0)
    a[i] = -a[i];
  }

  long t1 = System.currentTimeMillis();
  quicksort(a);
  long t2 = System.currentTimeMillis();
  System.out.println(t2 - t1);

 }

}

Scala

package quicksort

object QuicksortScala {

  def quicksort(xs: Array[Int]) {
    
    def swap(i: Int, j: Int) {
      val t = xs(i); xs(i) = xs(j); xs(j) = t
    }
    
    def sort1(l: Int, r: Int) {
      val pivot = xs((l + r) / 2)
      var i = l; 
      var j = r
      while (i <= j) {
        while (xs(i) < pivot) i += 1
        while (xs(j) > pivot) j -= 1
        if (i <= j) {
          swap(i, j)
          i += 1
          j -= 1
        }
      }
      if (l < j) sort1(l, j)
      if (r > i) sort1(i, r)
    }
    sort1(0, xs.length - 1)
  }
  
  def main(args : Array[String]) {
    var a : Array[Int] = new Array[Int](100000)
    var i : Int = 0
    for (e <- a) {
      a(i) = i*3/2+1;
   if (i%3==0) a(i) = -a(i);
      i = i+1
    }
    val t1 = System.currentTimeMillis();
    quicksort (a)
    val t2 = System.currentTimeMillis();
    println(t2-t1)
  }
}

Groovy

def swap(a, i, j) {
 temp = a[i]
 a[i] = a[j]
 a[j] = temp
}

def quicksort(a, L, R) {
 m = a[((L+R)/2).intValue()]
 i=L
 j=R
 while (i<=j) {
  while (a[i]<m) i++
  while (a[j]>m) j--
  if (i<=j) {
   swap(a, i, j)
   i++
   j--
  }
 }
 if (L<j) quicksort(a,L,j)
 if (R>i) quicksort(a,i,R)
}

def quicksort(a) {
 quicksort(a, 0, a.length-1)
}

// Sample data
a = new int[100000]
a.eachWithIndex {x, i ->
 a[i] = i*3/2+1
 if (i%3==0) a[i] = -a[i]
}

t1 = System.currentTimeMillis()
quicksort(a)
t2 = System.currentTimeMillis()
println t2-t1 

Note: There was a bug in Scala code of the first version of this post, which is now fixed. Scala now goes slightly better.