Hasta-la-vista Flatpress; Hola WordPress!

We have changed our software stack from flatpress to wordpress.

Sad to say, flatpress just wasn't quite managing.

The conversion between the two systems wasn't automatic and involved a judicious mix of PHP, Groovy, sed and bash scripting. Sadly it isn't 100% seamless…as/if you browse the articles on the site you may find a number of "rough edges"…notably, images are shown at their 'natural' size, rather than being scaled for beauty. My apologies to anyone who is seriously inconvenienced by this but there comes a point of diminishing returns…

Flatpress hosted 228 posts, so it did quite well. Here's to WordPress hosting many, many more.

Tags: Tools

Stop SQL Developer From Being So Annoying

To prevent the otherwise excellent SQL Developer from incessantly nagging about file associations under Windows, add the following to …\sqldeveloper\bin\sqldeveloper.conf:

AddVMOption -Dno.shell.integration=true

This works around the bug nicely.

The OTN thread is here.

BTW: in case you didn't know, Oracle's codename for SQL Developer is 'raptor'…"Because Raptors eat Toads."

Tags: Tools

Configuring Reference Data In Grails (Redux)

A while back, in Configuring Reference Data In Grails, I talked about declaring elements in resources.groovy that could then be used to populate drop-down boxes in GSPs (to give one example).

Today, I found out an alternative way of declaring things:

beans = {
  xmlns util: "http://www.springframework.org/schema/util"

  util.list(id: 'myList') {
    value 'String 0'
    (1..5).each { value "String $it" }
    value 'String n'
  }

  util.map(id: 'myMap') {
    entry key: 'k0', value: 'v0'
    (1..5).each { entry key: "k$it", value: "v$it" }    
    entry key: 'kn', value: 'vn'
  }
}

I was long used to doing this in 'plain' Spring but wasn't sure that I could actually use the 'util' namespace stuff in Grails.

Thanks to SpringSource's Jeff Brown for pointing out on grails-user how to do it.

Tags: Grails, Programming

Working Through Associations With GORM

I've recently been looking at an 'interesting' project.

The original developer (whom I'm guessing was a PHP-ite in a former life) took Grails for a spin and on the whole did a pretty fair job of getting his application up and running and keeping the customer happy and so deserves a fair amount of Kudos, not to mention Quatloos.

There was some 'grunge': a bit too much being done in GSP pages that should have been in controller code and not enough factoring into services, etc., but that's OK.

The application's domain handling was the one area that gave me pause to think, however.

There was a common pattern to it, which I'll try and explain…

The developer had used Grails' standard GORM domain classes to model the application's domain and (thus) initialize the database but throughout the various controllers I kept seeing code like this:

def id = ...
def domainInstance = DomainClass.get(id)
def query = """
some 20-line query 
walking the underlying relationships from DomainClassTable
to DomainClassTable to get at whatever was wanted
"""
def sql = sql.query(query)
  sql.eachRow { r ->
}

The developer was using GORM quite trivially and for anything even slightly more 'demanding' he would "drop down" into plain SQL.

My first action was to groan out loud…
My second action was to slap my bright, shiny forehead
My first thought was "Oh NO!" …
My second thought was "What a stupid developer"…

Then I calmed down a bit and reconsidered. I came to the conclusion that I was being a rather harsh: for those new to Grails and the Java platform it is not obvious just what GORM/Hibernate brings to the party.

Consider:

<?php
$username = "...";
$password = "...";
$hostname = "...";  
$dbh = mysql_connect($hostname, $username, $password) 
  or die("Can't to connect to MySQL");
mysql_select_db("...",$dbh) 
  or die("Could not select database ...");
$result = mysql_query("SELECT * FROM table");
while ($row = mysql_fetch_array($result,MYSQL_ASSOC)) {
  print "ID:".$row{'id'}.". ... ."<br>";
}
mysql_close($dbh);
?>

If this is what you have as your sole background and experience, and you are a busy guy with your nose to the grindstone (as we all tend to be), then a first read of the Grails documention (or the Hibernate or JPA documention for that matter) wouldn't really show you about walking through associations and the joys of an SQL-free lifestyle.

This is sad, but it is also true.
(It's also sad but true that there are plenty of Java guys out there who don't look past plain 1999-era JDBC…and end up with similarly grungy stuff as a result.)

If you trawl through the Grails documentation (http://grails.org/do … .x/guide/single.html) you'll see that there's a fair bit in there about declaring associations, establishing cascading saves, updates and deletes, etc. but very little on actually using these associations.

The Grails documentation (Section 5.2.1.2 "One-to-many") does show a very small example, but blink and you'll miss it!

So here goes my attempt at leading you to a SQL-free nirvana…

As is my wont, I'll talk through a trivial example application. In this case, its a school, with classes, students, teachers, rooms and books.

It has a very simple Object Model:

Note that I talk about an Object Model. As a developer, I am interested in the relationships between Objects. I am not interested in the underlying persistence mechanism's representation of that model (I'm guessing that there are a few join tables in there, but that's just 'noise' that threatens to drown out what I want to think about). The Entity-Relationship diagram doesn't hold much interest for me.

Given my interest in all things Object, here are a couple of the more interesting domain classes:

package school

class Student {
  String name
  List books
  static hasMany = [ books: school.Book ]
}
package school

class Book {
  String title
}
package school

class School {

  String name

  List classes
  static hasMany = [ classes: Class ]
}

Note that I have chosen to use Lists (rather than Sets, which is the default) to model the associations between my domain classes. This is often A Good Thing because it allows me to explicitly index into these associations, as will soon become apparent.

Here's the first example of walking through various Object association:

School.findByName("Grails High").classes[0].students[0].books[0].title

I'll bet that you can easily determine what that line does.

Easily.

Quickly.

Think about that.

Especially if you are a SQL-head.

If you have persisted with plain SQL (pun intended), you'll know that there's a fair bit of walking through join tables, projecting and so on happening here.

I can guarantee you that a solution using the appropriate SQL wouldn't be so easily comprehensible (so "close to the problem definition") as what is shown above.

<RANT>
I also feel confident in pointing out that the solution you are now sketching out in your head is probably non-portable between databases and most likely not portable between different schema representations of the same basic Object Model.

Ah..what the hell, I'll say it…what you were thinking was just plain wrong!

It probably/possibly would have correctly done data retrieval under some conditions, but I'll bet that you weren't thinking about the same schema, vendor-specific SQL dialect, etc. as I was.

Hence my statement.

In contrast, there's not too much of this ambiguity in the GORM version…

This is probably a whole other argument for another time, however.
</RANT>

Here's a neat second example:

The GSP that hits the database and navigates the inter-Object associations is clear, concise and "all business." It's outcome-focused, not heavily weighted to the mechanism of data retrieval:

<%@ page import="school.*" %>
<html>
<head>
  <title>Welcome to Bob's GORMy School</title>
  <style type="text/css" media="screen">
    [...elided...]
  </style>
</head>
<body>
<h1>Welcome to Bob's GORMy School</h1>
<%
  def school = School.get(1)
  def mkp = new groovy.xml.MarkupBuilder(out)
  mkp.div([class: "homePagePanel"]) {
    p school.name
    school.classes.each {c ->
      h2 ([style:'background:lightGray; font-weight: bold;'], "${c.code}: ${c.subject}")
      p "Teacher: ${c.teacher.name}"
      p "Room: ${c.room.designation}"
      p "Students (${c.students.size()}):"
      ul {
        c.students.each {s ->
          li s.name
          p "Books (${s.books.size()}):"
          ul {
            s.books.each { b ->
              li b.title
            }
          }
        }
      }
    }
  }
%>
</body>
</html>

The code and the solution are pretty clearly matched, so I'm guessing that once again you'll be able to work out what is going on…even if you don't know GORM or GSPs.

Pay particular attention to the ease with which the code can walk through associations between objects and also deal with the various cardinalities of the association.

(While you're at it, give praise to MarkupBuilder!)

Still not convinced, eh?

Oh well…take a week off and work on the equivalent SQL. Then take a further week off to recreate the simple page I show above using the resultant data in all it's stream-y, batch-oriented weirdness. I'll wait…

Let's do some comparison shopping.

The following code snippets are equivalent ways of undertaking the business operation "for each student, find all the titles of all their books":

println "GORM:"
Student.findAll().each { s ->
  s.books.each { b ->
    println "${s.name}: ${b.title}"
  }
}
println "SQL:"
def sql = new Sql(dataSource)
def qry = """
select s.name, b.title
from Student s
  inner join Student_Book sb on s.id = sb.student_books_id
  inner join Book b on sb.book_id = b.id
"""
sql.eachRow(qry) {
  println "${it.name}: ${it.title}"
}

Which more clearly matches the expected business operation?

Life is all about change, so try changing things ever so slightly:

println "GORM:"
Student.findAll().each { s ->
  println s.name
  s.books.each { b ->
    println b.title
  }
}

Trivial change, trivially easy with GORM.

I leave the equivalent change to the SQL morass as an exercise for the reader…

The drawback here is that your PHB may be tempted to change things around too often. There's probably a cure for that, however.

Go on…say it…be brave…it's a big issue.

Take a look here first.

Then I'll quote from Gavin King, the creator of Hibernate:

Note that I have not yet met a serious performance problem in Hibernate 2.x that I could not solve quite quickly.

(Note that we're a long way past the 2.x series now.)

You could argue that the developer of the application I am talking about was concerned with optimizing the application's performance. A laudable goal which we'll see about in a second…for now, just let me say that this application's whole database is only about 250Mb. My mobile phone would give good performance serving this database, regardless of how I sub-optimally I were to structure the queries! I exaggerate (slightly) but its clear that there's a quick "bang for buck" calculation to be made here, one that strongly argues against 'optimal' SQL.

Naive use of Hibernate (or any such tool) can lead to the n + 1 problem where a collection requires multiple queries to retrieve.

GORM tries to assist with this where it can, and makes it possible to tune the behaviour of an application for various situations. One of the most powerful tools for this is the ability to configure lazy/eager/batch fetching.

For example, if I look at the actual usage pattern of the association between Class and Student, I may conclude that it is appropriate to always eagerly fetch Student instances at the same time as a Class instance is fetched from the database. I might provide a mapping for Class as follows:

package school

class Class {
  String code
  String subject
  Teacher teacher
  Room room
  List students
  static hasMany = [ students: Student ]
  static mapping = {
    students fetch:"join"
  }
}

I may conclude that in some usage situations, the relationship between Student and Book is an eager one: all accesses to a Student instance involves looking at the associated list of Book instances.

println "GORM(eager):"
Student.findAll([fetch:[books: "eager"]]).each { s ->
  println s.name
  s.books.each { b ->
    println b.title
  }
}

You can peek around (p6spy is your friend) and see how the database is driven in this situation:

select
  this_.id as id7_1_,
  this_.version as version7_1_,
  this_.name as name7_1_,
  books2_.student_books_id as student1_3_,
  book3_.id as book2_3_,
  books2_.books_idx as books3_3_,
  book3_.id as id0_0_,
  book3_.version as version0_0_,
  book3_.title as title0_0_ 
from
  student this_ 
    left outer join
        student_book books2_ 
            on this_.id=books2_.student_books_id 
    left outer join
        book book3_ 
            on books2_.book_id=book3_.id

No n + 1 problem here…

In other usage situations, a lazy approach may be appropriate:

println "GORM(lazy):"
Student.findAll([fetch:[books: "lazy"]]).each { s ->
  println s.name
}

This gives pretty simple and effective SQL:

select
  this_.id as id7_0_,
  this_.version as version7_0_,
  this_.name as name7_0_ 
from
  student this_

Even in those situations where "only SQL will do", GORM allows the use of HQL to "get closer to the metal" while still avoiding a lot of drudgery:

Book.executeQuery("select b from Book as b, Student as s where (s.id=:st) and (b in elements(s.books))",
  [st: 2L]).each { println it.title }

Of course, you could do it the hard way:

select
  book0_.id as id2_,
  book0_.version as version2_,
  book0_.title as title2_ 
from
  book book0_,
  student student1_ 
where
  student1_.id=? 
  and (
    book0_.id in (
      select
        books2_.book_id 
      from
        student_book books2_ 
       where
        student1_.id=books2_.student_books_id
    )
  )

How obscure is that! There's nothing in that mess that tells me what business needs I am trying to meet. It's the database equivalent of "assembly language" and-just like all assembly languages-is best left to tools that know what they are doing.

It's good to have choice.

Really.

With GORM, I have a situation where I can start off simply and get progressively more sophisticated as the proven demands of my application require. I can build an Object Model for a domain without worrying about mechanics of what magic incantations I will need to make it work. I have a "tuning point" in my application that I can work with. This is All Good Stuff.

Object graph navigation is extremely powerful, so go out and practice. You may have to play with it a bit before it gets under your skin but you'll be far more productive as a result.

I've seen a system convert from GORM-style to plain SQL data access (driven by a manager's "SQL experience") "in the name of performance" only to find the resulting true performance drop through the floor. GORM/Hibernate does a lot of smart things that an average programmer just won't ever "get around to."

I'm really only touching the surface of things here (I haven't even talked about GORM's wonderful second-level cache, but I hope that all this helps.

Here's a nice starter resource for Hibernate, the technology underlying GORM: Hibernate: A Developer's Notebook. Its a good read, even if its not GORM-specific.

Tags: Grails, Programming

Pimped-Out Grails BootStraps

A while back, groovyblogs.org pointed me to a site suggesting that I should Pimp my Grails Bootstrap.

"Nice," I thought, "but it should be possible to clean things up a bit…make it a bit more 'groovy.'"

So here is my attempt:

import grails.util.Environment

class BootStrap {

  def init = {servletContext ->
    if (this.respondsTo(Environment.current.name))
      this.invokeMethod(Environment.current.name, servletContext)
  }

  def destroy = {
  }

  def production(servletContext) {
  }

  def test(servletContext) {
  }

  def development(servletContext) {
  }
}

A couple of things to note:

  • Note the use of the Grails 1.1+ Environment class, rather than the older (now deprecated GrailsUtil stuff)
  • The methods must be true methods, not closure definitions (otherwise respondsTo() doesn't know about them, it seems)
  • It's all optional: if you only need development(say), you only have to define development()
  • the method names should correspond to what would be returned by Environment.current.name: (for the predefined environments, these are given by: Environment.DEVELOPMENT.name, Environment.TEST.name, etc.)
  • In this example, respondsTo()/invokeMethod() need to be invoked via this. Don't know why; shouldn't be needed as far as I can see, so YMMV

There's also http://jira.codehaus … rg/browse/GRAILS-289, which hints at better things to come.

Tags: Grails, Programming

A Small Improvement In Grails' GORM Subsystem

What I learned today…

To quote Grails' Graeme Rocher:

Grails 1.2 has save(failOnError:true)

This should remove a small frustration…

Tags: Grails, Programming

Viewing Grails' In-Memory HSQLDB Database

Another example of "what I learned today."

Thanks to act:ualise | technology for this.

import grails.util.Environment

class BootStrap {

  def init = {servletContext ->

    if (Environment.current == Environment.DEVELOPMENT) {
      //
      // don't forget to use the correct URL as set in DataSource.groovy
      // typically:  jdbc:hsqldb:mem:devDB
      // 
      org.hsqldb.util.DatabaseManager.main()
      ...

A cheap and cheerful way of looking at Grails' in-built, transient in-memory database:

It's no p6spy or autobase, but it's good enough for now.

By the way: note the 'new' (Grails 1.1) way of determining the current execution environment. Much prettier than the old way.

[edit]

This makes it even easier:

org.hsqldb.util.DatabaseManager.main(['-url', 'jdbc:hsqldb:mem:devDB'] as String[])

Tags: Grails, Programming

What I Learned Today: Constraining Relation Cardinality In GORM

I wanted to set a limit for how many classes a Grails Domain class could relate to and I couldn't see how to do it, so I put a call for help out on the Grails user list (user@grails.codehaus.org).

Burt Beckwith ( ) showed me how:

class Author {
  static hasMany = [ books : Book ]
  String name
  static constraints = {
    books size: 0..3
  }
}

class Book {
  String title
}

Burt also pointed me to the doco: http://grails.org/do … onstraints/size.html.

Thanks, Burt!

Tags: Grails, Programming

What I Learned Today: Groovy Default Parameter Goodness

Default parameters can reference other parameters.

This little snippet of Groovy:

def f(x, y = x) {
  println "x=$x, y=$y"
}

f('hello', 'world')
f('hi')

Gives:

x=hello, y=world
x=hi, y=hi

Tags: Groovy, Programming

JBoss Briefing Breakfast

First things first: "Thank you, RedHat." Mum taught me to be polite. See, Mum, see :-)

This was a "Red Hat Briefing Session where you will learn about the newly launched JBoss Open Choice - Red Hat's strategy for simple, standard & flexible Java solutions."

The main message of the first session was: the industry wants simple; tomcat is enough. JBoss is responding to this demand.

Even the presenter ([the slightly tired-looking] Ms. Ivetta Kleinman, JBoss Solutions Architect) didn't seem at all impressed.

I agree with her advice: a cut-down server may seem like a good idea now but a bit of thinking about the future can be beneficial.

Lots of point solutions tend to end up inflicting death by a thousand puncture wounds.

I have been into plenty of sites that seemed to think that they can build a better JMS/Hibernate/SOA stack/SOAP framework/CORBA ORB/… than anyone else (and didn't need no stinkin' sub-optimal solution from someone "who knows nothing about our special needs") and eventually a) find out that that wasn't really the case, b) find that their home-grown, beautifully 'architected' system is now an increasingly expensive liability, c) find their architects have got bored and moved on to greener (and still innocent) fields, and d) can't find anyone willing to kill off their career by working with it.

Evidently, the industry still isn't mature.

The second part of the event was devoted to Ross Hall, Architect at Suncorp Personal Insurance. He gave a strong presentation saying how happy he was with JBoss Rules. I was glad to see someone actively pushing the idea that rules can be extracted/abstracted away from messy tangles of inline code.

He introduced Drools Advisor, a tool that they had developed to make creating form-based, rules-driven user interfaces easier.

This interested me: I had previously been involved in supporting an application whose sole raison-d'etre was to validate operator-input data against a schema [of sorts] to ensure consistency and correctness with respect to a myriad of conditions, exceptional circumstances, resource constraints, and so on. Partly driven by a groaning, bug-ridden and awfully messy database structure, a rewrite had become painfully necessary. I proposed adopting a rules engine, only to be told (by the lead maintainer) that "there are no rules in this system." No matter how hard I tried, I couldn't convince The Powers That Be that a mass of SQL stored procedures, Delphi-based "Business Objects" and UI validation constituted/contained a set of 'rules'.

Wisdom is knowing that "you can take a horse to water, but you can't herd cats." Or something. I'm not wise enough yet, I guess.

Wish I'd have Drools Advisor when I was engaged in battling the "unenlightened dark forces"…having a pretty demo around would have helped heaps.