Blogbody Rotating Header Image

August, 2006:

AutoQ: non-hosted version of HostedQA

Autoriginate is releasing a sister to HostedQA called AutoQ. You can read more about it here and get in line to be the first to try it out! It should be interesting making a product that is both SAAS and enterprise software. Anyone know any other companies doing something similar?

Running IE6 on OS X

Just found that CodeWeavers is running a 60 day trial for their beta software, CrossOver. Setting up IE6 to run on my mac (no virtualization, no dual booting) was completely painless – the CrossOver installer downloads the setup.exe and runs it for you.

So now my Mac can do:

ie6_icon.png

ie6_screen.png

Pretty slick stuff!

Keyboard for sale

I’m selling my Logitech G15 Gaming keyboard – if any of my “loyal readers” wants to buy it, let me know.

Help Wanted: Stealth Project

If anyone is interested in taking on some extra part time work (maybe 5-10 hours a week) for a stealth project in the area of social networking (can’t say more at this point), backed by a very solid team with a proven track record, let me know.

The technologies we’re working with include a mixture of things from Project Able and things from RIFE. Right now we’re only interested in bringing in people who are highly trusted and have some spare time. So if you’re one of my opensource acquaintances and have the time, let me know. Sorry, if I don’t know you it is best that you not ask to get involved right now.

Requirements include:

  • Self-starter, able to work without much direction.
  • Available for part time work (5-10 hours a week, maybe more later).
  • Excited to get involved in an early stage startup.
  • Extremely skilled with Java and Java web applications.
  • Very knowledgeable with SQL and large scale database deployments.
  • Solid understanding of JavaScript, AJAX, and CSS a plus.
  • Experience with RIFE, iBatis, and Spring a plus.

JDK Logging: not as bad as I thought

I have previously complained about JDK logging. Thanks to Niall’s comment in my original entry, however, I solved problem #2 in my original complaint, and making my original solution to problem #1 acceptable now.

So for those who wish to use JDK logging and want a nice wrapper class, here is a wrapper that works well for me:

/**
* Poritions of this code are fall under the following license:
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements.  See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License.  You may obtain a copy of the License at
*
*      http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.hostedqa.utils;

import java.util.logging.Level;
import java.util.logging.LogRecord;
import java.util.logging.Logger;

/**
* @author <a href="mailto:plightbo@gmail.com">Patrick Lightbody</a>
*
* Portions of the class have been borrowed from commons-logging,
* which was created by:
*
* @author <a href="mailto:sanders@apache.org">Scott Sanders</a>
* @author <a href="mailto:bloritsch@apache.org">Berin Loritsch</a>
* @author <a href="mailto:donaldp@apache.org">Peter Donald</a>
*/
public class Log {
private Logger logger;

public Log(Class clazz) {
logger = Logger.getLogger(clazz.getName());
}

public void severe(String msg, Exception e) {
_log(Level.SEVERE, msg, e);
}

public void severe(String msg, String... args) {
_log(Level.SEVERE, msg, null, args);
}

public void severe(String msg, Exception e, String... args) {
_log(Level.SEVERE, msg, e, args);
}

public void warn(String msg, Exception e) {
_log(Level.WARNING, msg, e);
}

public void warn(String msg, String... args) {
_log(Level.WARNING, msg, null, args);
}

public void warn(String msg, Exception e, String... args) {
_log(Level.WARNING, msg, e, args);
}

public void info(String msg, Exception e) {
_log(Level.INFO, msg, e);
}

public void info(String msg, String... args) {
_log(Level.INFO, msg, null, args);
}

public void info(String msg, Exception e, String... args) {
_log(Level.INFO, msg, e, args);
}

public void fine(String msg, Exception e) {
_log(Level.FINE, msg, e);
}

public void fine(String msg, String... args) {
_log(Level.FINE, msg, null, args);
}

public void fine(String msg, Exception e, String... args) {
_log(Level.FINE, msg, e, args);
}

private void _log(Level level, String msg, Exception e, String... args) {
if (logger.isLoggable(level)) {
// Hack (?) to get the stack trace.
Throwable dummyException = new Throwable();
StackTraceElement locations[] = dummyException.getStackTrace();
// Caller will be the third element
String cname = "unknown";
String method = "unknown";
if (locations != null && locations.length > 2) {
StackTraceElement caller = locations[2];
cname = caller.getClassName();
method = caller.getMethodName();
}


LogRecord lr = new LogRecord(level, msg);
lr.setThrown(e);
lr.setParameters(args);
lr.setSourceClassName(cname);
lr.setSourceMethodName(method);

logger.log(lr);
}
}

public Logger getLogger() {
return logger;
}
}

HostedQA is now free for opensource/not-for-profit use

Today we’re pleased to announce that HostedQA is now available free of charge for any opensource or other qualifying not-for-profit organization.

If you’re interested in trying it out, just sign up for a free trial and we’ll convert your account to a permanent one once we’ve verified your project’s status.

Able: Hibernate/JPA support?

I’m eager to try out Hibernate/JPA support with Able. Does anyone have experience using either of these and want to help take a stab at building it out? I’d want to use annotations and conventions as much as possible. Other nice to have features would include:

  • Minimal or zero XML configuration
  • If XML configuration is used, a nice way to reload the Hibernate session/configuration when changes occur (such as we do with iBatis)
  • Extensive use of annotations
  • Ideally I’d like to link the WebWork validation/type conversion rules in with the ORM rules

Let me know if you’re interested in this.

Project Able Mailing List

For those that want to get involved with Project Able, there is a mailing list for all OpenSymphony sandbox projects. Simply sign up for the sandbox@opensymphony.com mailing list by sending an email to sandbox-subscribe@opensymphony.com and you’ll be on the list.

UPDATE: This information is no longer accurate. Please see the new Able project on Google Code for the full details.

Project Able: a complete Java web stack

UPDATE: This information is no longer accurate. Please see the new Able project on Google Code for the full details.

Project Able is a full Java-based web development stack designed to make web development painless. In a sense, it is an attempt to bring together quality opensource tools in one cohesive stack, similar to what Rails has done for Ruby, while also encouraging common practices I’ve used in software engineering for a long time.

It is very similar to projects such as Trails, Grails, and AppFuse. However, there are a few key differences:

  • The stack components are different (WebWork, Spring, iBatis, etc).
  • In addition to the basic framework, Able also encourages common development techniques and patterns (more below).

Overview

Able was started for two reasons:

One: with the success of Ruby on Rails, it was clear there was a demand in the web development for a streamlined stack. In the Java world, there has been a long fascination with building up a hodgepodge of components in to a custom stack. But often, all we really want is to get started right away.

Two: having recently built HostedQA and BigBark in a very short time period, I realized that there was a common pattern for building web apps that I had been following for years. Able is an attempt to ingrain those patterns and techniques in to a single starting point for new webapps.

When looking at a framework like Rails, one quickly starts to see that Rails isn’t just a web framework and therefore similar to WebWork or Struts. In fact, it is much more: it provides the database layer, a build infrastructure, a testing infrastructure, and more. In short: it provides the complete package all under one roof.

Not a Library

Able is not a library. It is not like RIFE or like WebWork or Hibernate. Able is, instead, a starting point for building webapps. The code inside of Able becomes the code in your application. It is meant to be adjusted and mutated as your project goes on.

Of course, Able will evolve too, but for the most part your only need for Able will be when you start your work. Some, however, may wish to look at Able as a way to streamline their existing web development process. But most will checkout Able from SVN the day they start their project, and then never look back.

Part of the reason I’m doing this is because the more and more I’ve been using (and developing) opensource, the more I’ve found it is easier to simply grab a copy of the latest release of Library X and either fork it or provide hooks in to little known APIs that allow me to control the inner guts. At the end of the day, I’ve found that if I had simply just had the entire project’s code checked in to my project, I would have been more efficient.

In many ways, this is like code generation. That isn’t to say that there aren’t libraries that Able depends on (there are), but rather the extensions done to those libraries aren’t delivered as another library, but rather source code you are free to modify and improve over time.

I think this will provide the best and easiest way to get started quickly and still allow for a lot of freedom to change things moving forward.

The Parts of Able

Able can be split up in to two parts:

  • Components – these are the libraries Able depends on (iBatis, Spring, WebWork, etc).
  • Development patterns – these are the techniques used in Able that we encourage other projects to adopt (upgrade framework, build profiles, test fixtures, etc).

Components

Able uses the following components/tools:

Each of these libraries is the leader or considering one of the leaders in the space it occupies. They were all picked by me as personal favorites for various reasons. In the future, if people are curious, I’d be happy to go in to detail why each one was picked.

I imagine that the Spring and iBatis picks may be cause for some concern among certain people, but know that most of these choices were primarily based on three criteria:

  1. Does the framework provide control and insight in to what it is doing?
  2. Does the framework fail fast when it can?
  3. Does the framework allow for hooks in to it to let me customize it in unique ways?

If two particular frameworks pass that test equally, it ultimately came down to personal style. The good news is that with Able, you are free to just modify the source as you see fit. That’s one of the advantages I see with distributing Able as a project rather than a library/framework.

Development Patterns

Able also addresses very common development processes and packages them up in a ready-to-use form. They are:

  • Clean URLs in WebWork (ex: /profile/1/view). The IDs in the URLs are automatically extracted and turned in to parameters using the following convention: /profile/1/view becomes /profile/view?profileId=1.
  • Convention/annotation-based WebWork action configuration (no xwork.xml file at all). Simply create a class in the proper package and your action is configured. JSPs are assumed as the default result, but you can override them with a @Result annotation.
  • Additional support for common web techniques, such as the “Flash scope” (Redirect-After-Post pattern), simple checkbox support, and an easy way to mark WebWork actions as “page partials” that should not be decorated by SiteMesh.
  • A configured QuickStart setup (QuickStart is part of WebWork) that allows for quick deployment of your application during development.
  • A nice DAO wrapper around iBatis that utilizes Java 5 generics quite nicely, as well as provides support for multiple database vendor syntaxes.
  • Support for reloading iBatis configuration files during deployment to ease development (thanks to Steve Grimm for this!).
  • An upgrade framework that tracks your current build and helps take you to most up to date version (can be used for database changes and other changes). For more info, see here and here.
  • A test fixture built on TestNG and optimized for Java 5, utilizing code from Spring that ensures your Spring container is set up properly on every run. This helps integration testing immensely.
  • A concept of build profiles, including a development (default) and production profile. Here you can specify things like an in-memory database, a MockMailService, and anything else you’d like to use to streamline development.
  • IDE project generation specifically tailored for IDEA that automatically configures your IDEA project with Run/Debug profiles for launching the webapp and running integration tests. Your team really can simply “checkout and go”. (Note: IDEA 6.0 will provide the ability to save Run/Debug settings in ipr and iml files, making this feature less useful. But until IDEA 6.0 comes out of beta, it is still a big help.

Examples

Let’s take a look at an example of a few of these key items:

Clean URLs

Typically, Java-based web applications require fairly ugly URLs such as foo.action or foo.do. Not only do these URLs expose their implementation, they are just more painful to work with. Able helps by adopting a more REST-like URL strategy where the URL itself actually encodes some of the data typically found in query parameters.

For example, a typical WebWork URL might look like:

/viewPerson.action?personId=1

Using Able, we instead encourage a URL like this:

/person/1/view

Not only is the URL more concise and descriptive, it actually makes writing HTML easier. For example, using the viewPerson.action style, an update form might look like:

bc..

name=”person.firstName”/>

You now can simply do:

bc..

name=”person.firstName”/>

Similarly, I’ve found this style reduces the need for things like the ww:url tag and other mechanisms to make URL generation easier to deal with. By simply making the URLs themselves easier, there is little need to use those wrappers anymore.

The automatic parameter application only works for numeric values currently. Perhaps in the future we’ll support Strings and other types, though that requires a bit more work to distinguish a String from a legitimate sub-directory.

Directories are handled by the convention-based configuration (discussed next). The actual underlying implementation of this ActionMapper can be found in the AbleActionMapper class.

Convention-based configuration

Another pain I’ve experienced with WebWork is growing configuration files. The fact is that most of what was in those files was based on the same pattern over and over. Able takes those common patterns and makes them a standard convention. The rules for the conventions are:

  1. Actions must be stored in com.opensymphony.able.actions.
  2. Sub-packages in that package are treated as directories.
  3. Actions must be named in the style FooAction, meaning classes that don’t end in “Action” won’t be mapped.
  4. Results are assumed to be found at /path/to/action-resultValue.jsp. For example, if com.opensymphony.able.actions.foo.BarAction returns “success”, then Able will look for /foo/bar-success.jsp.
  5. If that JSP can’t be found, Able will try /foo/bar.jsp. This is done because there is a common pattern of sharing the same JSP for several result values.
  6. If you need to do override the conventions for result mappings, such as to provide a redirect, you can do so with the @Results annotation.

An example annotation for a redirect might be:

@Results(values = {
@Result(name = Action.SUCCESS, type = Redirect.class,
location = "http://www.hostedqa.com")
})
public class FooAction extends AbleActionSupport {
...

The interceptor stack is hardcoded and includes support for the flash scope, partial page rendering, and better checkboxes (see the next topic for more). If you need to change the interceptor stack, you’ll need to edit the class AbleConfiguration.

You can even tweak the file to do a combination of reading in an xwork.xml file and also using convention-based configuration – it’s up to you. However, having built two medium-sized applications using this style so far, I’ve found it is often easier to deal with the 10% exception cases by structure the class hierarchy.

I have had one case, for example, where I have an empty action class that simply extends another class. This is an alternative to action aliasing that actually turns out to be fairly nice to work with. But again, you are free to change AbleConfiguration as you see fit.

Common web development techniques

There are a few common web development techniques that either aren’t supported by WebWork or aren’t in WebWork’s scope. Either way, Able provides the following features:

  • A complete implementation of the “flash scope” or Redirect-After-Post pattern.
  • Better support for boolean checkboxes, specifically making submission of unchecked checkboxes easier.
  • A way to tag your actions as @Partial, which forces SiteMesh to not decorate the rendered view. This makes doing AJAX with SiteMesh very simple.

Flash scope support

Using the annotation-based result mapping (see above), you can replace a Redirect annotation with a Flash annotation. It behaves almost exactly the same, except that the subsequent request that you’ve redirected (flashed) to will have the original action pushed on to the stack. This means you can display content from the previous action (such as a confirmation message) easily.

Example:

@Results(values = {
@Result(name = Action.SUCCESS, type = Flash.class,
location = "/person/${person.id}/view")
})
public class FooAction extends AbleActionSupport {
...

Seamless un-checked checkbox support

Embarrassingly, WebWork has never made supporting un-checked checkboxes easy. The problem lies in the fact that HTML doesn’t submit a form value if the checkbox is not checked. Able fixes this by changing the WebWork checkbox template to actually render out two form items per checkbox: the checkbox itself and a hidden field.

Then an interceptor is applied that compares the hidden field and adds the value to the parameters to apply to your action if checkbox was not found (therefor not checked).

Partial page identification

People who use SiteMesh love it, but I’ve found that often SiteMesh users structure their URLs in a way that makes using SiteMesh simpler, rather than ways that make their URLs make more sense. Common URL patterns I’ve seen include “/secure” or “/member” (to have a root path that the logged in decorator should show) as well as “/partial” (to have a root path that shouldn’t be parsed).

This happens because SiteMesh makes it easy to map URL patterns such as /secure* or /partial* to specific decorators, or to be excluded.

With the increased use of AJAX, SiteMesh users find themselves needing to exclude much more content than they have in the past. The @Partial annotation, when applied to an action, allows for you to easily make some actions render back just partial HTML (to be used by Prototype’s Ajax.Updater, for example) without placing all your partial actions under a common URL root such as /partial*.

iBatis DAO wrapper

Using Java Generics, there is a base DaoService class you can extend for additional service classes for new entities. It encourages a practice of one service per entity (Foo -> FooService, Bar -> BarService, etc). The DaoService provides a Dao<T> object specifically for that entity. You can then do things like:

public User findById(long userId) {
return dao.selectSingle("byId", userId);
}

public User findByEmail(String email) {
return dao.selectSingle("byEmail", email);
}

public User findByFirstNameOrLastName(String firstName,
String lastName) {
return dao.selectSingle("byFirstNameOrLastName",
firstName, String lastName);
}

public List findByUpdatedAfterDate(Date updatedAfter) {
return dao.select("byUpdatedAfter", updatedAfter);
}

The key thing here is that the DAO object uses var-args and Generics. The var-args usage is unique in that the name of the query (ex: “selectByFirstNameOrLastName”) gets parsed for “and” and “or” keywords and then the parameters are placed in to a map with those keys. In the case of findByFirstNameOrLastName(), the iBatis query looks like so, found in User.xml:

bc..

The same applies for delete, update, and insert.

In addition to this, the DAO layer also looks for queries named “foo-vendor”, where vendor is something like “hsql” or “postgres”. This lets you write your application to work on multiple databases. Even websites should do this, simply for the fact that using an in-memory database helps with testing/development but you’d want to use a real database in production.

Upgrade framework

The best way to understand the upgrade framework is to look at my previous blog entry on the subject.

Build profiles

One thing that I really like that Rails encourages, and that I think most developers have done for years, is using the concept of build profiles. Out of the box, when you run “mvn install” you get a different build from when you run “mvn -Pprod install”. Looking at pom.xml, you can see that the “prod” profile simply copies over config.prod.xml to config.xml when building the webapp.

Inside of config.xml (the default/dev profile), a MockMailService is used, as well as JDBC url that indicates to Able to use an in-memory database and a sample database script (sample.sql) to populate initial data. This makes developing much easier, and encourages a “checkout and go” philosophy.

Inside of config.prod.xml, a real MailService is used and a real postgres database is specified. Tests are also skipped, since we assume that by the time you are ready for production you don’t need to run your tests.

Using this pattern, you might want to introduce a “staging” or “qa” profile as well.

Getting Started

Are you interested in getting started today? Great, welcome aboard! Right now Able is nothing more than code checked in to OpenSymphony’s sandbox SVN repository. You can find it at:

https://svn.opensymphony.com/svn/sandbox/able

Once you’ve checked out the code, simply do the following:

  1. Make sure you’ve downloaded maven 2.0.4 and have it in your PATH
  2. Make sure you’ve got the TestNG plugin (TestNG-J) downloaded and installed in IDEA.
  3. Run “mvn install”.
  4. Run “mvn idea:idea” to generate your IDEA project files (“maven eclipse:eclipse” should work too, but I haven’t optimized that yet)
  5. Open able.ipr in IDEA
  6. Run the “Run Webapp” target to launch Able at http://localhost:8080
  7. Run the “Services” target to run all the TestNG Spring service tests
  8. Rename the package com.opensymphony.able to whatever package you’d like to start with (this package is also referenced in a few classes, so you’ll need to change those too – soon I plan to refactor that hardcoded package out to a single configuration file).

That’s it! Let me know what you think and look out for Able graduating from a sandbox project in the not-to-distant future.

JDK logging really does suck

So I’d been trying out JDK logging for the last few months for a couple of projects, and I had been getting annoyed with two problems:

  1. There were no nice convenience methods on Logger, such as warn(String, Throwable).
  2. In getting around the first issue (by creating a wrapper Logger), everything gets reported in my logs are coming from my wrapped logger class!

I had assumed that this was just an error on my part and made a mental note to check in to it soon. But apparently I’m not the only one having this problem. On JavaBlogs, I found this blog entry describing the exact same issues:

Sometimes it’s nice to review the libraries being used and re-evaluate their value. This is especially useful when some code was written for Java 1.2 and is now being used on 1.5. log4j was handy in ’99, but Java 1.4 added java.util.logging. In the recent past, I’ve continued to use log4j mostly because it’s much easier to configure than java.util.logging. That’s a pretty silly reason and can be fixed with a little code, so I decided to give log4j the old heave ho. Well, I quickly found out that java.util.logging’s Logger is not as easy to use as log4j. java.util.logging expects you to type “logger.log(Level.INFO, message)” while in log4j it’s just “logger.info(message).” Plus, there isn’t a “isInfoEnabled()” method. There are many other convenience methods missing. This RFE from 2001, lists the problems. I quickly created a JvLogger class that extends Logger and adds the convenience methods. That didn’t work because the java.util.logging.LogRecord class scans through the stack trace entries to find out who called the log() method. Using JvLogger, it’s always JvLogger that calls log(). The problem is documented in this bug, also from 2001. It’s surprising that after all these years that something as simple as a logging API is still broken.

Amazing that Sun hasn’t addressed this issues. Looks like its back to log4j!