Next Gen Testing Tools

What Problem Would Next-Generation Functional Testing Tools Solve?:

Various people have tried various structures including table driven things (like FIT/FITnesse) and Domain Specific Languages. But none of these approaches has quite succeeded in transforming the state of the practice the way JUnit did for unit testing.

(Via Ruminations.)

Elisabeth Hendrickson has posted several posts on next gen testing tools recently.

With HostedQA, which, by the way, recently had some major developments I’ll be announcing soon, I tried to attack two main problems that crop up when writing very rich tests like functional and acceptance tests: infrastructure and * maintenance*. I’m sure that there is a lot to do on the reporting front as well, but I’m leaving that alone for now.

By infrastructure, I mean challenges like setting up the test environment, keeping it in a consistent state, ensuring you have all the tools/browsers/databases/whatever always available for automation, running your tests in a timely manner, etc. All of these are much more challenging that JUnit, simply because JUnit only concerned itself with in-memory state and fixtures that could be recreated in milliseconds, not minutes or hours.

When I talk about maintenance, I mean the issue of describing tests in a way that is usable by the test writer and evolving the test as the product and requirements evolve. Again, with JUnit test description was easy: the test writer had intimate knowledge with the item being tested and was able to write it in the same language. For complex testing, there are literally a handful of languages being tested (SQL, Java/Perl/Ruby/etc, HTML, JavaScript, Flash, PL/SQL, etc), so right off the bat there is a big disconnect between the simplicity of JUnit and this problem.

I’d like to dive in to the solutions I’ve come up with for both of these, but it’s getting late and I’m already in the wrong time zone. But I promise to follow up in the next day or two on how HostedQA has addressed infrastructure and maintenance problems, all with the goal of making functional testing as ubiquitous as JUnit is.

For now, I leave you with this: These are the two biggest challenges that I see with this type of testing. Do you agree? Disagree? What is your biggest barrier to entry for this type of testing?