What Problem Would Next-Generation Functional Testing Tools Solve?:
Various people have tried various structures including table driven things (like FIT/FITnesse) and Domain Specific Languages. But none of these approaches has quite succeeded in transforming the state of the practice the way JUnit did for unit testing.
Elisabeth Hendrickson has posted several posts on next gen testing tools recently.
With HostedQA, which, by the way, recently had some major developments I’ll be announcing soon, I tried to attack two main problems that crop up when writing very rich tests like functional and acceptance tests: infrastructure and * maintenance*. I’m sure that there is a lot to do on the reporting front as well, but I’m leaving that alone for now.
By infrastructure, I mean challenges like setting up the test environment, keeping it in a consistent state, ensuring you have all the tools/browsers/databases/whatever always available for automation, running your tests in a timely manner, etc. All of these are much more challenging that JUnit, simply because JUnit only concerned itself with in-memory state and fixtures that could be recreated in milliseconds, not minutes or hours.
I’d like to dive in to the solutions I’ve come up with for both of these, but it’s getting late and I’m already in the wrong time zone. But I promise to follow up in the next day or two on how HostedQA has addressed infrastructure and maintenance problems, all with the goal of making functional testing as ubiquitous as JUnit is.
For now, I leave you with this: These are the two biggest challenges that I see with this type of testing. Do you agree? Disagree? What is your biggest barrier to entry for this type of testing?