I wanted to prepare an article that specifies the very basic best practices for Automated Testing, whether its functional or non-functional. It seems to me that all the tools, whether open-source or commercial allow for these essential practices, presumably because this is the way we should all be doing it… some of us aren’t, you know who you are and you’re very naughty. Also your testing isn’t as good as it should be…
This article is very much a work-in-progress as I braindump…
- All scripts should have a manual equivalent, because this is the foundation upon which the automation is built. It doesn’t have to be complicated, it doesn’t have to be as thorough as the automated script, but it does have to accurately specify the steps to follow.
- Some of the tools are best used in a record-edit-playback loop (Loadrunner for example). Others pretty much require you to edit them as you progress through your script, adding checkpoints and things as you go (QTP springs to mind). If its a pain to go and insert checks after the recording, that is just tough. Your test is only as good as the conditions it checks for.
- A good understanding of the tool is required before you begin any testing phase. Yes, we all have to start somewhere, and that somewhere should always be with training additional reading.
- All scripts have transactions, whether its a page click, a text entry, a mouseclick, whatever. Defining your test by these transactions makes debugging easier, makes the scripts more readable and ultimately makes you look professional. Alternatively, a multi-step business process should be mapped as a single logical transaction.
- All scripts should include checkpoints for every “transaction” regardless of whether there’s a condition defined. Again its not complicated but your checkpoints need to be sufficient that if something is broken AT ANY POINT your test can detect it.
If you check for something unique to where you are in the AUT and set pass/fail criteria around that, that’s perfect. The sooner a test which has failed is known to have failed is the sooner the processor is moving on the next activity, saving time and money. And the sooner you’ll be debugging which is one of the fun bits in my opinion