One For and Two Against Test Scripts

The following is a follow-up to my own Technical Documentation as Reference Material as well as David Greenlees' "Idiot Scripts".

Several thoughts come to my mind when I think about test scripts written for others to use.

I have at times, even recently, asked other people to write test scripts. Not because I want to use them or have them ready to distribute to someone else but because I wanted to use it as a tool to give me insight their approach to testing. It probably isn't the most efficient method but it was what seemed to be the best solution for the circumstances.

To me the intent of scripts for use by business users or during UAT is basically the same: happy path/positive testing that shows that the system works as expected.

The problem I have with writing scripts for business users is that I expect them to know how the system works and test scripts are a horribly inefficient form of user documentation. Besides it leaves the intent of each step in obscurity. It makes more sense to me to teach the business user how to use the system and then let them use it, whether it's taught through a full blown manual, a help file, a training seminar, or a phone call. If the system is complicated enough that it isn't readily apparent to the average user how to use it they you're going to need some sort of training program regardless so why duplicate effort by writing test scripts?

The problem I have with writing scripts for UAT is the same as I mentioned above but it goes deeper. Some, perhaps most people, might not agree with me. When I think about writing UAT scripts, it gives my heart ethical palpitations! UAT isn't just verifying functionality, it's verifying that it's acceptable. Determining whether the software/website is acceptable is a judgement call that only the client can make. Granted acceptance criteria can be written out and those criteria can be negotiated between the client and the agency but it's still a subjective evaluation when it comes down to it. The specific problem then that I have with UAT scripts is that I, as the script writer, am determining whether the deliverable is or is not acceptable. If the client wants to write an objective set of steps that define acceptability they can do that but that's on them. And if they want to go through some sort of approval process then it just becomes a dog and pony show.

Technical Documentation as Reference Material

In my line of work, I have been asked many times over the years to produce test scripts so that the client can "test" and make sure that the system is working as it's supposed to. Another way to put this is to document a complex, technical process so that someone with no experience in the field can do it. For some reason, I thought the testing field was uniquely privileged with these requests but that's not true. Similarly purposed documents are commonplace across disciplines whether it's requirements gathering or system deployment.

While the actual function may vary the fallacy in these documents is uniform. If the purpose of the document was simply to provide information to skilled people, there'd be no problem. However, the specific, implicit purpose of these documents is more often than not for unskilled people. A document is no replacement for skill or training because a document is limited whereas the realm of possibilities is infinite. Without skill, you'll be swallowed by infinity faster than the fish swallowed Jonah.

So why do we keep doing this?

Because it's an industry practice?
This isn't elementary school where we do things just because it's popular. We should foster practices that produce the best results.

Because the client asked for it?
We're not just code monkeys pumping out a website, we're consultants. It's our duty to the client to educate them on why technical processes should be executed by people with skills in those processes. The truth of the matter is that by doing so, we help to insulate themselves from disaster.

Because the client offered to pay us a lot of money for it?
Do I even need to say that this would be unethical? I hope this is never a factor. Yes, the client is paying us but with that we have an obligation to better the client not just give them what they want because of it.

Skilled Work for Skilled People
The most important thing that we need to take to heart is, as I said before, technical processes should be executed by people with skills in those processes otherwise the exposure to risk becomes extraordinary.

• Why risk having a site that isn't designed to do what it needs to? (BA)
• Why risk having a site that can't do what it's designed to do? (Development)
• Why risk having a site that doesn't do what it's designed to do? (Testing)
• Why risk having a site that isn't up when it needs to be? (Deployment)

References, Not Instructions
With that in mind, I think we ought to shift our paradigm about how we think about technical documents. They're not instructions, they're references. They guide a familiar process for someone that lacks specific knowledge for a given situation. When necessary, training should be offered to fully realize the necessary knowledge transfer. The training though is not to impart a skill, such as coding in PHP, but to apply an existing skill to a situation, such as coding a widget within the existing framework and dependencies.