When I was in high school, a class that I was in was given a writing assignment. This was a type of class that ordinarily wouldn't have writing assignments and so the students were a little unsure as to the requirements. One of my classmates asked, "How long should the paper be?" And the perverted old man replied, "The paper should be like a woman's skirt: long enough to cover the subject but short enough to still be interesting."
Today I want to expound upon effective defect reports. I am an absolute stickler for detailed defect reports. I have no problem sending issues back that don't have sufficient information. Defect reports are adult writing assignments and we need to make sure that we are including all of the necessary information. I could run down my list of criteria that I expect in every bug report, but you can find those types of lists lots of places. Besides, the list, like everything in testing, is context-oriented.
As inappropriate as my teacher was, he did imply an important point: it's not about a measurement, it's about accomplishing a purpose. Skirts, and clothing in general, maintain a certain level of modest, protect us from the elements, and evoke intrigue. Defect reports, must be able to fulfill two very important functions, otherwise they are a failure.
An effective defect report makes reproduction and troubleshooting as simple as possible.
More often than not, the person who discovers the defect is not the person who resolves the defect. This is a simple efficiency issue. If you have a PM or Scrum Master reviewing issues, they need to be able to assess the priority and assign the issue to the most appropriate developer. The developer should be able to understand what's going on without having to guess or ask additional questions so they can spend less time identifying the issue and more time fixing.
An effective defect report archives the defect.
The purpose of this is for testing. Often times, the person who discovers the defect is not the person who verifies that it has been fixed. The tester needs to be able to know exactly what was wrong so that when they test the resolution, they can determine beyond a doubt whether or not the defect is fixed. If the tester doesn't know precisely what was wrong, it's impossible to know if it's fixed.
If your bug reports fulfill those functions then you're well on your way to getting an A+ on your writing skills. Remember, context is everything - sometimes bug reproduction is elusive and sometimes a conversation can convey imporant information that is hard to express in text. If you keep mind of the purpose then the details will fall into place.
Showing posts with label Technical Documentation. Show all posts
Showing posts with label Technical Documentation. Show all posts
Wednesday, June 6, 2012
Thursday, May 10, 2012
One For and Two Against Test Scripts
The following is a follow-up to my own Technical Documentation as Reference Material as well as David Greenlees' "Idiot Scripts".
Several thoughts come to my mind when I think about test scripts written for others to use.
I have at times, even recently, asked other people to write test scripts. Not because I want to use them or have them ready to distribute to someone else but because I wanted to use it as a tool to give me insight their approach to testing. It probably isn't the most efficient method but it was what seemed to be the best solution for the circumstances.
To me the intent of scripts for use by business users or during UAT is basically the same: happy path/positive testing that shows that the system works as expected.
The problem I have with writing scripts for business users is that I expect them to know how the system works and test scripts are a horribly inefficient form of user documentation. Besides it leaves the intent of each step in obscurity. It makes more sense to me to teach the business user how to use the system and then let them use it, whether it's taught through a full blown manual, a help file, a training seminar, or a phone call. If the system is complicated enough that it isn't readily apparent to the average user how to use it they you're going to need some sort of training program regardless so why duplicate effort by writing test scripts?
The problem I have with writing scripts for UAT is the same as I mentioned above but it goes deeper. Some, perhaps most people, might not agree with me. When I think about writing UAT scripts, it gives my heart ethical palpitations! UAT isn't just verifying functionality, it's verifying that it's acceptable. Determining whether the software/website is acceptable is a judgement call that only the client can make. Granted acceptance criteria can be written out and those criteria can be negotiated between the client and the agency but it's still a subjective evaluation when it comes down to it. The specific problem then that I have with UAT scripts is that I, as the script writer, am determining whether the deliverable is or is not acceptable. If the client wants to write an objective set of steps that define acceptability they can do that but that's on them. And if they want to go through some sort of approval process then it just becomes a dog and pony show.
Several thoughts come to my mind when I think about test scripts written for others to use.
I have at times, even recently, asked other people to write test scripts. Not because I want to use them or have them ready to distribute to someone else but because I wanted to use it as a tool to give me insight their approach to testing. It probably isn't the most efficient method but it was what seemed to be the best solution for the circumstances.
To me the intent of scripts for use by business users or during UAT is basically the same: happy path/positive testing that shows that the system works as expected.
The problem I have with writing scripts for business users is that I expect them to know how the system works and test scripts are a horribly inefficient form of user documentation. Besides it leaves the intent of each step in obscurity. It makes more sense to me to teach the business user how to use the system and then let them use it, whether it's taught through a full blown manual, a help file, a training seminar, or a phone call. If the system is complicated enough that it isn't readily apparent to the average user how to use it they you're going to need some sort of training program regardless so why duplicate effort by writing test scripts?
The problem I have with writing scripts for UAT is the same as I mentioned above but it goes deeper. Some, perhaps most people, might not agree with me. When I think about writing UAT scripts, it gives my heart ethical palpitations! UAT isn't just verifying functionality, it's verifying that it's acceptable. Determining whether the software/website is acceptable is a judgement call that only the client can make. Granted acceptance criteria can be written out and those criteria can be negotiated between the client and the agency but it's still a subjective evaluation when it comes down to it. The specific problem then that I have with UAT scripts is that I, as the script writer, am determining whether the deliverable is or is not acceptable. If the client wants to write an objective set of steps that define acceptability they can do that but that's on them. And if they want to go through some sort of approval process then it just becomes a dog and pony show.
Tuesday, April 24, 2012
Technical Documentation as Reference Material
In my line of work, I have been asked many times over the years to produce test scripts so that the client can "test" and make sure that the system is working as it's supposed to. Another way to put this is to document a complex, technical process so that someone with no experience in the field can do it. For some reason, I thought the testing field was uniquely privileged with these requests but that's not true. Similarly purposed documents are commonplace across disciplines whether it's requirements gathering or system deployment.
While the actual function may vary the fallacy in these documents is uniform. If the purpose of the document was simply to provide information to skilled people, there'd be no problem. However, the specific, implicit purpose of these documents is more often than not for unskilled people. A document is no replacement for skill or training because a document is limited whereas the realm of possibilities is infinite. Without skill, you'll be swallowed by infinity faster than the fish swallowed Jonah.
So why do we keep doing this?
Because it's an industry practice?
This isn't elementary school where we do things just because it's popular. We should foster practices that produce the best results.
Because the client asked for it?
We're not just code monkeys pumping out a website, we're consultants. It's our duty to the client to educate them on why technical processes should be executed by people with skills in those processes. The truth of the matter is that by doing so, we help to insulate themselves from disaster.
Because the client offered to pay us a lot of money for it?
Do I even need to say that this would be unethical? I hope this is never a factor. Yes, the client is paying us but with that we have an obligation to better the client not just give them what they want because of it.
Skilled Work for Skilled People
The most important thing that we need to take to heart is, as I said before, technical processes should be executed by people with skills in those processes otherwise the exposure to risk becomes extraordinary.
References, Not Instructions
With that in mind, I think we ought to shift our paradigm about how we think about technical documents. They're not instructions, they're references. They guide a familiar process for someone that lacks specific knowledge for a given situation. When necessary, training should be offered to fully realize the necessary knowledge transfer. The training though is not to impart a skill, such as coding in PHP, but to apply an existing skill to a situation, such as coding a widget within the existing framework and dependencies.
While the actual function may vary the fallacy in these documents is uniform. If the purpose of the document was simply to provide information to skilled people, there'd be no problem. However, the specific, implicit purpose of these documents is more often than not for unskilled people. A document is no replacement for skill or training because a document is limited whereas the realm of possibilities is infinite. Without skill, you'll be swallowed by infinity faster than the fish swallowed Jonah.
So why do we keep doing this?
Because it's an industry practice?
This isn't elementary school where we do things just because it's popular. We should foster practices that produce the best results.
Because the client asked for it?
We're not just code monkeys pumping out a website, we're consultants. It's our duty to the client to educate them on why technical processes should be executed by people with skills in those processes. The truth of the matter is that by doing so, we help to insulate themselves from disaster.
Because the client offered to pay us a lot of money for it?
Do I even need to say that this would be unethical? I hope this is never a factor. Yes, the client is paying us but with that we have an obligation to better the client not just give them what they want because of it.
Skilled Work for Skilled People
The most important thing that we need to take to heart is, as I said before, technical processes should be executed by people with skills in those processes otherwise the exposure to risk becomes extraordinary.
- Why risk having a site that isn't designed to do what it needs to? (BA)
- Why risk having a site that can't do what it's designed to do? (Development)
- Why risk having a site that doesn't do what it's designed to do? (Testing)
- Why risk having a site that isn't up when it needs to be? (Deployment)
References, Not Instructions
With that in mind, I think we ought to shift our paradigm about how we think about technical documents. They're not instructions, they're references. They guide a familiar process for someone that lacks specific knowledge for a given situation. When necessary, training should be offered to fully realize the necessary knowledge transfer. The training though is not to impart a skill, such as coding in PHP, but to apply an existing skill to a situation, such as coding a widget within the existing framework and dependencies.
Subscribe to:
Posts (Atom)