Technical Documentation as Reference Material

In my line of work, I have been asked many times over the years to produce test scripts so that the client can "test" and make sure that the system is working as it's supposed to. Another way to put this is to document a complex, technical process so that someone with no experience in the field can do it. For some reason, I thought the testing field was uniquely privileged with these requests but that's not true. Similarly purposed documents are commonplace across disciplines whether it's requirements gathering or system deployment.

While the actual function may vary the fallacy in these documents is uniform. If the purpose of the document was simply to provide information to skilled people, there'd be no problem. However, the specific, implicit purpose of these documents is more often than not for unskilled people. A document is no replacement for skill or training because a document is limited whereas the realm of possibilities is infinite. Without skill, you'll be swallowed by infinity faster than the fish swallowed Jonah.

So why do we keep doing this?

Because it's an industry practice?
This isn't elementary school where we do things just because it's popular. We should foster practices that produce the best results.

Because the client asked for it?
We're not just code monkeys pumping out a website, we're consultants. It's our duty to the client to educate them on why technical processes should be executed by people with skills in those processes. The truth of the matter is that by doing so, we help to insulate themselves from disaster.

Because the client offered to pay us a lot of money for it?
Do I even need to say that this would be unethical? I hope this is never a factor. Yes, the client is paying us but with that we have an obligation to better the client not just give them what they want because of it.

Skilled Work for Skilled People
The most important thing that we need to take to heart is, as I said before, technical processes should be executed by people with skills in those processes otherwise the exposure to risk becomes extraordinary.

• Why risk having a site that isn't designed to do what it needs to? (BA)
• Why risk having a site that can't do what it's designed to do? (Development)
• Why risk having a site that doesn't do what it's designed to do? (Testing)
• Why risk having a site that isn't up when it needs to be? (Deployment)

References, Not Instructions
With that in mind, I think we ought to shift our paradigm about how we think about technical documents. They're not instructions, they're references. They guide a familiar process for someone that lacks specific knowledge for a given situation. When necessary, training should be offered to fully realize the necessary knowledge transfer. The training though is not to impart a skill, such as coding in PHP, but to apply an existing skill to a situation, such as coding a widget within the existing framework and dependencies.

Thoughts on Effective Project Management

Effective project management and execution has many components. To name a few that comes to mind:

• Planning
• Communication
• Cooperation

Projects don't always go smoothly though and so we compensate with:

• Process
• Documentation
• Incentives (or bribery)

Don't get me wrong, process, documentation, and incentives are not bad things but they're not guarantees for great projects. Just because you have process doesn't mean your project will be executed well. Just because you have documentation doesn't mean the project will be universally perfectly understood. Just because you have incentives does not mean your team will be motivated.

When the project struggles, don't panic and don't think that there's going to be a miracle cure. So what should you do? Breathe. Relax. Rather than jumping to a change in the model, figure out first if the model is being effectively executed. Sometimes all it takes is some individual coaching or team training.

Think for a second about treating a medical ailment:
Symptoms: Excruciating pain, being unable to walk or stand
Solution: Prescribe heavy medications, such as Oxycodone. If you can't feel the pain, then what does it matter right? But if you only treat the symptoms and don't fix the problem, the symptoms will reoccur as soon as the drugs wear off.

Problem: The leg is broken
Solution: Set the let and/or perform surgery to bolt the bone back together. YES! We are out of pain and we're mobile again. But if you only fix the problem and don't eliminate the cause, then someone is sure to come along and break their leg--OR WORSE!

Cause: The rug is bunched up
Solution: Smooth out the rug. If the rug is prone to bunching, tack it down or figure out what causes it to get bunched up. Perhaps it's to close to a door that opens frequently. If tacking is not an option or it can't be moved further from the door, replace the rug with one that has a non-slip back or that is shorter.

I contend that most issues in project management and execution are like this. Simple causes that, when ignored, cause expensive problems and exhibit symptoms disconnected with the cause itself. Fixing the rug is a simple fix. It's not a change to the model, it's just taking the existing model and working the kinks out.

[Not] Everything Is Urgent!

When projects get down to the wire, sometimes certain people (you know who they are) become prone to throwing out the rules for determining defect priority. As I previously wrote in Prioritizing Defect Reports, there are four factors that I consider when setting priority: Severity, Exposure, Business Need, and Timeframe. Unfortunately, Timeframe becomes a stumbling block when they fall prey to the terminal thought, "The deadline is right around the corner and all of these issues need to be done, therefore they are all Urgent!"

I call this a "terminal" thought because it leads to a disastrous method of project management: panic. Panic management occurs when organization goes out the window; and that's exactly what happens when all issues are prioritized the same. The priority indicator loses its value. Even when time is running short and all of the issues are crucial to launch, issues varying levels of priority and some need to be done before others. And what happens when nothing has any priority? Developers decided themselves which issues to do and when.

When we get to crunch time, I think it's appropriate to not only redefine the spans of time for the Timeframe factor but redefine the factor. Instead of thinking about a Timeframe, think about a Sequence:

• Urgent: Drop everything and work on this
• High: Complete before working on any lower priority issues
• Normal and Low: After confirming with the project manager that there is nothing more important that needs to be done, concentrate on the Normal priority issues first but may incorporate Low priority issues if there's an efficiency advantage. Low priority issues are completed last.

By maintaining your system of priorities, you'll help keep your team focused and everyone will have a clearer vision of the outstanding risk in meeting the deadline.

The Twitter Gun

While I've had a Twitter account for a couple of years, I have only recently begun to utilize it. And while I'm definitely not an expert in Twitter strategy, I am a user and therefore I know what practices annoy the bajeebers out of me.

Twitter is like a mini blog and should generally follow the same guidelines as a regular blog. So in a regular blog, at least every one that I've ever read, each entry is unique. Yes, there may be recurring themes and topics but no one ever reposts a single entry multiple times. By and large, Twitter should be the same. However, some companies I've noticed like to tweet the same thing over and over. Perhaps they have one important message that they're trying to share like, "We're hiring" but they also like to share other useful things like, "Hey check out this link". And so after each time they post a "Hey check out this link" tweet, they repost the "We're hiring" tweet.

To me this practice is Twitter SPAM. It's unprofessional. It's rude to your followers. It encourages your followers not to pay attention to your tweets - which is the absolute last think you want to happen. And if you happen to be a company that claims to be a social media expert, it can be bad for business.

Consider the similarities and contrasts between two types of guns: the flare gun and handgun. Both are used to launch projectiles. Both are used to convey a message. A flare gun, however, will only shoot off one round at a time while a hand gun can fire many rounds in short succession. You run towards a flare gun but away from a handgun.

Twitter, used effectively, is like a flare gun because you want people to come to you. Therefore, you carefully plan your tweets and each contains a unique message. On the other hand, Twitter, used as I described above, is like a handgun. There's no strategy; you're just blasting away and people are going to run because it's the same thing over and over.

Of course, I think there's a third type of Twitter gun too, the machine gun. These are the companies that just don't stop sending out tweets. Seriously? While you are a news service, you're going to send out a tweet every three minutes when you post a new article? UN-SUB-SCRIBE! Maybe some people don't mind that but to me it just clogs up the feed and makes the useful information impossible to glean.

So if you run a company with an active Twitter strategy, think about how you're affecting your followers. Plan your tweets and don't inundate your followers with repetitive information.

The Quality Assurance Misnomer

Over the years, the industry standard title for a software tester has become “Quality Assurance Analyst/Engineer”. I don’t know the history of this but I do know that it’s not without controversy. When most people in the software industry hear the title, they think of the person’s role as being someone who assures the quality of the software product.  That’s a big problem though because, as a Quality Assurance Engineer myself, that’s not what my job is, nor is it generally the job of any software tester that I know.

Here’s the root of the problem: Quality is a business decision that evaluates whether a product or part thereof is GOOD or BAD and that is a decision that lies in the hands of the product owner – that person which is or represents the product purchaser/user. The reason for this is because they are the person that best knows whether the purchaser/user will be satisfied. Testers don't know the user so it would be presumptuous at best to put that responsibility on them

What I do is I poke, stress, and exercise the software in every way that I can think of and make observations. I compare those observations to documentation and common heuristics that help identify when intended functionality deviates from actual functionality. And then my job is actually to help inform that business decision by reporting all of my observations through conversation, defect reports, and progress reports to the product owner. You see, I can report RIGHT or WRONG based on my comparison to documentation and heuristics but that’s different than GOOD or BAD.

Determining GOOD or BAD is a murky process that is above the pay grade of a lowly software tester and that has to factor in many components including:

• Test coverage
• How stale is the testing (i.e. when was the last time part x was tested?)
• How many failed test cases are outstanding
• What the severity is of the outstanding defects
• Amount of time allowed for planning, developing, and testing
• The gut factor (i.e. does it feel like it’s ready?)

Then taking all of this information, to determine whether the product is GOOD or BAD, you need to have a sense of whether the customer will be satisfied as I mentioned above. The only way to develop this sense is to have an active, direct interaction with the customer from the beginning of the project or to just leave it directly up to the customer. Though, I don’t recommend the latter because without somehow quantifying an acceptance level, they may never agree that the product is good.

All that said, I don’t think that the title “Quality Assurance Analyst/Engineer” is BAD, just misunderstood. While I don’t assure quality through analysis and/or engineering, I do analyze and engineer so that quality can be assured.

WordPress: PHP Is Not Installed

So I'm starting to work on a little project at home involving a website built in WordPress. This is all new to me so naturally, there are going to be bumps in the road. Anyway, I managed to install Apache, MySQL, and PHP. However, when attempting to install WordPress, I was continually presented with the error:

Error: PHP is not running
WordPress requires that your web server is running PHP. Your server does not have PHP installed, or PHP is turned off.

WHAT??? The PHP test file worked perfectly fine! I googled and googled some more trying to find a solution to this and while there question appeared frequently, there was never a good answer. I finally figured it out myself though!

The problem was in how I was accessing install.php. Through Windows Explorer, I had navigated to my WordPress directory and double clicked on readme.html. The file opened in Firefox. Then I clicked the hyperlink in Step 2 "wp-admin/install.php". This opened the file with the error, and here's the problem: look at the address bar:

file:///C:/apache/htdocs/wordpress/wp-admin/install.php

In order to make it work, you need to hit it from LOCALHOST

http://localhost/wordpress/wp-admin/install.php

BAM! Done! 

Environmental Variables:
OS: Windows XP SP3
Webserver: Apache 2.2
MySQL: 5.5.21
PHP: 5.2.17
WordPress 3.3.1

Gems of Wisdom from Fellow Software Testers

This past week, I've been feeling very cerebral and the timing couldn't have been better because I've been able give my blog some much needed attention and I've also been able to get almost caught up on the backlog of blogs that I follow. In my reading, I have come across a number of gems of wisdom that I think are well worth sharing:

James Bach in Why Scripted Testing is Not for Novices

• [A] scripted tester, to do well, must apprehend the intent of the one who wrote the script. Moreover, the scripted tester must go beyond the stated intent and honor the tacit intent, as well– otherwise it’s just shallow, bad testing. - TW: This problem is a direct result of divorcing test design and execution. And for a novice tester, they simply don't have the skills yet to "read between the lines" of the script to see the intent.

Michael Bolton in Why Checking Is Not Enough

• But even when we’re working on the best imaginable teams in the best-managed projects, as soon as we begin to test test, we begin immediately to discover things that no one—neither testers, designers, programmers, nor product owner—had anticipated or considered before testing revealed them.
• It’s important not to confuse checks with oracles. An oracle is a principle or mechanism by which we recognize a problem. A check is a mechanism, an observation linked to a decision rule.
• Testing is not governed by rules; it is governed by heuristics that, to be applied appropriately, require sapient awareness and judgement.
• A passing check doesn’t tell us that the product is acceptable. At best, a check that doesn’t pass suggests that there is a problem in the product that might make it unacceptable.
• Yet not even testing is about telling people that the product is acceptable. - TW: I've been trying to promote this concept for at least a year in my own evolutionary understanding of my craft. You can expect my own blogging on this topic soon.
• Testing is about investigating the product to reveal knowledge that informs the acceptability decision.

Michael Bolton in What Exploratory Testing Is Not (Part 3):  Tool-Free Testing

• People often make a distinction between “automated” and “exploratory” testing. - TW: This is the first sentance and BAM! did it cause a paradigm shift for me!
• That traditional view of test automation focuses on performing checks, but that’s not the only way in which automation can help testing. In the Rapid Software Testing class, James Bach and I suggest a more expansive view of test automation: any use of tools to support testing.

Anne-Marie Charrett in Please don't hire me

• If you want me to break your code - TW: This set my brain going for at least two hours thinking about the implications of this. It's a great point though, testers don't break code. Look forward to more on this in the future.

I hope you find some wisdom in this too.

Counting My M&M's

Like millions of other Americans, I keep a stash of munchies in my desk drawer at work. I find that a little treat in the afternoon is a highly effective way to keep me focused on my work. One of my snacks, interestingly (or oddly), has evolved into a bit of a ritual.

It all started back in 2010. I was at my local Dominick's before work picking up a donut, something for lunch, and stock for the stash. I happened down the candy aisle and saw that they had the jumbo bags of M&M's on sale. I'm always a sucker for the lowest price per unit so I couldn't resist. Fast forward to the afternoon-snack time-and a funny little thought popped into my head: "Are there the same number of each color of M&M's in the package?" I decided to find out, just for the heck of it.

The jumbo sack of M&M's is 42 ounces so there are two things to keep in mind. 1. I do not eat the entire thing in one helping. 2. I was not about to dump out the whole sack and count them right then and there. Instead, I started a spreadsheet. Whenever I decided to have a handful of M&M's, I would first sort them by color, count them, and then record the stats in my spreadsheet. It didn't take too long for multiple passersby to notice and remark upon my method for eating M&M's. Suddenly I had a reputation that needed to be maintained! Never again could I reach into a sack of M&M's and NOT count each color.

Two years later, I'm now on my fourth sack of M&M's. I don't eat them every day and I don't always have them in my stash. But when I do, I keep statistics. Today, I am pleased to announce the public sharing of this information on my M&M Counter page!

The M&M Counter page has several fun features:

• The graph at the top shows the combined stats for all of the M&M's that I have consumed over the years.
• Then, I provide a graph and chart that are updated in real time that show the stats for the sack that I'm currently in the process of devouring.
• Finally, I have provided a form, for what I think is the most exciting feature of all, to allow YOU to join in the fun. You are welcomed and encouraged to contribute your own counts. Currently, the form only supports the regularly colored candies. Pick the closest option when selecting the size of the package. Soon, I will include a graph or two to display the user submitted data.

So yeah, it's a little weird. There's nothing scientific about this. There's no ploy to identify certain colors as subject of discrimination. It's just for fun. Enjoy!

Go to the M&M Counter Now

Paper Airplane Science Experiment

When I was in elementary school, we would have a science fair every other year and all of the students would do some sort of science project. In fifth grade I simulated acid rain and in seventh grade I made recycled paper. I didn't find the acid rain experiment to be fun at all and it had been the only thing I could think of. The paper experiment was more fun because I got to make a big mess but it still didn't engage me, even though it was something I really wanted to do. So for  you parents out there trying to help your kid find something different to do for a science experiment, here's an idea: PAPER AIRPLANES.

Lots of little kids like making paper airplanes and if you think about it, it's super-simple to turn it into a science experiment and there's a a number of different approaches that you can take. The idea when doing different experiments, is that you want to identify all of the different variables and then just change ONE. In any approach, you'll form hypotheses around the same questions:

1. Which airplane flies the farthest?
2. Which airplane flies the straightest?

Experiment Method 1: Airplane design
Go online (or to the library) and find instructions on how to create several different designs of paper airplanes. Then construct them using the SAME size and type of paper. If you want, you could measure the wing dimensions and area and use those numbers to form your hypotheses.

Experiment Method 2: Airplane material
Pick a single airplane design and change up the type of paper used. Make sure the size of the paper doesn't change though. You could use construction paper, aluminum foil, printer paper, wrapping papper, cardstock, or whatever else you can think of. Here you'll probably want to weigh each airplane and use that for forming hypotheses.

Experiment Method 3: Airplane size
Again, pick a single airplane design but this time change up the size of the paper but use same type of paper. You may need to cut down large pieces of paper to make smaller planes. I say, find the biggest cut of paper you can find. Most schools have large rolls of paper for making big posters or bulletin boards. For each variation, make sure that the length to width proportion of the paper remains constant.

Experiment Method 4: Airplane proportions
Pick a single airplane design and type of paper. This time, vary the dimensions of the paper, while maintaining the same number of square inches. For example: (12x10) and (15x8) both have 120 square inches of area. You could also do (12x10) and (10x12) - rotating the axis of the plane. In doing this type of experiment, you should take lots of measurements in forming your hypotheses: wing area, plane length, wing width, plane height.

Conducting the experiment
Reduce the potential for errors:

• I would make three copies of each plane to try to account for any variations there might be in the construction. Also, practice making the plane first and then make new ones for the experiment.
• Throw and measure each plane at least five times. The more times you throw it, the more accurate your data will be, unless, of course, the plane gets really beat up from doing a nosedive into the floor. You can always disregard measurements (so you take 10 but only use the first 5 because you see that after that the data started to skew from bent noses).

Location selection:

• You should try to find a wide open place to throw the airplanes. The school gymnasium would be a great place because it's wide open and there won't be any wind.

Location preparation:

• Place a mark on the floor from where you will launch the planes each time (Use gaffers tape in the gymnasium so you don't gunk up the floors!). Mark out a straight line from that point. You might want to put a target on the wall or something that is in line with the starting point and straight line. When you throw the planes, aim for the target and that will help ensure that you throw consistently.

Throwing:

• Again, aim for the target and make sure that your throwing arm is lined up with your target.

Measuring:

• Measure the distance from the throwing point to where the plane LANDED. Have someone there to help watch in case the planes want to slide on the floor.
• Measure the angle between the center line and where the plane lands.

One Last Idea:
You could also experiment with airplane surface friction. I don't know how you would MEASURE friction though so I'm just floating this out there for you. Using glossy inkjet photo paper, the same size paper, dimensions, and airplane design, make one plane with the glossy side out and one plane with the back side out.

Share Your Results
If you decide to try this experiment in one form or another, please share your results. I would love to hear what you discover!

Editior Fail at Yahoo! News

While browsing the news at Yahoo! over my lunch break on my way to read Dear Abby, I noticed this rather prominent error. Don't rely on spell checker!

Predator II: Just Die Already!

What do they do in in Hollywood if a movie does well at the box office? Make a sequel and use the success of the first movie to milk consumers for more money. Predator did well, earning almost $100M so naturally, Predator II came along three years later. Sequels usually have a common relationship with the original. In the first movie, the hero, after great struggle, finally kills the villain and then the sequel comes along and somehow the villain is back to life causing even more havoc. I see that and get to thinking, "Gosh darn! Why won't that thing JUST DIE ALREADY!?" I ask that question about software defects often because often times, defects have sequels too (and so should blogs)!

Despite my best efforts - thorough coverage, testing every test case I can think of, making sure everything WORKS - out of nowhere pops up some bug that I thought was squashed weeks ago! Sometimes it doesn't matter how big your muscles are, how many bullets you have in your gun, or how many other people have their guns pointed at the same monster, they still come back to life.

This can be caused by any number of things. It could be laziness on the part of the tester (or the developer) but I don't think that's usually the case. Here are some other causes:

1. Unclear requirements
2. Inexperience
3. Lack of time
4. Introduction of new/altered code
5. Using [testing] the software in a way that it wasn't previously used

That last one is a big ball of mud - "WHY DIDN'T YOU TEST IT THAT WAY BEFORE???" See causes 1-4. The truth is, software is extremely complex and it's impossible to account for every scenario. As a tester, you must know how to prioritize and how to look for the scenarios that are most likely to occur and those which have the gravest consequences because there just isn't time/budget/people-power to do it all.

When defects do reoccur, use it as a learning catalyst. Was there an obvious breakdown in the development or testing methodology? Is this a "new" method of execution that should be noted for the future and with application other scenarios? Is there a different testing technique or tool that might have been able to expose that the defect was not fully resolved earlier?

Old defects will continue to pop up. Don't be discouraged but remain vigilant. Keep sharpening your skills and be ready for the enemy to return, because he always does, sometimes with Aliens.

Please study for my interview

I recently volunteered myself into the position of reviewing resumes, identifying candidates, and executing first round interviews for a QA position at my company. This has been a real learning experience for me as I've garnered a better understanding of the hiring process; but more so, it has been an eye opening experience. The great majority of applicants in the field are ROBOTS, figuratively speaking. Requirements get turned into scripts and that's how you test. As long as you have the right process, the right charts, the right formula, that's all you need.

I disagree. Don't get me wrong, I want someone that can transform requirements into test scripts but I what I really want is someone that can THINK. Someone that can test what's not in the requirements document. To quote Anne-Marie Charrett, I want to hire someone that says, "Don't hire me if you want perfect software. Don't hire me if you're looking for a tester to only "check" if things look ok."

For my interviews, I developed a standard list of questions to ask. Each question was carefully thought out and has a specific purpose for being asked. Here's my list:

1. How did you get into software testing?
2. What do you like about testing?
3. What are your frustrations with testing and how do you deal with them?
1. For example: How do you deal with being the resident naysayer?
2. For example: How do you deal with defects that just never seem to get fixed?
4. What comes to mind when you hear the term ‘Quality Assurance’?
5. Compare and contrast automated and manual testing.
6. Compare and contrast scripted and exploratory testing.
7. How confident are you in your ability to deliver defect free software?
8. What essential information should be included in defect report?
9. How do you determine if something is working correctly when you have no documentation?
10. What’s your experience with Agile?
11. How do you sharpen your testing skills?

So there you go. If you're ever on the other side of the table being interviewed by me, nothing I ask should catch you by surprise. In fact, I encourage you to study for my interview. However, for now, I'm not going to explain any of my rationale; you'll have to come back for that later. For those of you that conduct interviews yourselves, is there anything you think I should add? Please share in the comments.

Using XQuery in MS SQL for Novices Part 2

How to use the value() function

Just the other day I started the conversation about how to use XQuery functions in SQL to pull specific XML data out of a table. It's time now for Part 2.

In the first discussion, I explained how to use the query() function. It's a very nice function because it returns both the XML nodes and the data within them. However, if all you want is the specific data in the node, having the nodes display in each result can make it more difficult to read the results. For this purpose we can use the value() function.

The value() function is ever-eager to throw the error "'value()' requires a singleton (or empty sequence)". It's also eager to throw the error "The value function requires 2 argument(s)." Obviously, these errors occur when we don't provide all of the necessary information.

The two arguments required by the values() function are:

1. The reference to the node
2. A data type specification in which to host the returned information

The requirement of a singleton is admittedly, not something I fully understand. It is solved, however, by appending a number one in brackets - [1] - to the end of the first argument. Just put everything inside of the single quotes and the argument itself inside of parentheses.

This is how it looks:

SELECT XMLColumn.value('(//XML-Node-Name)[1]', 'DataType(#)') FROM Table

Don't forget to add a comma between the arguments!
 
We can further tidy up the results by aliasing the column (assigning it a custom name of our choice) and using the exists() function.

SELECT XMLColumn.value('(//XML-Node-Name)[1]', 'DataType(#)') 'Alias Column Name' FROM Table WHERE XMLColumn.exists('//XML-Node-Name') = 1

What should be the easiest archaeological dig in the world

When I entered the field of software testing, the last last thing on my mind was archaeology. Software testing is about the future whereas archaeology is about the past. Technology is clean and non-physical but archaeology is dirty and labor-intensive. However, the two fields aren't all that different. A good software tester is going to thrive on artifacts, but not the kind buried in the ruins of ancient civilizations.

In software projects, an artifact is any piece of documentation that relates to the project:

• RFP
• RFP Response
• SOW
• Project Plan
• Requirements
• Design comps
• Story boards
• User personas
• Test scripts
• Defect reports
• Meeting summaries
• etc.

The artifacts mentioned above help software testers the same way bones help paleontologists. Both are vital in the acquisition of knowledge. Testers need to have a thirst for project knowledge. How can one assert if the product works as intended without having an authoritative source? Regardless of approach, the more the tester knows about how the software is supposed to work, the more thoroughly they can test through broader coverage and more intricate scenarios.

When projects don't have a decent amount of documentation, you're asking for scope creep and unsatisfied client expectations. Testers, get dirty and dig into the documentation. Insist on having access to it and demand it early. Study it thoroughly. Use it to help design your tests. If the documentation doesn't exist, consult the project experts (well, consult them anyway) and then write it down. Once it's written down, it can't be disputed - it might be changed later but that can happen to any document. Moral of the story: use everything at your disposal to learn about the project.

Using XQuery in MS SQL for Novices Part 1

A recent project at work presented me with a new challenge. Generally in a database, at least the ones that I've  had to work with, each different type of data is stored in its own column. Well, the project currently under test downloads information from a third party system in the form of an XML file. Even though we don't use every bit of data in provided by the third party, we still want to preserve it in its original format. Therefore we store the entire XML string in the database.

When XML data is stored in a database, there are at least a couple different data types that can be used.

1. It could be stored as just regular old text using the varchar data type. In that case whatever program is using the data has to convert it back into XML in order to use it. The Umbraco content management system likes to do it this way.
2. On the other hand, it could be stored natively using the XML data type, which is what our system does. Storing the data AS XML presents some interesting challenges as well as many opportunities when it comes to digging for information within the XML string itself -- most of which I'm still figuring out. For today, I'd like to start with just the basics.

Doing It Old School

If all you're trying to do is find the rows where a specific series of characters occurs, you can search just like you always do... almost. 

SELECT * FROM Table WHERE XMLColumn LIKE '%key-text%'

This traditional query will throw an error faster than a lazy developer's code. Instead, you need to change data types on the fly:

SELECT * FROM Table

WHERE CAST(XMLColumn AS varchar(max)) LIKE '%key-text%'

So to reiterate, this query uses the CAST command to convert the XML column into a text data type.

Being All Fancy About It

Like I said, the old school works fine if you're just trying to identify entire records in the table that match your criteria. If you're just looking for specific nodes in the XML data, the approach above is NOT very efficient. You can harness the power of the XML data type to do things much more quickly! Since v.2005, MS SQL Server has supported XQuery. I'm not a programmer so I don't yet understand the full power that this yields. I've found, however, that the existing help on the internet was not written for people like me so here's what I've learned so far:

STEP 1: Using query()

Assuming that we just want to find the data stored in a specific XML node, we can use the Query() function.

SELECT XMLColumn.query('//XML-Node-Name') FROM Table

A couple of notes about what's going on there:

1. Query() is a function. Inside of the function is one ore more arguments (RAWR! J/K), which are written inside of the parentheses. Each argument must be enclosed in single quotes and multiple arguments are comma separated. The query() function will accept accept ONE argument.
2. The double slashes // at the start of the argument tell the system that you're looking for a node and for our purposes, it doesn't matter whether it has child or parent nodes.
3. You need to put the goods of the function inside of the single quotes.
4. The node name is CASE SENSITIVE
5. Don't capitalize the the function: e.g. type "query" not "Query"

When this query runs, unless you add a WHERE clause, it's going return a result for every single row in the table, including the rows where the specified node does not exist. If the node does not exist for that row, there will be an empty cell in the results table. If the node does exist, the data returned will look like this:

<xml-node-name>XML data stored in node</xml-node-name>

STEP 2: Using exists()
Because using query() alone can be a little messy returning empty rows, we can use another function that checks YES or NO does something exist in the XML data.

SELECT XMLColumn.query('//XML-Node-Name') FROM Table WHERE XMLColumn.exists('//XML-Node-Name') = 1

 This query will only return the rows where the specified node exists without any empty rows. A couple of notes about what's going on here:

1. I used the same argument for both functions because if the node doesn't exist that I'm looking for, that row doesn't need to be returned.
2. The = 1 part is telling it to evaluate to TRUE (e.g. Yes, it exists) whereas, if I set it to = 0 it would be the opposite (e.g. No, it doesn't exist). Setting it to 0 would only return the blank rows in this example.

Please let me know in the comments if you've found this useful. Look for more posts to come as I learn to harness more power of the XQuery!

Helpful but technical links (also probably the first ones you came across when googling "XQuery SQL Server":

http://msdn.microsoft.com/en-us/library/ms345122%28v%3dsql.90%29.aspx

http://msdn.microsoft.com/en-us/library/ms345117%28v%3dsql.90%29.aspx

Collaborative Cursing SQL Style

Earlier this week at work, I had some discretionary time. I'm not quite sure what inspired me but I decided I wanted to brush up on my SQL scripting skills.  I'm not by any means a SQL scripting pro but I know how to find the data that I'm looking for. In my line of work, I'm generally just looking to verify that the data being written to the database is correct. Generally a simple query will do the trick but occasionally more complex scripts, like a cursor or a loop, make it easier. Unfortunately, I couldn't remember how to write a cursor. Oops. In fact, I couldn't even remember what they were called. Oops again. But that's why I wanted to practice so that when I actually need to write one, it'll be easy. I finally figured it out with the help of Google but I still ran into a couple of bumps in the road. The end result does work but I welcome tips for improvement.

Here's my cursor:

DECLARE @total int,
  @city varchar(64),
  @state varchar(16),
  @table char(64)

DECLARE city_cursor CURSOR FOR
 SELECT DISTINCT City, State FROM MyAddressTable
 ORDER BY City

SET NOCOUNT ON

CREATE TABLE #City_Count_Temp
 (
 City VARCHAR(64),
 State VARCHAR(16),
 Total INT
 )

OPEN city_cursor
FETCH NEXT FROM city_cursor INTO @city, @state

WHILE @@FETCH_STATUS = 0
BEGIN
 SELECT @total = (SELECT COUNT(City) FROM MyAddressTable WHERE City = @city and State = @state)
 
 --SELECT @city AS City, @state AS State, @Total AS Total

INSERT INTO #City_Count_Temp VALUES (@city, @state, @total)
 
 FETCH NEXT FROM city_cursor INTO @city, @state
 
END
CLOSE city_cursor
DEALLOCATE city_cursor

SET NOCOUNT OFF

SELECT * FROM #City_Count_Temp

DROP TABLE #City_Count_Temp

It's actually a pretty simple script. The purpose of the script is to count the occurrences of each city in the table and then output it into a nice pretty list. It accomplishes this by first querying the table for the unique list of City/State combinations. Then using this list it again curses through the table counts the number of occurrences of each and dumps the information to a temporary table. Finally, there's  a simple query of the temporary table that produces the final list.

So the bumps in the road that I couldn't quite figure out... First, the only way I could get the script to make a pretty list was write the data to a temporary table. You might have noticed a commented out line in the middle. Originally, this was how the script provided the output without writing to the temp table. The problem was that SQL Management Studio displayed a separate result for each curse of the script. If there's another way to do this, I'd love to know!

Also, I tried to set the table name (MyAddressTable) as a variable but for some reason, it would error every time saying that I needed to declare the variable (even though I had declared the variable).

So that's my little cursor. I'm happy to answer any questions about it. I'm also happy to receive any suggestions about how to improve it

Rules for Driving in Chicago

1. Watch out for the taxis - they are the absolute worst drivers there are
2. Watch out for the out-of-towners - possibly even worse than the taxi drivers, you know the taxi is going to cut you off but out-of-towners are just plain unpredictable
3. Watch out for the buses - they'll cut you off too so just stay out of the buss lane
4. Watch out for the bikers - they're incompetent when it comes to obeying the rules of the road
5. Know where you're going and how to get there
6. NO LOLLY-GAGING! Keep up with the traffic in front of you, if you're going to make a turn then TURN. To sum it up in one word: Commit
7. If you're not in the correct lane to make your turn, don't stop at the intersection and expect someone in the turn lane to let you in while you hold up everyone behind you - go around the block
8. Watch out for pedestrians
9. Watch out for bikers II - there are a lot of them, often they have their own lane so give them their space (don't cut them off when making a turn and look before you open your door when parking on the street)
10. Watch out for potholes - they're deep, jagged, and everywhere
11. Watch out for people double parked
12. When turning, get all the way over in the turn lane or to the edge of the street

What have I missed? Feel free to add more rules in the comments.

Concerned About API Healthcare's Acquisition? What You Can Do

I know I'm not the only one that hopes Kronos' acquisition of API Healthcare falls through. I also know that most people don't feel comfortable talking about it for fear of their jobs. THERE IS SOMETHING YOU CAN DO. Contact the Federal Trade Commission.

The FTC, in conjunction with the Department of Justice, is the government agency that approves mergers and acquisitions and makes sure that antitrust laws aren't broken. If you think this is a bad idea, contact the FTC.

Here are your options:
By email: antitrust@ftc.gov
This method isn't considered confidential, so if that's a concern...

By mail:
Federal trade Commission
Bureau of Competition-H374,
Washington, D.C. 20580
If you mark the mail "Confidential" it will be treated as such.

By phone:
1–877–382–4357

As I understand it, the FTC has issued a second request for information. They don't always do this so hopefully it means they see a potential problem. It also means time is fleeting so ACT NOW.

For more information, visit ftc.gov or view this PDF from their website that gives a nice overview.

API Heathcare's Acquisition by Kronos Remains Unpopular

My feelings about the pending sale of API Healthcare to Kronos is no secret. I think this is a horrible idea. As it turns out, I'm not the only one that thinks this. I have it on good authority that in addition to me, but the API employees AND customers are not in favor either!

Two and a half years ago when Francisco Partners bought API, there was a lot of uncertainty. The employees knew things were going to get shaken up. There was a shake up and really, I would say that the operation of the entire company  changed. The balance between making money, providing a quality product, and treating employees fairly shifted dramatically. I hate to say it but employees were on the losing end of that switch. Nonetheless, the core focus of the company remained steadfast and probably even got more precise. Because of that focus, the vast majority of employees remained loyal to the company. There were only a handful of employees that left or were let go.

Fast forward to 2011. Employee retention has become a big problem. I don't have exact number of departures but I'll let you draw your own conclusions but it's pretty obvious that there's a problem when the company has to resort to bribery. If you were a customer right now, how would you feel knowing employees are jumping ship?

Rumor has it that the clients aren't any more excited about this either. Why would they be? There are two major competitors in the market. They chose API over Kronos for a reason and now they're potentially being forced into using a product they don't want or with a company they don't want to work with.

If ever there was a time for the regulators to step in and say, "NO!!!" this is it. Competition is good for the market and for consumers. There is nothing good that could come of this acquisition. All Kronos wants is to eliminate their biggest competitor because that will translate to more money for them AND they won't have to work as hard to try to maintain some sort of edge in the market.

Who's Your Tester?

Check out my latest blog at Vodori.