Is There A Problem Here?

msn video

To use this product, you need to install free software

---

This product requires Microsoft Internet Explorer 6 with Microsoft Media Players 10 and Macromedia Flash 6 or higher versions, or Mozilla Firefox 1.5 with Macromedia Flash 8, or Safari 2.0.4 with Macromedia Flash 8. To download these free software applications, click the links below and follow the on-screen instructions.

Step 1: download firefox 1.5download firefox 1.5

Step 2: Download Macromedia Flash PlayerMacromedia Flash player is free to download.
If still having problems, uninstall Flash and then re-install Flash.

Once the installations are complete, reload this page.

The Bananananananana Principle

... as the little boy said, "Today we learned how to spell 'banana', but we didn't learn when to stop." ... In honor of that little boy, we can elevate his idea to a principle, The Banana Principle:

Heuristic devices don't tell you when to stop.

- Gerald M. Weinberg,
An Introduction to General Systems Thinking

I just had the following exchange with my 12 year old daughter Jessica.

Me: How do software testers know when to stop testing something?

Jessica: When you die! . . . Or when you get really tired of it.

---

The Banana Principle does not mean that heuristics cannot be useful in determining when to stop. It means that heuristics do not tell us when to stop using the heuristic. There is a tendency to start transforming the most useful heuristics into laws -- in our minds. Heuristics should help us think and not replace thinking. This includes continual questioning of even the most useful heuristics.

Finally, a tester certification test that I might like!

I am not a fan of any of the current software tester certification programs. Perhaps it is because I take the word certification too literally -- which means that I expect it to have real meaning. When I think of certification, I usually think in lines with the IEEE's definition of certification.

certification

The process of confirming that a system or component complies with its specified requirements and is acceptable for operational use.

Perhaps my thinking is biased by my past IV&V testing work. If one can go to a weekend class and become certified, then I question the value of that certification. I believe that certifications based on ability to memorize terms and practices free of context are of little value -- and may do more harm than good. Yet, the purpose of this blog post is not to provide arguments against the current crop of tester certification options. If you'd like to see some concerns about certification, read the following.

• Against Certification, James Bach
• AST Certification Debate, Tim Coulter
• Open Certification at ACM SIGCSE, Cem Kaner

If we must have a test certification based on a computer-scored multiple-false test, then I think I have found a test that is better than any others I've yet seen. This test covers many aptitudes and skills that I believe are important for testers:

• Precision reading
• Requirements interpretation
• Persistence
  
• Exploratory learning
• Domain knowledge
• Heuristic based problem solving
  
• A good sense of humor
• Critical analysis
• Looking at problems from many angles
• Recognizing context
• Understanding the software platform
• Agility
• Troubleshooting
• Working under pressure
• A good memory, or good note taking skills
• and don't be easily offended
  

Give it a try.

The Impossible Quiz

If I had to select testers based on passing a test, I think I'd take someone that has gotten further than I in this quiz over someone with a software tester certification. I believe this quiz is a better measure of whether or not one is "acceptable for operational use". :)

How far can you get?

And did you notice the bug on question 30?

Excuses, Excuses

I have heard a variety of responses from developers in response to bugs I report. Some are good, some bad, and some are just plain ugly. Here are a few handfuls.

• That's strange.
• How'd you do that?
  
• It works on my machine.
• I already fixed that. You'll have it in the next build.
  
• No user would do that.
• That's not how you're supposed to do that.
  
• It's a data problem. Tell the users to fix the data.
  
• That's a cool bug! Show me again.
  
• I didn't touch that code.
• It works as designed.
• It works as coded. [Well, duh. What else would it do?]
  
• That's not a bug, it's a feature.
• I can't test everything.
• Thank you.
  

... and my absolute favorite (this came from a development manager)

• Don't judge it, just test it.

The difference between the good and bad is often the relationship between developer and tester. Testers need developers to create something to test. Respect your developers. Communicate with respect and help turn the ugly responses into good responses.

As I've heard James Bach say: Testers don't create quality. Developers create quality.

What's your favorite bug report response?

Ugly Babies: Another Reason Why We Need Testers

... I got on the train ... And I noticed that the woman across from me in the aisle had her baby with her. Ugly baby. Ugly baby.

From the other end of the coach comes this guy and he was very drunk and he was staring at the baby. ... And the guy said, "I'm looking at that ugly baby. That's a horrible looking baby lady. Where'd you get that baby from?"

And the woman said, "I don't have to take that!" And she snatched the emergency cord and the train came to a screeching halt, and the conductor came running in.

Now this was his moment. At this moment he represented the Pennsylvania Railroad. And he said, "what's going on here?"

And the woman said, "This man just insulted me. I don't have to spend my money and ride this railroad and be insulted. I'd rather walk."

And the conductor said, "calm down! Calm down! Madame there's nothing, nothing that the Pennsylvania Railroad will not do to avoid having situations such as this. Perhaps it would be more to your convenience if we were to ... let you sit somewhere else in the coach. And as a small compensation from the railroad ... we are going to give you a free meal; and maybe we'll find a banana for your monkey."

- Flip Wilson


We are like parents when it comes to our creations: we can be blind to ugly.

I am a believer in developer testing. Good unit testing is essential to good software. Practices like Test Driven Development (TDD) help ensure that software works from the start. Anyone that's been in the software development business for long knows that the cost of defects exponentially grows as a project progresses. Preventing and finding defects early should be a goal in every software project. Early testing by developers does not remove the need for testing by someone other than the coder.

There are a number of reasons for "independent" testing. These reasons often include references to requirements testing, end-to-end testing, integration testing, system testing, and independent verification. These reasons focus on testing things as complete systems or from a user's viewpoint. I have another reason: Creators can be blind to ugly.

Creators have an emotional attachment to their creation. This attachment can get in the way of objective evaluation. It is difficult to be both creator and critic. It is as if creativity and critique are opposite sides of a scale. We can be both, but increasing one can harm the other. Creators need to balance the two without allowing self-criticism to hamper creativity.

I think its good for creators to have an emotional attachment to the product of their thoughts and labor. External critics can help a creator improve their creation without stifling creative thinking.

Creating and critiquing are separate skills. When one group can focus on one and another group can focus on the other: together they are stronger (if the critics are more tactful than the drunk in Flip's story).

At least that's my opinion. Now I open the floor to the critics. :)

Software Development Life Cycle Explained

A couple weeks back, there was some discussion on the software-testing group about the use of the term "SDLC" on resumes. (Matt Heusser posted some excerpts from this conversation here.) My warning flags go up when people claim to have a "full understanding" of the SDLC. I sometimes see this as an indication that someone may not be as experienced as they claim. The "SDLC" will vary from one company to another and even one project to another. SDLC is a process documentation term -- and there are many differing processes used to develop software. Its not how people talk in the real world.

I recently re-stumbled upon a description of the SDLC that seems to be fairly common across many companies and projects. Perhaps this is what people are referring to when they claim full knowledge of the SDLC. :)

Software doesn’t just appear on the shelves by magic. That program shrink-wrapped inside the box along with the indecipherable manual and 12-paragraph disclaimer notice actually came to you by way of an elaborate path, through the most rigid quality control on the planet. Here, shared for the first time with the general public, are the inside details of the program development cycle.

1. Programmer produces code he believes is bug-free.


2. Product is tested. 20 bugs are found.


3. Programmer fixes 10 of the bugs and explains to the testing department that the other 10 aren’t really bugs.


4. Testing department finds that five of the fixes didn’t work and discovers 15 new bugs.


5. See 3.


6. See 4.


7. See 5.


8. See 6.


9. See 7.


10. See 8.


11. Due to marketing pressure and an extremely pre-mature product announcement based on over-optimistic programming schedule, the product is released.


12. Users find 137 new bugs.


13. Original programmer, having cashed his royalty check, is nowhere to be found.


14. Newly-assembled programming team fixes almost all of the 137 bugs, but introduce 456 new ones.


15. Original programmer sends underpaid testing department a postcard from Fiji. Entire testing department quits.


16. Company is bought in a hostile takeover by competitor using profits from their latest release, which had 783 bugs.


17. New CEO is brought in by board of directors. He hires programmer to redo program from scratch.


18. Programmer produces code he believes is bug-free.

Even the above demonstrates some naivete.

Bugs are not necessarily the fault of a developer. Many bugs are defects in the requirements and design; not the code of any specific developer.

Developers rarely get royalty checks or bonuses.

I've never known of a "testing department" serving a single "developer". Now that's quite some tester to developer ratio.

Perhaps "SDLC" is just a term we use to model something that is really complex.

Installing Windows Vista

I have not yet made the jump to Windows Vista on any of my personal machines.

We have installed Vista on some machines in the test lab where I work. When we started playing with a Vista beta, Vista was not allowed on our network but our customers were using Vista; demanding that we test our products on Vista. There were numerous technical and political battles that had to be fought to get Vista installed in our test lab.

We quickly discovered that Microsoft's minimum system requirements are just the minimum to install the OS. I can't imagine any user being happy with Vista on a machine that just met the minimum requirements.

We encountered incompatible DVD drives from a major PC manufacturer. We followed Microsoft's instructions for copying the DVD to a hard drive for installation from the hard drive instead of the DVD only to have the Vista install inform us that it cannot be installed as Microsoft instructed.

We've had to call Microsoft for permission to reinstall failed installations.

I have decided that I do not need this trouble at home. I have not yet seen any feature worth the upgrade. I would rather not have to buy new hardware. For now, I am going to stick with Windows 2000, Windows XP, and Linux.

For those that really want to make the leap, I suggest you watch the following video to see what kind of machine is compatible with Vista ... or is it?


Vista install in 2 minutes

Please let me know how your install goes.

Free Internet: Flushing The Web

Although we understand that there's a lot of crap on the web, we also believe strongly in providing equal opportunity access to all our users.

- Google

The folks at Google are working hard to provide innovative services to the world. A couple months ago, they announced the Beta release of their free in-home wireless broadband service: Google TiSP.

Google TiSP promises free, fast, and easy to install wireless broadband service. Sorry, this services is only available in the United States and Canada.

Check it out here.

What is TiSP?

Toilet Internet Service Provider

When things go wrong with TiSP, they go very, very wrong. Let's leave it at that.

Happy Flushing!

Now where'd I put my WiiHelm?

ERROR!

Sometimes you just gotta read the instructions.

Here's a short student film directed by a friend.


Error!

What's the message here? Which character do you relate to?

I think I relate to the person that broke the computer in the first place. I could'a done that.

Coffee Break Machine Testing

It's good to have some idea as to what something does before you start testing it -- or eating it.

Cookie Monster does a little testing in a 1967 IBM training film.


Thanks to UtterlyGeek.

What do you call this kind of testing?