Six Things That Go Wrong With Discussions About Testing (via James Bach’s Blog)

Talking about software testing is not easy. It’s not natural! Testing is a “meta” activity. It’s not just a task, but a task that generates new tasks (by finding bugs that should be fixed or finding new risks that must be examined). It’s a task that can never be “completed” yet must get “done.”

Confusion about testing leads to ineffective conversations that focus on unimportant issues while ignoring the things that matter. Here are some specific ways that testing conversations fail:

  1. When people care about how many test cases they have instead of what their testing actually does. The
Continue Reading

Regression Test Tool for Trash Walking (via James Bach’s Blog)

My recent flirtation with trash-pickup-as-physical-exercise has led me down a familiar path. Even though it is not my responsibility to clean a public road in the first place, once I do it, I find that I feel irrational ownership of it. I want it to stay clean. But since I’ve adopted about 9 miles of road so far, it takes too long to walk the whole route in a day (remember I have to make […]

Continue Reading

Test Coverage Parallels in Trash Walking (via James Bach’s Blog)

First, about scope… As I began my trash walking (see here and here), I quickly found that I needed guidelines on what counts as my work space and work product. I am collecting trash along the road, so what does that entail? Here is what I came up with: I began with a broad operational definition of trash: “any loose, inanimate object of low value that may disturb the tranquility of the touring experience.” This […]

Continue Reading

Test Talk About Trash Walks (via James Bach’s Blog)

So, for exercise, I’m picking up trash. Here is a picture of me all kitted up: Perfectly equipped for road trash collection! So far, I’ve done 37 miles of trash collecting. And I can’t help but see some interesting parallels with software testing… Just Like Testing #1: I can use tools, but I cannot automate my work. I have to make a lot of judgments about what to pick up and what to leave. It […]

Continue Reading

The Unnecessary Tool (via James Bach’s Blog)

My wife bought a Steel Grip 36in Lightweight Aluminum Pick Up Tool. I saw it on our combination dining room/craft/office table and asked her what it was for. “My eye pillow fell behind the bed and I can’t reach it.” she told me. (This led to some confusion for me at first because I thought she was referring to an iPillow, presumably an Apple product I had never heard of.) “I can easily get that […]

Continue Reading

Floating Point Quality: Less Floaty, More Pointed (via James Bach’s Blog)

Years ago I sat next to the Numerics Test Team at Apple Computer. I teased them one day about how they had it easy: no user interface to worry about; a stateless world; perfectly predictable outcomes. The test lead just heaved a sigh and launched into a rant about how numerics testing is actually rather complicated and brimming with unexpected ambiguities. Apparently, there are many ways to interpret the IEEE floating point standard and learned […]

Continue Reading

Deeper Testing (2): Automating the Testing (via Developsense Blog)

Here’s an easy-to-remember little substitution that you can perform when someone suggests “automating the testing”: “Automate the evaluation and learning and exploration and experimentation and modeling and studying of the specs and observation of the product and inference-drawing and questioning and risk assessment and prioritization and coverage analysis and pattern recognition and decision making and design of the test lab and preparation of the test lab and sensemaking and test code development and tool selection […]

Continue Reading

Deeper Testing (1): Verify and Challenge (via Developsense Blog)

What does it mean to do deeper testing? In Rapid Software Testing, James Bach and I say: Testing is deep to the degree that it has a probability of finding rare, subtle, or hidden problems that matter. Deep testing requires substantial skill, effort, preparation, time, or tooling, and reliably and comprehensively fulfills its mission. By contrast, shallow testing does not require much skill, effort, preparation, time, or tooling, and cannot reliably and comprehensively fulfill its […]

Continue Reading

The Test Case Is Not The Test (via Developsense Blog)

A test case is not a test. A recipe is not cooking. An itinerary is not a trip. A score is not a musical performance, and a file of PowerPoint slides is not a conference talk. All of the former things are artifacts; explicit representations. The latter things are human performances. When the former things are used without tacit knowledge and skill, the performance is unlikely to go well. And with tacit knowledge and skill, […]

Continue Reading

Drop the Crutches (via Developsense Blog)

This post is adapted from a recent blast of tweets. You may find answers to some of your questions in the links; as usual, questions and comments are welcome. I had a fun chat with a client/colleague yesterday. He proposed—and I agreed—that test cases are like crutches. I added that the crutches are regularly foisted on people who weren’t limping to start with. It’s as though before the soccer game begins, we hand all the […]

Continue Reading