Floating Point Quality: Less Floaty, More Pointed (via James Bach’s Blog)

Years ago I sat next to the Numerics Test Team at Apple Computer. I teased them one day about how they had it easy: no user interface to worry about; a stateless world; perfectly predictable outcomes. The test lead just heaved a sigh and launched into a rant about how numerics testing is actually rather complicated and brimming with unexpected ambiguities. Apparently, there are many ways to interpret the IEEE floating point standard and learned people are not in agreement about how to do it. Implementing floating point arithmetic on a digital platform is a matter of tradeoffs between accuracy

Continue Reading

Deeper Testing (2): Automating the Testing (via Developsense Blog)

Here’s an easy-to-remember little substitution that you can perform when someone suggests “automating the testing”: “Automate the evaluation and learning and exploration and experimentation and modeling and studying of the specs and observation of the product and inference-drawing and questioning and risk assessment and prioritization and coverage analysis and pattern recognition and decision making and design of the test lab and preparation of the test lab and sensemaking and test code development and tool selection […]

Continue Reading

Deeper Testing (1): Verify and Challenge (via Developsense Blog)

What does it mean to do deeper testing? In Rapid Software Testing, James Bach and I say: Testing is deep to the degree that it has a probability of finding rare, subtle, or hidden problems that matter. Deep testing requires substantial skill, effort, preparation, time, or tooling, and reliably and comprehensively fulfills its mission. By contrast, shallow testing does not require much skill, effort, preparation, time, or tooling, and cannot reliably and comprehensively fulfill its […]

Continue Reading

The Test Case Is Not The Test (via Developsense Blog)

A test case is not a test. A recipe is not cooking. An itinerary is not a trip. A score is not a musical performance, and a file of PowerPoint slides is not a conference talk. All of the former things are artifacts; explicit representations. The latter things are human performances. When the former things are used without tacit knowledge and skill, the performance is unlikely to go well. And with tacit knowledge and skill, […]

Continue Reading

Drop the Crutches (via Developsense Blog)

This post is adapted from a recent blast of tweets. You may find answers to some of your questions in the links; as usual, questions and comments are welcome. I had a fun chat with a client/colleague yesterday. He proposed—and I agreed—that test cases are like crutches. I added that the crutches are regularly foisted on people who weren’t limping to start with. It’s as though before the soccer game begins, we hand all the […]

Continue Reading

Very Short Blog Posts (31): Ambiguous or Incomplete Requirements (via Developsense Blog)

This question came up the other day in a public forum, as it does every now and again: “Should you test against ambiguous/incomplete requirements?” My answer is Yes, you should. In fact, you must, because all requirements documents are to some degree ambiguous or incomplete. And in fact, all requirements are to some degree ambiguous and incomplete. And that is an important reason why we test: to help discover how the product is inconsistent with […]

Continue Reading

Rethinking Equivalence Class Partitioning, Part 1 (via James Bach’s Blog)

Wikipedia’s article on equivalence class partitioning (ECP) is a great example of the poor thinking and teaching and writing that often passes for wisdom in the testing field. It’s narrow and misleading, serving to imply that testing is some little game we play with our software, rather than an open investigation of a complex phenomenon. (No, I’m not going to edit that article. I don’t find it fun or rewarding to offer my expertise in […]

Continue Reading

Very Short Blog Posts (30): Checking and Measuring Quality (via Developsense Blog)

This is an expansion of some recent tweets. Do automated tests (in the RST namespace, checks) measure the quality of your product, as people sometimes suggest? First, the check is automated; the test is not. You are performing a test, and you use a check—or many checks—inside the test. The machinery may press the buttons and return a bit, but that’s not the test. For it to be a test, you must prepare the check […]

Continue Reading

Accountability for What you Say is Dangerous and That’s Okay (via James Bach’s Blog)

[Note: I offered Maaret Pyhäjärvi the right to review this post and suggest edits to it before I published it. She declined.] A few days ago I was keynoting at the New Testing Conference, in New York City, and I used a slide that has offended some people on Twitter. This blog post is intended to explore that and hopefully improve the chances that if you think I’m a bad guy, you are thinking that […]

Continue Reading

How Michael Bolton and I Collaborate on Articles (via James Bach’s Blog)

(Someone posted a question on Quora asking how Michael and I write articles together. This is the answer I gave, there.) It begins with time. We take our time. We rarely write on a deadline, except for fun, self-imposed deadlines that we can change if we really want to. For Michael and I, the quality of our writing always dominates over any other consideration. Next is our commitment to each other. Neither one of us […]

Continue Reading