{ } Programming

Systematic debugging: finding bugs with the scientific method

A concrete debugging method: reproduce the problem, form hypotheses, shrink the case, and use debuggers and bisection instead of trial and error.

Faced with a bug, instinct almost always leads in the same direction: change something at random, rerun, hope. Sometimes it works, and that is exactly why the habit persists. But when the problem is serious, trial and error can cost days. Effective debugging is not a matter of brilliant intuition: it is a method, very similar to the scientific one, that anyone can learn and apply.

Step one: reproduce the problem

A bug you cannot reproduce is a bug you cannot fix with certainty. Before touching the code, try to answer precise questions:

  • what exactly are the steps that lead to the error?
  • does it happen always or only sometimes?
  • does it happen with all data or only with some?
  • does it happen in every environment or only in one (production, a certain browser, a certain operating system)?

The goal is a reliable reproduction, ideally a command or automated test that fails every time. Once you have it, you have already done half the work.

Actually reading the error message

It sounds trivial, but many bugs are hunted for hours while the answer is written in the error message. Read it in full, calmly: exception type, message, file and line number. If there is a stack trace, find the first line that belongs to your code and not to a library: that is often where the problem originates.

Forming hypotheses and testing them

The heart of the method is a simple cycle:

  1. observe the wrong behaviour;
  2. form a precise hypothesis about the cause ("the total is wrong because the discount is applied twice");
  3. design a check that confirms or refutes the hypothesis;
  4. run the check and accept the result, even if it refutes your favourite idea.

The difference from trial and error is subtle but decisive: every attempt teaches you something. Even a refuted hypothesis narrows the field. Writing down the hypotheses you have already ruled out avoids repeating the same checks and helps you reason when the problem gets complicated.

Shrinking the case

A bug that shows up in a complex system is hard to analyse. It pays to shrink it progressively to the minimal case that reproduces it:

  • if the error happens with a ten-thousand-line file, try with half, then a quarter;
  • if it involves a page with twenty components, remove them one at a time;
  • if it depends on a sequence of operations, drop the unnecessary ones.

Often, while you shrink the case, the cause becomes obvious on its own. And if it does not, the minimal case is perfect for sharing with a colleague or attaching to a bug report.

The tools

Tool When to use it
Prints and logs Quick checks of a variable's value at a precise point
Debugger Following execution step by step and inspecting the full state
Conditional breakpoints Stopping only when a condition is true, for example at iteration 500
Automated tests Pinning down the reproduction and verifying the fix
git bisect Finding the commit that introduced a regression

The debugger deserves an investment of time. In Python, all you need is to put breakpoint() somewhere in the code to pause execution and inspect variables, evaluate expressions and step line by line. Modern editors offer the same feature with a graphical interface.

Bisection

When you know that an earlier version worked and the current one does not, git bisect automates a binary search through the history:

git bisect start
git bisect bad                 # the current version has the bug
git bisect good v2.1.0         # this version worked
# Git proposes an intermediate commit: test it and answer
git bisect good   # or: git bisect bad
# ... repeat until Git names the guilty commit
git bisect reset

With a thousand commits in between, about ten checks are enough to find the responsible one. If you have a test that reproduces the bug, you can even hand the whole job to git bisect run.

Explaining the problem out loud

A surprisingly effective technique is describing the problem, line by line, to someone, even an object on your desk: this is so-called rubber duck debugging, named after the rubber duck used as a listener. Forcing yourself to explain what each line should do often reveals the point where reality diverges from expectations.

After the fix

Once you have found the cause, the work is not over:

  1. write a test that fails without the fix and passes with it;
  2. ask yourself whether the same error might exist elsewhere in the code;
  3. check that you fixed the cause and not just the symptom;
  4. in the commit message, explain what was causing the bug.

In short

  • Before fixing, get a reliable reproduction of the problem.
  • Read the error message and stack trace in full.
  • Work through testable hypotheses, not random attempts, and note what you have ruled out.
  • Shrink the case to the minimum and use debuggers and git bisect to narrow the search.
  • Close every fix with a test that stops the bug from coming back.
MM
Marco Marzaduri

Marco Marzaduri works in cybersecurity, digital forensics and web development. With degrees in computer science and psychology, he combines technical analysis with an understanding of the human factor.

Read also