Overview

This page is the active-recall companion to the Software Testing learning track: four fixed drills that force retrieval instead of passive re-reading. Work through them in order -- short-answer recall first, then scenario judgment, then hands-on implementation, then a checklist to confirm real automaticity. Every answer is hidden in a <details> block; try each item yourself, out loud or on paper, before opening it. Every prompt below uses its own mocked, self-contained input -- different function names, domains, and scenarios from anything in the learning track's 86 worked examples -- so recognizing an answer isn't enough; you have to actually reconstruct the reasoning. Nothing on this page touches a live third-party host, a real database, or the network: every scenario is entirely self-contained, in memory, reasoned about on paper.

Recall Q&A

Thirty-two short-answer questions, one per concept (co-01 through co-32). Answer from memory, then check.

Q1 (co-01 -- why-test-and-aaa). In one sentence, what does a test actually encode, and name the three phases arrange-act-assert structures it into.

Q2 (co-02 -- test-discovery-and-run). A file is named helpers_test.py and contains a function named check_helper(). Will plain pytest (no arguments) discover and run it? Why or why not?

Q3 (co-03 -- assertions). Why does pytest not need a assertEqual/assertTrue/assertIn method zoo the way some other testing frameworks do?

Q4 (co-04 -- exception-testing). Write, from memory, the general shape of a pytest.raises block that asserts a call raises a KeyError whose message contains the text "missing".

with pytest.raises(KeyError, match="missing"):
    do_the_call_that_should_raise()

Q5 (co-05 -- fixtures). A fixture is decorated @pytest.fixture and contains a single yield statement instead of a return. What does the code AFTER that yield do, and when does it run?

Q6 (co-06 -- parametrization). @pytest.mark.parametrize("n,expected", [(1, 1), (2, 4), (3, 9)]) decorates one test function. How many times does pytest actually run that function's body, and how are the runs reported?

Q7 (co-07 -- approx-and-floats). Why does 0.1 + 0.1 + 0.1 == 0.3 evaluate to False in Python, and what pytest helper fixes an assertion built on that comparison?

Q8 (co-08 -- markers-and-selection). What is the practical difference between @pytest.mark.skip and @pytest.mark.xfail on a test that currently fails?

Q9 (co-09 -- test-organization). A fixture is defined once in conftest.py at the root of a test directory. Which test files can use it, and does any file need to import it?

Q10 (co-10 -- test-pyramid-vs-trophy). A team has 500 unit tests, 40 integration tests, and 5 e2e tests. Is this shaped more like the pyramid or the testing trophy? What would make it look like the other one?

Q11 (co-11 -- test-doubles-taxonomy). Name all five kinds of test double in the Meszaros taxonomy, and state the one dimension that separates mocks from every other kind.

Q12 (co-12 -- stubbing). A test replaces a PaymentGateway collaborator with an object whose charge() method always returns {"status": "approved"}}, no matter what arguments it receives. What kind of double is this, and what is it NOT doing?

Q13 (co-13 -- mocking-and-verification). What does mock.assert_called_once_with(x, y) actually check, distinct from just checking the unit's return value?

Q14 (co-14 -- patching). monkeypatch.setattr("mymodule.send_email", fake_send) is called inside a test. When does the real send_email function get restored, and why doesn't the test need to restore it manually?

Q15 (co-15 -- spies). A test wraps a real Logger object with MagicMock(wraps=real_logger). After calling the unit under test, the assertion checks spy.info.call_count == 2. Did the real logger's info() method actually execute? How do you know?

Q16 (co-16 -- fakes). An InMemoryUserRepository implements the exact same save()/find_by_id() interface as the real database-backed repository, using a plain Python dict internally. Why is this a fake and not a stub?

Q17 (co-17 -- tdd-red-green-refactor). Put these three TDD steps in the correct order: "refactor while staying green," "write a failing test," "make the test pass with the simplest code that works."

Q18 (co-18 -- property-based-testing). How does a property-based test (Hypothesis/fast-check) differ from a parametrized test (co-06) in HOW it chooses its inputs?

Q19 (co-19 -- shrinking). A Hypothesis test fails on a randomly generated list of 340 integers. What does Hypothesis do next, automatically, before reporting the failure to you?

Q20 (co-20 -- strategies). What does st.integers(min_value=0, max_value=100) describe, and what does calling assume(x % 2 == 0) inside a property test do to the values Hypothesis generates?

Q21 (co-21 -- coverage). A function has 100% line coverage from its test suite. Does that prove the function is correct? What does 100% line coverage actually guarantee?

Q22 (co-22 -- mutation-testing). A mutation tool changes if x > 0: to if x >= 0: in your source, reruns your test suite, and the suite still passes. What does this "surviving mutant" tell you, and what should you do about it?

Q23 (co-23 -- integration-testing). A unit test replaces a database with an in-memory fake. What would need to change to turn that same test into an integration test?

Q24 (co-24 -- contract-testing). Two teams own a consumer service and a provider service that talk over an API neither team can fully integration-test together on every commit. What does contract testing let each team verify independently, without running the other team's service?

Q25 (co-25 -- e2e-and-test-containers). What does an end-to-end test drive that an integration test typically does not, and what problem do test-containers solve for tests that need a real dependency like a database?

Q26 (co-26 -- test-isolation-and-determinism). A test suite passes when run alone but fails when run as part of the full suite in a certain order. What category of problem is this, and name two specific sources of nondeterminism this topic teaches you to control.

Q27 (co-27 -- reading-reports). A pytest run ends with the line 3 failed, 47 passed, 2 skipped in 1.84s. Before opening any traceback, what does this summary line alone already tell you?

Q28 (co-28 -- bdd-given-when-then). Rewrite this unit-test-style description as a Given/When/Then scenario: "test that withdrawing more than the balance raises an error." Name the audience BDD's ubiquitous language is written for that a raw unit test name usually isn't.

Q29 (co-29 -- gherkin-feature-scenario). Name the four keywords, in order, that structure a single Gherkin scenario body (not counting the Feature/Scenario lines themselves), and what does And do.

Q30 (co-30 -- step-definitions). A .feature file contains the step Given a cart with 3 items. What does a step definition do with this line, and name the two Python tools this topic teaches that can run it.

Q31 (co-31 -- scenario-outline-examples). A Scenario Outline has three placeholder variables and an Examples table with 5 data rows underneath it. How many times does the scenario actually run, and what is this the BDD analog of from the pytest side of this topic?

Q32 (co-32 -- bdd-vs-tdd-and-atdd). A change adds a purely internal caching helper with no user-visible behavior. A separate change adds a new customer-facing checkout discount rule. Which of these two changes more naturally earns a BDD acceptance scenario over a plain unit test, and why?

Applied problems

Fifteen scenarios spanning beginner through advanced material. Each describes a realistic engineering situation without naming the specific tool or technique -- decide what applies and why, then check your reasoning against the worked solution. Every scenario below is invented for this drill and does not reuse any function, fixture, or domain from the 86 worked examples.

AP1. A test named test_discount() asserts apply_discount(100, 0.1) == 90. A teammate asks "how do I know which part of this test is the setup versus the check?" What's your one-sentence answer, and how would you restructure the test to make the answer obvious just from reading it?

AP2. A new hire runs bare pytest in a project and sees collected 0 items, even though they can see test functions in the file they're looking at. What are the two most likely naming mistakes causing this?

AP3. A test asserts compute_total(cart) == 42.30 and fails intermittently with values like 42.299999999999997. What's the root cause, and the one-line fix?

AP4. A suite has 40 tests that hit a real external payment API and take 6 minutes to run, so engineers stopped running them locally before pushing. What two concepts from this topic would you reach for first to fix this, and in what order?

AP5. Two tests both call a shared shopping_cart fixture. Test A adds an item and asserts the cart has 1 item; Test B asserts the cart is empty. Run alone, both pass. Run together, Test B sometimes fails. What's the fixture's scope almost certainly set to, and what should it be instead?

AP6. A code review comment says "this test verifies the mock was called, but never checks what the function actually returned to its caller." What kind of double is likely being misused here, and what should a more complete test add?

AP7. A test needs os.environ["FEATURE_FLAG"] to read "on" for the duration of one test only, without affecting any other test in the suite, and without a try/finally block. What's the exact mechanism?

AP8. A property test asserts sorted(reverse_list(xs)) == sorted(xs) for any list xs of integers, generated by Hypothesis. What TWO distinct properties is a well-written version of this test likely also checking about reverse_list, beyond just "the same elements come back"?

AP9. A Hypothesis test that asserts parse(render(value)) == value fails on a huge, oddly-shaped generated value. Before even reading the failure, what should you expect Hypothesis to have already done to that failing input, and why does that matter for how quickly you can debug it?

AP10. A function has 100% branch coverage, and the whole suite is green. A bug ships anyway. What class of testing gap does this scenario most directly illustrate, and which technique in this topic is specifically designed to expose it?

AP11. A team wants to verify their order-service and their shipping-service agree on the shape of the "order placed" event, but shipping-service's team is in a different timezone and the two services are never deployed together in CI. What testing technique fits this constraint directly, and why not just write an end-to-end test instead?

AP12. A test spins up a real, ephemeral Postgres instance via a container library, runs a write then a read against it, then tears the container down -- all inside one test function. Which two concepts does this single test combine, and what's the key difference from a test that uses an in-memory fake repository instead?

AP13. A suite passes locally every time but fails roughly one run in twenty on CI, always on a different test. What's the FIRST thing this topic teaches you to suspect, before assuming CI's hardware is simply slower?

AP14. A product manager reads a .feature file and says "I understand exactly what this scenario checks, but I have no idea which function in the codebase actually implements it." Is that a defect in the .feature file? Why or why not?

AP15. A team is deciding whether a newly proposed regression (a subtle off-by-one bug reported by a single internal power user, in a rarely-used admin CSV export) should get a full BDD acceptance scenario or a targeted unit test. Which fits better, and what's the one-sentence judgment call this topic teaches for making this decision generally?

Code katas

Eight self-contained exercises. Every kata uses in-memory data and stdlib-only dependencies -- no live network host, no real database, no Docker container -- so every kata is runnable anywhere this topic's pinned toolchain (pytest 9.1.1, Hypothesis 6.156.6) is installed, with nothing else to set up.

Kata 1 -- Write a failing test first, for a function that doesn't exist yet

Write test_slugify() asserting slugify("Hello World!") == "hello-world" BEFORE writing slugify at all. Run it and confirm it fails with an ImportError or NameError (co-17, co-23's red step). Then implement the simplest slugify that makes it pass, then refactor it to handle repeated whitespace and leading/trailing punctuation while the test stays green throughout.

Kata 2 -- Parametrize a boundary-heavy function

Write a classify_grade(score: int) -> str returning "F"/"D"/"C"/"B"/"A" for standard letter-grade cutoffs. Write ONE parametrized test (co-06) covering every boundary value (59, 60, 69, 70, ... 100) rather than five separate test functions -- confirm each row shows its own id in -v output.

Kata 3 -- Build a stub, then a mock, for the same collaborator

Write a NotificationService that depends on an injected sender object with a send(to, body) method. Write one test using a STUB sender (co-12, returns a canned success value, assertion checks the service's own return value) and a second test using a MOCK sender (co-13, assertion checks sender.send.assert_called_once_with(...)) for the same underlying behavior -- confirm both pass while asserting genuinely different things.

Kata 4 -- A property test that catches a seeded bug

Write dedupe(xs: list[int]) -> list[int] with a deliberately planted bug (e.g. it also silently drops legitimate duplicates of the value 0). Write a Hypothesis property test (co-18) asserting len(set(dedupe(xs))) == len(set(xs)) over st.lists(st.integers()) -- confirm it fails, read the shrunk counterexample (co-19), fix the bug, and confirm the same test then passes.

Kata 5 -- Freeze time and seed randomness in one fixture

Write a function greeting_id() -> str that combines the current hour and a random 4-digit suffix. Write a fixture that both freezes the clock (monkeypatch, co-14) and seeds the RNG (co-26) so two separate test runs of the same test produce the IDENTICAL output -- confirm by running the test twice and diffing the captured output.

Kata 6 -- Deliberately plant a surviving mutant, then kill it

Write a is_adult(age: int) -> bool returning age >= 18. Write ONE test that only checks age = 25 and age = 10. Manually mutate the source to age > 18 (simulating what mutmut would try, co-22) and confirm your existing test suite still passes against the mutant -- then add a boundary test for age == 18 and confirm the mutant is now caught.

Kata 7 -- Diagnose and fix an order-dependent test pair

Write two tests sharing a module-scoped fixture that returns a mutable dict. Test A mutates it; Test B asserts on its original state. Run them in both orders (pytest test_x.py::test_a test_x.py::test_b vs. reversed) and observe the order-dependent failure (co-26) -- then fix it by narrowing the fixture's scope to function and confirm both orders now pass identically.

Kata 8 -- Write one Gherkin scenario and bind it two ways

Write a .feature file with one Given/When/Then scenario for a simple Stack (push, then pop returns the pushed value). Bind it once with pytest-bdd step definitions and once with behave step definitions (co-30) against the SAME .feature file -- confirm both runners report the identical scenario passing.

Self-check checklist

Confirm each item without checking the learning track first. If you hesitate, that concept needs another pass.

• I can name the three phases of arrange-act-assert and explain what each one is responsible for. (co-01)
• I can state pytest's exact file- and function-naming discovery convention from memory. (co-02)
• I can explain why pytest doesn't need a separate assertEqual/assertTrue method zoo. (co-03)
• I can write a pytest.raises(..., match=...) block from memory, including what match actually checks. (co-04)
• I can explain what code placed after a fixture's yield does and when it runs. (co-05)
• I can state how many times a @pytest.mark.parametrized test body runs for N rows, and how those runs are reported. (co-06)
• I can explain why 0.1 + 0.2 == 0.3 is False in Python and name the pytest helper that fixes an assertion built on it. (co-07)
• I can explain the practical difference between @pytest.mark.skip and @pytest.mark.xfail. (co-08)
• I can explain how a conftest.py fixture becomes available to test files without any import. (co-09)
• I can describe the pyramid and the testing trophy and state which tier each one weights most heavily. (co-10)
• I can name all five kinds of test double in the Meszaros taxonomy and the one dimension that separates mocks from the rest. (co-11)
• I can explain what a stub does and does NOT do, distinct from a mock. (co-12)
• I can explain what assert_called_once_with(...) verifies, distinct from a return-value assertion. (co-13)
• I can explain when monkeypatch's patches get restored, and why no manual teardown is needed. (co-14)
• I can explain what makes a spy different from a mock, in terms of whether the real behavior still executes. (co-15)
• I can explain what makes a fake different from a stub, in terms of whether it's a genuinely working implementation. (co-16)
• I can state TDD's three steps in the correct order and explain why the test must fail first, for a real reason. (co-17)
• I can explain how a property-based test chooses its inputs differently from a parametrized test. (co-18)
• I can explain what shrinking does to a failing generated input, and why it matters for debuggability. (co-19)
• I can explain what a Hypothesis strategy describes, and what assume() does to generated values that fail its condition. (co-20)
• I can explain what 100% line coverage actually guarantees and what it does NOT guarantee. (co-21)
• I can explain what a "surviving mutant" tells you about your test suite, and what action it calls for. (co-22)
• I can explain what distinguishes an integration test from a unit test that stubs the same seam. (co-23)
• I can explain what contract testing lets two independently-deployed services verify without running together. (co-24)
• I can explain what an end-to-end test drives that a typical integration test doesn't, and what problem test-containers solve. (co-25)
• I can name at least three distinct sources of test nondeterminism this topic teaches you to control. (co-26)
• I can read a one-line pytest summary (N failed, M passed, K skipped) and state what each count means before opening any traceback. (co-27)
• I can rewrite a plain test description as a Given/When/Then scenario and name BDD's intended dual audience. (co-28)
• I can name Gherkin's four step keywords in order and explain what And does relative to the keyword before it. (co-29)
• I can explain what a step definition does and name the two Python BDD runners this topic teaches. (co-30)
• I can explain how many times a Scenario Outline with an Examples table of N rows actually runs, and its pytest-side analog. (co-31)
• I can state, in one sentence, the judgment call for choosing a BDD acceptance scenario over a plain unit test for a given change. (co-32)
• I can explain what correctness-vs-pragmatism means for this topic -- name one technique that exposes it directly, and why 100% of any single metric (coverage, or an all-green suite) is never by itself proof of correctness. (correctness-vs-pragmatism)

← Previous: Capstone