Test Case Design with the Classification Tree Method: Deriving Unit Tests You Can Defend in an Audit

Where Do Your Test Cases Actually Come From?

Ask a team how it arrived at a given set of unit test cases and you get one of two answers. The good answer traces each test case to a requirement and to a stated reason for choosing that input. The common answer is some version of “the engineer who wrote the module knew what to try.”

The second answer fails an audit, not because the tests are bad but because no argument is attached to them. The Classification Tree Method (CTM) replaces that missing argument with a visible one. Razorcat, the Berlin developer of TESSY, describes the objective of the CTM as transforming “a (functional) definition of a problem systematically into a set of error-sensitive, low redundancy set of test case specifications.” Per Razorcat, the method stems from the former software research laboratory of Daimler in Berlin.

The Method in Three Moves

Razorcat’s Classification Tree Editor page reduces the CTM to two steps: identify the test-relevant aspects (classifications) and their values (classes), then combine classes from all classifications into test cases. It helps to separate partitioning from combining, so treat it as three moves.

1. Identify the classifications

A classification is an aspect you expect to influence the behaviour of the test object. Per Razorcat’s manual, an aspect is relevant “if the user wants to use different values for this aspect during testing.” Aspects come from the functional specification, not the source code, which keeps CTM a black-box technique.

2. Partition each classification into classes

Values of each aspect are grouped into equivalence classes. Razorcat’s manual states the rule plainly: values are equivalent for the test if, when one value from a class causes a test to fail and reveal an error, “every other value out of this class will also cause the same test to fail and will reveal the same error.” The ISTQB Certified Tester Foundation Level syllabus states the same principle, adding that partitions “must not overlap and must be non-empty sets.”

The manual imposes two obligations that are easy to skip and expensive to skip:

• Completeness: every possible value of a test-relevant aspect must be assigned to a class.
• Uniqueness: a value must be assigned to a single class, not to several.

Classes can be partitioned again. Per Razorcat, repeated partitioning reduces complexity, because each class is then considered in isolation, and it documents the stages of reasoning, which “serves understandability and traceability of the result.” Boundary values enter here: boundary value analysis is the opposite idea to equivalence partitioning, and CTM expresses both by carving out a class holding one extreme value.

3. Combine classes in the combination table

Leaf classes form the head of a combination table. Each row is one test case, defined by marking which leaf classes supply its values. Per Razorcat, exactly one class of each classification is considered per test case, and the combined classes must be logically compatible.

The discipline lies in what you do not do. Combining every class with every other is not the intention of the method, Razorcat’s manual says: that “could be done automatically by a tool” and produces loss of overview and excessive effort. The objective is “a minimal, non-redundant but sufficient set of test cases.”

A Worked Example: An Ice Warning Indicator

Razorcat’s manual works a small automotive example that shows the reasoning well. The specification: a dashboard ice warning depends on an outside temperature sensor reporting from -60 °C to +80 °C. Above 3 °C the warning shall be off; at lower temperatures it shall be on.

Move 1. One input, so one base classification: sensor temperature, which Razorcat notes is obviously the only test-relevant aspect here.

Move 2, first cut. Ask whether invalid values are possible before splitting on the threshold. Razorcat’s manual points at the physical failure modes: a short circuit or a cable interruption produces a value outside the sensor’s range. So the first partition is valid versus invalid, and invalid splits into too low and too high.

Move 2, the trap. It is tempting, Razorcat writes, to form two classes from the valid temperatures: -60 °C to 3 °C where the warning is on, 3 °C to 80 °C where it is off. GSAS note: apply Razorcat’s own uniqueness rule to that partition and 3 °C sits in both classes, because the specification says the warning is off above 3 °C and on at lower temperatures, and never says what happens at exactly 3 °C. Razorcat makes the general point elsewhere in the manual, in a separate worked example: finding an omission or contradiction in the functional specification during test case design is a valuable result, and “it is generally more likely to detect a problem in the functional specification if the test case specification is systematic.”

Move 2, repeated partitioning. Razorcat then splits the warning-on class by sign of temperature, since one arbitrary value per class is otherwise enough and a tester could satisfy it with 2 °C forever.

Move 3. Marking leaf classes in the combination table yields a test case specification.

The table below is GSAS’s own combination table, extending Razorcat’s ice-warning example with boundary classes. Razorcat’s published partitioning for this example yields at least four test cases; the eight below add the boundaries and the threshold row.

Source for the specification and the partitioning approach: Razorcat, TESSY 5.1 User Manual, section 3.2. Test case selection is ours.

Eight test cases from one input variable, each with a written reason. Row 5 pays for the exercise.

Razorcat also gives two bounds. In the manual’s larger is_value_in_range example the minimum criterion (each leaf class used once) is 7, the maximum criterion (all permitted combinations) is 105, and the worked specification lists 14 test cases against 15 leaf classes, matching Razorcat’s rule of thumb that leaf class count sets the order of magnitude.

CTM Versus Ad-Hoc Test Invention

Where This Meets the Standards

The obligation to test. Razorcat’s manual summarises the IEC 61508 position: Part 3 requires module testing to show the module performs its intended function and does not perform unintended functions, with results documented. Per the manual, functional and black-box testing is highly recommended from SIL 1 upward, and dynamic analysis and testing is recommended at SIL 1 and highly recommended at SIL 2 and above. CTM is a black-box, specification-based technique, the family those tables point to. See our IEC 61508 page.

The automotive table. ISO 26262-6:2018, Table 8, “Methods for deriving test cases for software unit testing”, has four rows: 1a analysis of requirements, 1b generation and analysis of equivalence classes, 1c analysis of boundary values, and 1d error guessing. CTM is a worked instance of row 1b, and because it carries boundary classes inside the same tree it produces row 1c evidence in the same artefact. Razorcat certifies TESSY’s core workflow to ISO 26262-08:2018, and the test cases that workflow executes are the ones the tree produces. Our ISO 26262 page sets out the full table and the ASIL ratings.

The status of the tool. Razorcat states that TESSY’s core workflow, and its release and test process, have been certified according to ISO 26262-08:2018 and IEC 61508-3:2010, with re-certification of TESSY 4.1 by TÜV SÜD Rail GmbH extending scope to EN 50128 and IEC 62304, and that TESSY was classified a T2 offline tool under EN 50128:2011. A separately purchasable Tool Qualification Pack is the baseline for qualifying TESSY as a verification tool according to DO-178B/C.

One boundary deserves precision: Razorcat states directly that the Classification Tree Editor, which covers test preparation, is not part of that certified core workflow. CTM is applied by a human being, as Razorcat says, and its output is a design rationale, not a computed result. Use CTM for the argument and TESSY’s certified workflow for the evidence.

How Razorcat Implements CTM in the Classification Tree Editor

The CTE is Razorcat’s graphical realisation of the method, integrated into TESSY or standalone on Windows 7/8/10/11 (64-bit). Per Razorcat, the CTE requires a valid licence, and a TESSY licence includes the permission to run it; an evaluation licence can be requested online after installing and starting CTE. Razorcat versions the standalone CTE separately from TESSY, at 4.3 with its own user manual at the time of writing. Per Razorcat, the editor has a tree area for the root, compositions, classifications and classes; a Test Table for marking classes into test cases, test sequences and test steps; and a palette that also defines AND, OR, XOR and NEGATION dependencies, which is how you rule out combinations the problem forbids.

Two Razorcat implementation details matter most in embedded work. First, TESSY can generate an initial tree from the test object interface and the interface dictionary, always containing Inputs and Outputs subtrees, with nodes for parameters, globals, return value and the parameters and return values of called functions; with “Update generated tree based on interface changes” enabled, it spots interface elements not attached to any CTE node and offers to merge a new tree into the existing one. Razorcat adds that a generated tree always needs review and adjustment by a tester. Second, test data assigned to a tree node is applied automatically when that class is marked, and CTE values are read-only in the Test Data Editor, which keeps the tree the single source of the design. Special values including *min*, *max*, *min+1* and *max-1* are available, exactly the notation boundary value work needs.

Specifications export to XML, XLSX, TXT, SVG, PNG and JPG, so the derivation travels into a review pack or safety case appendix. The rest of the TESSY workflow then takes over: requirements imported via ReqIF from DOORS, Jama Connect and Visure, or from Polarion via a plug-in; then coverage measurement, which per Razorcat covers statement (C0), branch (C1), decision, MC/DC, multiple condition, entry point, function and call pair coverage, as our MC/DC coverage guide explains.

Razorcat’s public product documentation is currently the TESSY 5.1 manual, so if your programme is scoping a later release, ask GSAS to confirm the applicable feature scope and availability.

Related reading

• Test Case Quality in Embedded Unit Testing, what a good case looks like before you design one
• Unit Testing Fundamentals for Embedded C and C++, the harness the designed cases run in
• Fault Injection and Robustness Testing, abnormal inputs at the interface, and fault injection for the branches the interface cannot reach
• Razorcat TESSY, the tool these techniques are run with in India, the UAE and Sri Lanka

Sources

• Razorcat Development GmbH, TESSY 5.1 User Manual, sections 3.1 and 3.2 (CTM, equivalence partitioning, ice warning and is_value_in_range examples, minimum and maximum criteria), preface (certification scope, core workflow, Tool Qualification Pack), section 6.8 (CTE tree generation, Test Table, dependencies, special values).
• Razorcat, CTE product page: the two steps of the method, export formats, standalone availability.
• Razorcat, CTE downloads page: CTE 4.3, system requirements, and the licence statement that a TESSY licence includes the permission to run the CTE.
• Razorcat, TESSY product page: CTE integration, requirements import, coverage measurements.
• ISTQB, Certified Tester Foundation Level Syllabus v4.0, section 4.2.1, Equivalence Partitioning.
• ISO 26262-6:2018 Table 8, as set out on our ISO 26262 page.

CTE and TESSY in India, the UAE and Sri Lanka

GSAS Micro Systems is the authorized Razorcat engineering partner for India, the UAE and Sri Lanka, covering TESSY and the standalone Classification Tree Editor with licensing, INR invoicing and the part no licence covers: sitting with your team while the first trees are drawn against real requirements, so classifications reflect your specification and not your source code. Our engineers support automotive, aerospace, medical and industrial programmes from Bengaluru, Hyderabad, Chennai, Pune, Mumbai and Delhi NCR. If your team is preparing for a safety assessment and wants test cases that survive the question “why these?”, request a quote or a guided evaluation.