How to Evaluate a Unit Testing Tool for Embedded Software: A Buyer's Framework for Indian Teams

A Feature Grid Does Not Predict Whether a Unit Test Tool Will Work

Most unit test tool evaluations in India start the same way: a spreadsheet of features, three vendors across the columns, a tick in almost every row. It predicts almost nothing, because the two ways these purchases actually fail are not on it: the tool cannot drive the compiler and debugger your project is committed to, or the evidence it produces does not answer the question your assessor asks. Both surface late, after the licence is paid for.

Here is a buyer-side framework instead: six questions for any candidate tool, and a pilot that measures the answers rather than accepting them.

Question 1: Does It Drive Your Compiler and Your Debugger?

“Supports embedded C and C++” is not an answer. A unit test tool does not merely read your code, it builds and runs it: it generates a test driver, compiles that driver with your cross-compiler, links it, and executes it. Every step is toolchain-specific, so what you need is a named combination of your compiler, at your version, with your debugger, against your target device.

Razorcat treats that combination as an explicit configuration object. Per Razorcat, TESSY lets you “configure your desired compiler/debugger with the help of the integrated Test Environment Editor”, and Razorcat publishes a filterable matrix of supported environments, selectable by compiler vendor, compiler version, target vendor and target version, which then shows the exact Test Environment Editor configuration for that selection. Combinations not yet available can be requested, per Razorcat.

Measure: hours from installation to the first passing test on your compiler, not a demo project.

Question 2: Can It Execute on the Target, or Only on the Host?

Host execution is fast and convenient. It is also a different processor and a different compiler back end from the one that ships, and a reviewer eventually asks whether the tested binary resembles the delivered one. Razorcat lists “test execution on hosts, simulators and hardware” among TESSY’s key features for unit and integration testing.

Instrumentation is not free. The NASA Langley practical tutorial on MC/DC, written with authors from the FAA, Boeing and Rockwell Collins, is blunt: software probes add overhead for memory and CPU cycles, so instrumented software “may not perform properly in memory or throughput critical components”, and the probes may cause the compiler to generate code that will not perform properly. TÜV SÜD’s report on TESSY carries the same warning as a condition of use, stating that compiler options “should be used consciously, and their potential interference with the testing should be analyzed (e.g. code instrumentation may influence code optimization)”.

Measure: run one identical suite on host and on target, then record the result differences and the flash and RAM growth of the instrumented build.

Question 3: Which Coverage Levels, and How Are They Obtained?

Coverage requirements come from your standard and your integrity level, not from preference. In DO-178B’s Table A-7, per the NASA tutorial, objective 7 requires statement coverage for software levels A to C, objective 6 requires decision coverage for levels A and B, and objective 5 requires MC/DC for level A only. The FAA recognises RTCA DO-178C in Advisory Circular 20-115D. DO-178C carries the same structural coverage objectives at the same software levels in its Table A-7, so the mapping above is the one to work from on a current project. Automotive teams have the equivalent conversation against ISO 26262-6:2018.

Razorcat lists eight coverage measurements provided in TESSY: statement coverage (C0), branch coverage (C1), decision coverage, MC/DC, multiple condition coverage (MCC), entry point coverage (EPC), function coverage (FC) and call pair coverage (CPC). Per Razorcat, coverage is optionally enabled when running tests, “which causes TESSY to automatically instrument the source code for the selected coverage measurements”, and call pair coverage “meets the requirements of ISO 26262 for safety-relevant software development as an alternative code coverage procedure for integration tests instead of function coverage (FC) measurement”.

The list is the easy part. Ask the mechanical questions the NASA tutorial recommends for any coverage tool: which statement types are monitored, whether coverage is measured on source or object code, whether a limit applies to the conditions tracked in one Boolean expression, which algorithm determines independent effect, and how relational operators are handled. Our MC/DC coverage guide goes deeper.

Question 4: What Qualification Evidence Exists, for Your Standard?

Marketing language and certification language diverge most sharply here, so read the actual documents. Razorcat states that TESSY “is qualified for safety-related software development according to IEC 61508, IEC 62304, ISO 26262 and EN 50128”, and publishes the certificate and the report behind it. TÜV SÜD certificate Z10 078930 0004 Rev. 01, issued 2023-11-15 and valid until 2028-11-12, records that the verification tool “fulfills the requirements for support tools classified T2 according to IEC 61508-3 and EN 50128” and is suitably validated for use according to IEC 62304. Testing covered IEC 61508-3:2010, IEC 62304:2006 with AMD1:2015, ISO 26262-8:2018 and EN 50128:2011/A2:2020, and the report is a mandatory part of the certificate.

Three evaluation lessons sit inside that paragraph.

A certificate has a scope, and your standard may sit outside it. DO-178C is a separate regime: the FAA recognises RTCA DO-330, Software Tool Qualification Considerations, in AC 20-115D, and per the NASA tutorial, tool qualification is required if the tool reduces, eliminates or automates an objective of DO-178B and its output is not otherwise verified. Under DO-178C the equivalent criteria are in section 12.2 and RTCA DO-330. Avionics teams should plan tool qualification as project work rather than assume a certificate transfers. See our DO-178C page.

A certificate has a version list. TÜV SÜD’s report identifies exactly which TESSY releases were assessed against which standards, and on which operating system. Ask any vendor for the equivalent table, then check the release you will install is on it.

Qualification does not remove your obligation. The report classifies TESSY’s tool impact as TI = 2 under ISO 26262-8:2018, because on failure the tool “may mask existing errors in the code being tested”, and notes that tool error detection can vary between TD1, TD2 and TD3 depending on the measures applied in the user’s own development process. TÜV SÜD writes that “the user’s development process including complete verification and validation should be conducted according to ISO 26262-8 in order to achieve the best possible TCL value”. See our ISO 26262 and IEC 61508 pages.

Question 5: How Does It Behave in CI?

A tool that only works when an engineer clicks through a GUI gets run before audits and at no other time. Interactive use is how tests are authored; automation is how they stay true. Per Razorcat, TESSY exposes an API for the project database and the test database, and “a command line executable provides all commands of the API for usage within DOS batch files”, alongside an integrated batch shell supporting scripts for automated testing. Razorcat’s TESSY flyer lists “command line scripting for continuous integration”, a dedicated Jenkins plugin, and change-based regression testing in which TESSY detects changes in the source code and executes only the tests for changed code.

Measure: run the pilot suite headless on a build agent. Confirm failures return a non-zero exit status, reports land as archivable files, and record runtime across a realistic module count.

Question 6: How Are Tests Stored, Diffed and Merged?

Test data is an engineering artefact, not scratch work. Locked inside a proprietary binary database it cannot be reviewed, merged across branches, or compared between releases. Razorcat’s answer is a textual representation: per the TESSY flyer, the script perspective “allows textual editing of tests as well as review of changes in a dedicated test scripting language”, test data converts between script format and the TESSY internal format in both directions, and auditing of test changes via textual test scripts is a listed feature.

Ask about migration in the same breath. Razorcat’s FAQ notes that opening a database created by TESSY V3.0 to V4.3 in V5.1 triggers a conversion, that the conversion is irreversible, and that configuration adjustments are usually required. That is normal for this class of tool, and exactly why the upgrade path belongs in the evaluation rather than in year three. Razorcat has since compiled a Version 6 feature set including an ASCII, YAML-based database backup meant to make test assets diffable under version control; its public product pages currently document TESSY 5.1, so ask GSAS to confirm Version 6 scope.

The Evaluation Matrix

A Four-Week Evaluation That Produces Evidence

Week 1, environment. Install, configure your compiler and debugger, reach one passing test on real project code. Log every support ticket raised; the count matters more than the outcome.

Week 2, one real module. Choose genuine complexity, not a demo, and build test cases from requirements rather than from source. The NASA tutorial names the opposite as a recurring failure: trying to meet the MC/DC objective apart from requirements-based testing, by using the source code to derive inputs for all test cases.

Week 3, coverage and evidence. Enable the coverage levels your standard requires, run on target, produce the report you intend to hand an assessor, and have someone outside the team read it cold.

Week 4, automation and version control. Run headless in CI, commit the test artefacts, change one test on a branch so a reviewer sees the diff, then re-run only what changed.

Two habits from the NASA tutorial’s pitfall list are worth carrying throughout: budget for tool qualification during planning rather than after selection, and stay wary of erroneous expectations about coverage tools. Coverage helps confirm that requirements-based testing was adequate; it is not itself a method for finding bugs. Our ISO 26262 unit testing checklist covers the surrounding process work.

Related reading

• Unit Test Tool Licensing, Evaluation to Production, what happens after the evaluation, commercially
• Unit Testing Fundamentals for Embedded C and C++, the capabilities you are actually evaluating
• Is 100% Code Coverage Enough?, the coverage levels to test the tool against
• Razorcat TESSY, the tool these techniques are run with in India, the UAE and Sri Lanka

Where TESSY Sits, and Where GSAS Sits

TESSY is developed by Razorcat Development GmbH in Berlin. Everything attributed to TESSY 5.1 above comes from Razorcat’s own published material: the TESSY product page, the TESSY 5 flyer, the TESSY 5.1 support FAQ, and the TÜV SÜD certificate and its report. The Version 6 items noted above come from the Version 6 feature set Razorcat compiled, which is not yet on Razorcat’s public product pages.

Per Razorcat’s Version 6 feature set, the Hyper Coverage feature integrates the Accemic Technologies hardware trace port analysis tool, using FPGA-based online trace data processing to acquire branch and MC/DC coverage during system and integration testing. Accemic supply that trace hardware; GSAS does not carry it. In the publicly documented TESSY 5.1, Hyper Coverage is the software accumulation of coverage results across tests, testing levels and test tools.

Keep one distinction clean while you evaluate. TESSY is dynamic testing: it executes your code and measures structural coverage. Static analysis tools such as Perforce Helix QAC and Klocwork reason about code without running it, and neither replaces the other, as our static plus dynamic ISO 26262 verification guide explains.

GSAS Micro Systems is the authorized Razorcat engineering partner for India, the UAE and Sri Lanka. We run this evaluation alongside your team rather than hand over a licence key: confirming your compiler, debugger and target against Razorcat’s environment list before you commit, setting up the on-target execution path, wiring the command line interface into your pipeline, and walking the certificate and its report with your functional safety lead. Our engineers support automotive, aerospace, railway, medical and industrial programmes from Bengaluru, Hyderabad, Chennai, Pune, Mumbai and Delhi NCR, and across the UAE and Sri Lanka.

See the TESSY product page, the Razorcat partner page and our automotive solutions page, or request a guided evaluation and we will scope it against your standard, your integrity level and your toolchain.