In short
Trace-based coverage records which instructions executed and which branches were taken while tests run on the real target, using the processor's trace hardware rather than code added to the build. That gives object code coverage of the binary under test. It is strong evidence that tests exercised the shipped code, but mapping it to source-level metrics such as MC/DC needs analysis of how the compiler translated each decision, and the coverage tool needs the same confidence or qualification argument as any verification tool.
Structural coverage is the part of a safety verification plan that most often turns into a schedule problem. The tests exist, they pass, and then someone asks the question an auditor will ask: how do you know these tests exercised the code that ships?
Hardware trace answers that question in a particular way. It records what the processor executed while the tests ran, on the target, without adding anything to the build. This article explains what that measurement is, how it relates to the coverage metrics ISO 26262 and DO-178C discuss, where it fits next to instrumented unit test coverage, and what a reviewer will challenge. Both standards are paywalled, so this article quotes no clause numbers from them. It relies on public secondary sources and on our own ISO 26262 and DO-178C compliance pages.
If you want the SEGGER-specific version of this story, read J-Trace PRO for ISO 26262 and DO-178C trace evidence. This article is the vendor-neutral method.
Two ways to measure coverage
Instrumented coverage inserts counters or markers into the code at build time. The instrumented build runs the tests and reports which points were reached. It is precise at the source level and works on almost any target, but the binary under test is not the binary you ship, and the added code costs memory and execution time.
Trace-based coverage uses the processor’s trace hardware. On Arm cores the ETM emits a compressed program flow stream; a probe captures it and a debugger reconstructs which instructions executed and which branches went which way. Nothing is added to the build. Arm’s Keil documentation for ULINKpro describes this: streaming instruction trace provides complete code coverage information, and requires the target device to have an ETM. SEGGER says J-Trace captures every executed instruction non-intrusively in real time, and lists code coverage among the analysis features of its Ozone debugger.
The two are not competitors. They answer slightly different questions.
| Instrumented coverage | Trace-based coverage | |
|---|---|---|
| Binary under test | Instrumented build | The build under test, unmodified |
| Level measured | Source constructs | Object code, mapped to source |
| Target requirements | Memory and time for instrumentation | Trace hardware and a route for trace off the chip |
| Strength | Source-level metrics such as MC/DC, unit level | Shows what the real binary executed, integration and system level |
| Typical tool | Unit test tools such as Razorcat TESSY | Trace probes with a coverage-capable debugger |
Object code coverage and the source-level metrics
The coverage metrics teams talk about, statement, branch or decision, and MC/DC, are usually defined over source code. Trace measures object code. The bridge between them is where a reviewer will focus.
DO-178C. As our DO-178C compliance page summarises, DO-178C permits structural coverage analysis at source code, object code or executable object code level, with the applicant choosing the most appropriate level, and at Level A it requires additional verification of code the compiler generates that cannot be traced to source. EASA’s Certification Memorandum CM-SWCEH-002, written against ED-12B/DO-178B, has a section on using object code coverage for equivalence to MC/DC. It describes object code coverage as attempting to achieve structural coverage on the object code, the assembly language code, instead of the source code, and states that the section combines an earlier EASA memorandum with the CAST-17 position paper on structural coverage of object code. If you intend to claim MC/DC equivalence from object code coverage, read that section and agree the approach with your certification authority before you build the evidence.
ISO 26262. As our ISO 26262 compliance page explains, the unit-level structural coverage metrics are statement coverage, branch coverage and MC/DC, rated as recommendations rather than mandates, with MC/DC carrying the strongest recommendation at ASIL D and a rationale required for the combination chosen. At the architectural level the metrics are different: function coverage and call coverage. Trace is a natural fit for those, because it records every function entry and call on the real target during integration testing.
MC/DC specifically. MC/DC requires showing that each condition in a decision independently affects the outcome. The NASA practical tutorial on MC/DC (NASA/TM-2001-210876) is a widely cited public reference. Trace records branch outcomes in object code, and whether those outcomes demonstrate MC/DC for a source decision depends on how the compiler translated the decision. That varies with compiler and optimisation level, so it is an analysis, not an automatic result. Our MC/DC coverage guide covers measuring MC/DC at the unit level.
A coverage strategy that holds up
Most safety programmes end up combining the two methods:
- Unit level: requirements-based unit tests with instrumented coverage, including MC/DC where the integrity level calls for it.
- Integration and system level: requirements-based tests on the target, with trace-based coverage of the binary under test, giving function and call coverage and object code statement and branch coverage.
- Reconciliation: explain every region neither method covered. Uncovered code is either a missing test, a missing requirement, or code that should not be there.
The article Is 100% code coverage enough? explains what each metric does and does not prove, which is the argument you will make in step 3.
What a reviewer will ask about trace-based coverage
- Which binary was measured? Record the build identity of the binary that ran and show it is the configuration under test.
- Was trace complete? Trace can overflow when the core generates more data than the trace route carries. Show that the capture had no gaps, or that gaps were detected and the affected runs repeated. The off-chip trace explainer covers overflow.
- How does object code map to source? Explain how the debugger maps instructions back to source, and how compiler-generated code with no direct source counterpart is handled.
- What confidence do you have in the tool? Under ISO 26262 the coverage tool needs a tool confidence argument; under DO-178C you assess whether DO-330 tool qualification applies. Check what qualification material your tool vendor provides. Our DO-178C page notes that SEGGER makes no DO-178C or DO-330 qualification claim for J-Trace or Ozone: they are instruments for observing on-target execution, and the qualification argument is yours.
Tools GSAS supplies today
- Arm Cortex-M: Arm ULINKpro with Keil MDK, and SEGGER J-Trace PRO with Ozone.
- Arm Cortex-A and Cortex-R, and RISC-V: J-Trace PRO Cortex-A/R, J-Trace PRO RISC-V, and Arm Development Studio with DSTREAM-ST.
- Unit-level coverage including MC/DC: Razorcat TESSY.
For on-target coverage on multicore AURIX/TriCore and RH850 ECUs, talk to our FAE team about the right probe for your device and trace interface.
Where GSAS fits
GSAS Micro Systems supplies trace probes, debuggers and unit test tools in India with INR quotations and local application engineering. Our engineers in Bengaluru, Pune, Chennai and Hyderabad help automotive and aerospace verification teams design a coverage strategy that combines unit-level and on-target evidence, and support teams in Mumbai and Delhi NCR on site and remotely.
Related reading
- Timing analysis from trace for AUTOSAR ECUs
- Non-intrusive debug of safety ECUs on production silicon
- SEGGER J-Trace PRO streaming trace
- Choosing debug and trace tools for Indian automotive Tier-1s
References
- EASA, Certification Memorandum CM-SWCEH-002, Software Aspects of Certification (Issue 01, 11 August 2011): https://www.easa.europa.eu/sites/default/files/dfu/certification-docs-certification-memorandum-EASA-CM-SWCEH-002-Software-Aspects-of-Certification.pdf
- Hayhurst, Veerhusen, Chilenski and Rierson, A Practical Tutorial on Modified Condition/Decision Coverage, NASA/TM-2001-210876, May 2001: https://ntrs.nasa.gov/citations/20010057789
- Certification Authorities Software Team position papers (CAST-17, Structural Coverage of Object Code), archived FAA index: https://web.archive.org/web/20150121123803/https://www.faa.gov/aircraft/air_cert/design_approvals/air_software/cast/cast_papers/archive/
- Arm Keil, ULINKpro: https://www.keil.com/arm/ulinkpro/
- SEGGER, J-Trace product family: https://www.segger.com/products/debug-probes/j-trace/
- SEGGER, Ozone debugger: https://www.segger.com/products/development-tools/ozone-j-link-debugger/
- GSAS, ISO 26262 compliance page: https://gsasindia.com/compliance/iso-26262
- GSAS, DO-178C compliance page: https://gsasindia.com/compliance/do-178c
Also appears in:
Building for Automotive & Mobility?
Talk to our application engineers for personalized tool recommendations.
More from SEGGER
View all →