When Qbv Breaks PTP: TSN Scheduling and Time Sync Conflicts

Time aware scheduling and time synchronisation are supposed to be two halves of the same design. On a Linux bench they are two subsystems that share an egress path and know nothing about each other, and the scheduler is the one holding the gate.

The failure looks like this. gPTP is locked. You install a taprio qdisc to add the schedule your architecture calls for. Sync stops, and not gradually: the ports go quiet, the receipt timeouts fire, and within seconds every node has decided it is the grandmaster. Nothing in the logs says the word gate.

This article is about the three separate mappings that have to agree before a PTP frame reaches the wire. If your sync was never working in the first place, start at gPTP troubleshooting instead, because a schedule is not the first thing to suspect on a domain that never formed. Every option, default and log string below traces to a source in the References.

The symptom, and what it is not

PTP locked, then you installed a qdisc

This fault has one property that makes it easy to confirm and easy to misread: it worked one command ago. That rules out most of what you would normally check. Not the PHY or the cable, because the link is up and other traffic still crosses it. Not the domain number, the transport or the destination MAC, because those did not change. What changed is which queue PTP frames land in, and when that queue may transmit.

What the public record actually says

Two reports of exactly this sit open in public trackers, both filed in April 2021 under the same title, one in the openil fork of linuxptp and one in the Avnu TSN documentation repository. Both are still open, five years on, and the linuxptp one now sits in a repository under NXP’s GitHub archive organisation, where the thread was still drawing comments as late as November 2024.

The reporter describes a three node setup, a source, a hop and a sink, with taprio on the hop. After configuring taprio, with and without flags 0x1, ptp4l stops being in sync and stops exchanging messages, while with ETF, with CBS, or with no qdisc at all, everything works and the clocks stay in sync. The logs show ports moving from LISTENING to MASTER on ANNOUNCE_RECEIPT_TIMEOUT_EXPIRES, each node selecting its own local clock as best master, then repeating master sync timeout and master tx announce timeout. Another participant reports the variant with missing timestamp on transmitted sync followed by MASTER to FAULTY on FAULT_DETECTED (FT_UNSPECIFIED).

The threads are not empty. Contributors and users suggested an all gates open entry, raising neighborPropDelayThresh, raising tx_timestamp_timeout, and a kernel commit. The all gates open entry was the first suggestion and it did not help: the reporter re-ran with sched-entry S ff 50000 added and reported the same behaviour, and a later participant hit the same fault with a single gatemask 0xff entry. One participant reported that raising tx_timestamp_timeout fixed it for them; another found that omitting the flags parameter entirely made it work. Late in one thread a passing commenter names the priority to traffic class mapping as one cause in three sentences, the reporter confirms it helps, and the thread moves on without anyone writing the mechanism down. Both end unanswered: the Avnu thread on a 2024 request for progress, the openil one on a 2024 set of technical questions about which clock the txtime assist mode uses that nobody replied to.

The material is out there in fragments. The chain that connects them is not.

One caveat about what follows. The closed gate mapping below is the mechanism this article documents, and no schedule quoted in these reports leaves PTP’s traffic class without a window: the reports are the public trail of the symptom, not a demonstration of that mechanism.

The give-away: the failure is directional and instant

Two observations narrow this fast. The failure appears at the instant the qdisc is installed and disappears when it is removed. And it is directional: a schedule is an egress property, so it can only silence what the scheduled port sends, not what it receives, which means a capture on each side of the scheduled node should disagree.

There is a lookalike to rule out first. The TSN Documentation Project notes that qdisc setup resets the NIC and that this can make ptp4l out of sync, so any qdisc setup done after the clocks are already synchronised has to be followed by repeating and verifying the synchronisation steps. That one is a transient at install time. The gate problem does not clear on its own.

How a gate control list can starve PTP

The scheduling mechanism people call Qbv is IEEE Std 802.1Qbv-2015, Enhancements for Scheduled Traffic, which the IEEE 802.1 TSN task group lists as a completed amendment that has been rolled into IEEE Std 802.1Q-2018. On Linux you meet it as the taprio qdisc.

Three mappings that have to agree

A PTP frame passes through three translations before the schedule sees it, and no layer checks the next one.

Socket priority. The application sets SO_PRIORITY, which the socket manual describes as the protocol defined priority for all packets sent on that socket, used by Linux to order the networking queues, with values outside the range 0 to 6 requiring CAP_NET_ADMIN. ptp4l exposes it as socket_priority, documented as configuring SO_PRIORITY to support routing ptp4l traffic through Linux qdiscs for traffic shaping. It must be in the range 0 to 15 inclusive, the default is 0, and it is only available with the IEEE 802.3 transport, the -2 option, and is silently ignored on the UDP transports.

Priority to traffic class. The taprio map parameter maps priorities 0 to 15 to a specified traffic class, the same semantics as mqprio. The queues parameter then maps each traffic class onto a contiguous range of hardware queues as count@offset.

Traffic class to gate bit. Each sched-entry carries a command, a gate mask and an interval. The gate mask is a bitmask where each bit is associated with a traffic class, so bit 0 being on means traffic class 0 is active for that entry, and the interval says how long that state is held. The cycle time is the sum of all the intervals in the schedule.

Now put the defaults in. ptp4l’s socket priority is 0. The map used in the first two tc-taprio examples, and in the TSN Documentation Project’s example, is written map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2, which the man page reads as priority 3 classified as traffic class 0, priority 2 as traffic class 1, and the rest as traffic class 2. Priority 0 is part of the rest, and the project’s commentary on the same map is blunter: all best effort traffic goes into the last queues.

So a stock ptp4l lands in the class those examples treat as best effort, and whether it transmits depends on whether that class ever gets an open gate.

PTP gets no privileged treatment in the schedule

There is no exemption. The taprio qdisc implements a simplified version of the scheduling state machine defined by IEEE 802.1Q-2018, in which each gate state allows outgoing traffic for a subset, potentially empty, of traffic classes. Event, general and management messages are frames with a priority like any other, and the reserved destination MAC that gPTP uses buys nothing at this layer.

What a closed gate does to a Sync, and why the timers then fire

If PTP’s traffic class has no bit set in any entry, its frames sit in the queue. The peer receives nothing, which is the same input it would get from a cut cable, so the receipt timeouts run their course. announceReceiptTimeout is the number of missed Announce messages before the last Announce expires, default 3. syncReceiptTimeout is the number of sync and follow up messages that may go missing before triggering a Best Master Clock election, and the linuxptp gPTP example config sets it to 3.

You can predict the moment. In the public report the daemon logs announceReceiptTimeout is 5 with logAnnounceInterval is 2, that is 4 seconds, and the port flips from LISTENING to MASTER about 20 seconds after start. Five intervals of four seconds. Every node runs the same arithmetic in parallel, so they all elect themselves and the domain fragments rather than stops.

There is a second path to the same symptom, which is why opening a gate does not always fix it. The schedule can delay a frame’s egress by up to a cycle, and ptp4l polls the kernel for the transmit timestamp for only tx_timestamp_timeout milliseconds, default 10. Held past that, ptp4l logs a missing timestamp and takes the port to FAULTY, clearing it after fault_reset_interval, default 4, that is 16 seconds. A contributor raised exactly this in the Avnu thread: the reporter’s cycle was 1 ms and the timeout in their log was 1 ms, which was the default at the time; the current ptp4l manual documents the default as 10.

Triage in five minutes

Work in this order. Each step either exonerates a layer or hands you the fix path.

1. Confirm the qdisc is the trigger. Delete it (tc qdisc del dev <if> root) and watch the daemon. If sync returns within a few intervals, you are in the right article. If not, go and triage the sync domain itself.
2. Dump the installed schedule and read the gate masks. tc -s qdisc show dev <if> prints the map, the queue ranges, the clockid, the flags, the base time, the calculated cycle time, and one line per entry in the form index 0 cmd S gatemask 0xff interval 500000. Write the masks out in binary. Bits never set in any entry are traffic classes that never transmit.
3. Find which traffic class PTP lands in. Read the map in that same output as a lookup table: index by priority, value is the traffic class.
4. Check the socket priority the daemon is actually using, not the one in the config file you think is loaded. Run ptp4l with -l 7, read the config item it prints for socket_priority, and confirm the transport is L2, because on UDP the option is ignored without comment.
5. Count frames per class at egress. The child qdisc statistics in tc -s qdisc show give backlog and drops per queue, and ethtool -S <if> gives per queue counters such as tx_queue_3_packets. A queue whose counter is frozen while others advance is the answer, and the evidence to keep for an escalation.

If steps 2 to 4 describe a closed path, go to fix path A. If they line up correctly and PTP is still silent, go to fix path C.

Fix path A: give PTP a class with an open gate

Two ways round, and they differ in what else they disturb.

Move PTP to a class that already has a window. Set socket_priority in the ptp4l config to a priority the map sends to a class with an open gate. This touches one daemon rather than the whole traffic plan. In the public thread a commenter proposed exactly this, and the reporter confirmed that socket_priority 2 or 3 helped, but only with ETF in offload mode; ETF without offload reproduced the same fault. Note also that the reporter had been running --socket_priority 1 all along, and under that example map priority 1 lands in the same traffic class as priority 0, so the change that helped was moving classes, not moving off the default.

Or widen the schedule. Add an entry whose gate mask includes PTP’s class, or set that bit in an existing entry. One entry can open several classes, and an entry with every bit set opens all of them, which is a useful bisection step even if it is not the shipping configuration.

On width and repetition. ptp4l expresses its message intervals as powers of two seconds, so a gPTP port emits a small number of small frames per second, while a schedule cycle, being the sum of its entry intervals, is usually far shorter. PTP therefore does not need a wide window, it needs one that recurs: a short window every cycle beats one long window once per second, because what has to be covered is the worst case wait, not the total bandwidth.

The width still has a floor: a window that cannot fit the frame is a closed window for that frame. If you share a class, bound what can sit in front. max-sdu sets, per traffic class, the maximum L2 payload that can egress that class, with 0 meaning the port MTU, and the man page describes the interference it bounds as the transmit time of that maximum SDU plus the L2 header plus the L1 overhead. Compute the floor from your own link rate and frame sizes. No public source fixes a number here, and we are not going to invent one.

Fix path B: transport and tagging traps

The option that does nothing. If your config sets socket_priority and your transport is UDPv4, the ptp4l default, the marking is not applied and no message says so. For an 802.1AS domain the transport should be L2 anyway, which is why the gPTP article treats transport as failure number two.

Untagged frames have no PCP. Priority inside Linux and the PCP field in a VLAN tag are different things that get discussed as if they were one. taprio classifies on the internal priority; the tag carries PCP only if something writes it. The VLAN interface’s egress-qos-map defines a mapping of Linux internal packet priority to the VLAN header prio field for outgoing frames, and ingress-qos-map does the reverse on receive. Without an egress map, your chosen socket priority never appears on the wire and every downstream device classifies the frame by its own default. The interface you install the qdisc on matters for the same reason: a schedule on the physical interface and a tag applied by a VLAN interface are two steps that can disagree.

Sender marking against switch rewriting. When the PCP on the wire is not what you set, capture at both ends of the link. If the frame leaves with the right PCP and arrives with another, a bridge reclassified it. Compare like with like: a mirror port can strip tags, so a tag missing in a mirror capture is not proof it was missing on the wire. When logs and capture disagree, measure at the wire.

Fix path C: offload mode and timestamping interactions

The flags parameter selects between three different machines, and the failure modes do not carry over.

With no flags, neither offload path is enabled and scheduling is done in software. With flags 0x1, txtime assist, taprio sets a transmit timestamp according to the interval the packet needs to go out in and then uses the ETF qdisc to sort and transmit at the right time. With flags 0x2, full offload, taprio passes the gate control list to the NIC, which executes it cyclically in hardware. The two are mutually exclusive; flags 0x3 is invalid.

The ETF drop rules are the trap in txtime assist mode. ETF drops any packet with a txtime in the past or that expires while waiting to be dequeued, drops packets from sockets that do not use the qdisc’s clockid as the reference for their txtime, and currently drops any packet which does not have a socket associated with it, or whose socket does not have the SO_TXTIME option set, unless skip_sock_check is configured. The ptp4l manual documents no transmit time option at all; socket_priority is the only egress control it offers on the L2 transport. So a PTP frame entering an ETF class carries no SO_TXTIME, and the drop is silent from the daemon’s side: it sees a message it believes it sent, and no timestamp. Both public configurations set skip_sock_check on their ETF children, which disables this check, so this is a trap to rule out on your bench rather than the explanation for those threads. What the public thread does record is that raising the priority worked only with ETF in offload mode, while ETF without offload produced LISTENING to FAULTY on FAULT_DETECTED. One participant separately reported that the fault tracked flags 0x1 for them and disappeared when they omitted the flags parameter; their dump shows no ETF qdisc, so txtime assist mode itself, not the ETF drop rules, is what they were escaping.

Full offload moves the clock the schedule runs on. The tc-taprio manual states that the clockid argument must be omitted with flags 0x2, because in that case the clockid is implicitly the NIC’s PTP hardware clock and therefore not necessarily synchronized with the system’s CLOCK_TAI. Read that carefully: the schedule is now phase locked to the clock ptp4l is disciplining, so while the servo steps the PHC, the windows move under the traffic.

Timestamping method is a separate axis from gating. Moving from software to MAC hardware timestamping does not open a closed gate: whether a frame is allowed out of the queue and how accurately its departure is measured are different questions. What the timestamping method changes is how much of the transmit path ends up inside the measurement, and a schedule can hold a frame for up to a cycle before it leaves. Under a schedule that delay lands in the software timestamped number and in the tx_timestamp_timeout budget at the same time, so software timestamping can look worse than it is when the schedule is what changed.

When timestamps arrive but the offsets are nonsense. Check three things before touching the servo: whether the schedule and the daemon reference the same clock, whether tx_timestamp_timeout exceeds the worst case egress delay the schedule can impose, which is bounded by the cycle, and whether ETF is dropping a fraction of frames rather than all of them, which leaves the servo fed by a biased subset rather than by nothing.

When sync survives but the offset drifts

This is the version that ships: the domain forms, the servo locks, and the number is quietly wrong.

Asymmetric windows create path asymmetry. Peer delay measures the round trip to the neighbour and assumes the two directions are equal. Put a schedule on one end and not the other, and the measured delay carries a constant bias the servo then subtracts from every offset. ptp4l offers delayAsymmetry, documented as the time difference in nanoseconds of the transmit and receive paths, which is the right tool for characterised hardware delay and the wrong one for a scheduling artefact you have not identified.

A scheduled hop is a hop. If the box holding the schedule also forwards PTP for others, the queueing delay it adds is corrected by nothing unless it participates in the domain as a time aware device, a transparent clock or a boundary clock rather than a silent forwarder. Forwarding gPTP frames unchanged while adding scheduled queueing delay is the worst option, because the domain forms and is wrong. That is the argument the gPTP article makes about non time aware bridges, with more force here, because a schedule makes the residence time both larger and periodic.

Measure over cycles, not seconds. A schedule imposes a periodic delay, so sampling the offset once a second against a sub millisecond cycle aliases: you see a slow wander that has nothing to do with the oscillator. Log the offset and the path delay together across many full cycles and correlate against the cycle, not against wall time.

For where scheduled traffic sits in a vehicle network, see our automotive Ethernet guide and the domain and zonal architectures piece.

Where GSAS fits

GSAS Micro Systems is an engineering partner to teams building automotive Ethernet and TSN systems in India. TSN scheduling faults are expensive because they present as time sync faults, and time sync faults get assigned to whoever owns the clocks, who then debugs a subsystem that is working correctly.

Bring three artefacts and this gets short: the full tc -s qdisc show output for the scheduled interface, the ptp4l and phc2sys configs and logs from process start, and a capture from both sides of the scheduled link. Our applications engineers in Bengaluru, Pune, Chennai and Hyderabad work the mapping through with your team on your hardware and your schedule, not on a reference setup that already agrees with itself.

If you are bringing up a TSN bench, or planning the measurement setup for one, start at our automotive Ethernet capability page or talk to an engineer.

References

• tc-taprio(8) manual, iproute2, for gate masks, map, queues, base-time, cycle-time, flags, txtime-delay and max-sdu: https://man7.org/linux/man-pages/man8/tc-taprio.8.html
• tc-taprio(8) mirror: https://man.archlinux.org/man/tc-taprio.8.en
• tc-etf(8) manual, iproute2, for SO_TXTIME, delta, deadline_mode, offload, skip_sock_check and the drop rules: https://man7.org/linux/man-pages/man8/tc-etf.8.html
• ptp4l(8) manual, for socket_priority, network_transport, tx_timestamp_timeout, syncReceiptTimeout, announceReceiptTimeout and fault_reset_interval: https://manpages.debian.org/unstable/linuxptp/ptp4l.8.en.html
• linuxptp ptp4l documentation: https://linuxptp.nwtime.org/documentation/ptp4l/
• socket(7) manual, SO_PRIORITY: https://man7.org/linux/man-pages/man7/socket.7.html
• ip-link(8), iproute2, VLAN egress-qos-map and ingress-qos-map: https://man7.org/linux/man-pages/man8/ip-link.8.html
• TSN Documentation Project for Linux, Configuring TSN Qdiscs: https://tsn.readthedocs.io/qdiscs.html
• TSN Documentation Project for Linux, Synchronizing Time with Linux PTP: https://tsn.readthedocs.io/timesync.html
• nxp-archive/openil_linuxptp issue 19 (formerly openil/linuxptp), “Configured TAPRIO - ptp4l do not sync” (open, opened 20 April 2021): https://github.com/nxp-archive/openil_linuxptp/issues/19
• Avnu tsn-doc issue 24, “Configured TAPRIO - ptp4l do not sync” (open, opened April 2021): https://github.com/Avnu/tsn-doc/issues/24
• Avnu Linux TSN documentation repository: https://github.com/Avnu/tsn-doc
• IEEE 802.1 TSN task group index, completed and ongoing projects: https://1.ieee802.org/tsn/
• A platform vendor’s TSN reference documentation, systemd services for TSN endpoint configuration, consulted for the gate control list shape and the separation of schedule and time sync services: https://eci.intel.com/docs/2.5/components/tsnrefsw/systemd-tsn.html