Time Sensitive Networking (TSN) is an important networking standard that has sparked a lot of interest. Richard Chavez wrote and excellent overview of what TSN is and what it can do in this blog.
The majority of custom TSN implementations come with a custom application that allows a user to configure the TSN traffic classes and shaping parameters. For linux based systems that use COTS hardware and/or software it can be difficult to understand the customization options and implement them in a way that is desired.
In this article I will cover how to configure the DornerWorks TSN IP on a Linux platform, the steps detailed here can be used to configure any TSN implementation within reason.
The Linux kernel uses a subsystem called “Traffic Control” to manage traffic shaping, scheduling and policing. It does this using queueing discipline(s) (qdisc). Each interface has a qdisc associated with it and each qdisc is a configurable description of how packets should be managed. The most simple qdisc and the one that each interface defaults to is “Packet limited First In First Out” (pfifo).
When using a TSN implementation, it is desirable to use qdiscs that can manage hardware offloaded traffic classes. Three that are usable with the DornerWorks TSN IP are Multiqueue Priority Qdisc (MQPRIO), Time Aware Priority Shaper (TAPRIO), and Credit Based Shaper (CBS).
Traffic classes are prioritized mappings between qdiscs and hardware queues. There are 16 traffic classes available in Linux Traffic Control, ranging in priority from 0 to 15, the hardware offloading in the DornerWorks TSN IP handles up to 4 QOS (Extendable up to 8 depending on version and resource availability) which can be mapped into the linux traffic classes on a 1 to 1 basis.
QDISCs can be managed in Linux using the application “tc”, which is a part of the “iproute2” suite of tools.
MQPRIO is one of the most basic qdiscs, it is a simple priority based system, during times of high traffic or in the situation where two packets are in contention, the packet with the higher priority is sent first. Hardware queues in the DornerWorks TSN IP are mapped to traffic classes in the traffic control subsystem. For example, a new qdisc can added to the eth1 interface, as the parent qdisc with a handle of 100 using the following command.
tc qdisc add dev eth1 parent root handle 100 \ mqprio num_tc 4 \ map 0 1 2 3 0 0 0 0 0 0 0 0 0 0 0 0 \ queues 1@0 1@1 1@2 1@3 \ hw 1
The second line: mqprio num_tc 4, sets the type of qdisc to mqprio and the number of traffic classes to 4. MQPRIO can use up to 16 traffic classes and queues, the third line maps all 16 available mqprio queues to different traffic classes, with the unused queues 4 through 15 being mapped to traffic class 0. The fourth line defines the quantity of each queue and it’s traffic class, there is one queue for each traffic class. The last line hw 1 sets the hardware offload flag. The interface eth1 will now route each traffic class out of the corresponding hardware queue.
TAPRIO creates a list of schedules with corresponding gate masks, each gate mask is a bitmask that sets which traffic class can be sent during the scheduled interval, the interval time in nanoseconds is the period of time that the gate is open. TAPRIO uses Time Aware Shaper (TAS) in the DornerWorks TSN IP to schedule packet traffic out of an interface. The Time Aware Shaper allows mapping up to 16 traffic classes and adding a schedule for each traffic to an interface.
In the example below a taprio qdisc is being added to the eth1 interface with a root handle of 400. Four traffic class queues are added with mappings into the first four qdisc queues. Base-time indicates the time that the schedule will be started.
TAPRIO uses the available PTP clock to keep time for the schedule, in this case the PTP clock that is a part of the DornerWorks TSN IP on eth1 will be used. Sched-entry lines add schedules to the TAS, the S flag indicates the “SetGateStates” command, it is followed by bit mask indicating which gates will be open for that period, the final field is the time the gates should be open in nanoseconds. Eight schedules are being set in the example, the first entry allows all traffic through for 500000000 nanoseconds, the second is a guard band period of 500000000 nanoseconds which allows no traffic through.
The following six schedules allow a single traffic class through for 100000 nanoseconds followed by a guard band of 100000. It is advisable to follow each open gate state with a guard band. The final field is the flags, 0x2 sets the flag for full hardware offloading, which the DornerWorks TSN IP is fully capable of handling.
tc qdisc add dev eth1 parent root handle 400 \ taprio num_tc 4 map 0 1 2 3 2 2 2 2 2 2 2 2 2 2 2 2 \ queues 1@0 1@1 1@2 1@3 base-time 300000000000 \ sched-entry S 0xff 500000000 \ sched-entry S 0x00 500000000 \ sched-entry S 0x01 100000 \ sched-entry S 0x00 100000 \ sched-entry S 0x02 100000 \ sched-entry S 0x00 100000 \ sched-entry S 0x04 100000 \ sched-entry S 0x00 100000 \ flags 0x2
CBS uses credits that are accumulated and used as packets are sent, each traffic class has its own set of parameters that configures how quickly credits are accumulated or used, and what the minimum and maximum credit values are. Once all of the credits for a traffic class are used, it must sit idle for some period of time to reaccumulate more credits. The CBS qdisc must be added on top of an existing interface qdisc. The CBS shaping algorithm is applied to individual traffic classes and not to an interface as a whole.
CBS uses a credit based shaper to increase or decrease the bandwidth of each traffic class as packets are sent on an interface. In this example we will assume the MQPRIO qdisc from the previous section was applied to interface eth1. To replace traffic class 0 from the above MQPRIO qdisc with a CBS qdisc the following command is executed. The parent handle of the interface qdisc is 100, the minor number for traffic class 0 is 1, traffic class 1 minor number is 2 and so on.
Calculating the CBS parameters of idleslope, sendslope, hicredit and locredit are beyond the scope of this article, there are tools that can be used to calculate these values. The offload flag is set to 1 to enable offloading of Credit Based shaping to hardware, supported by the DornerWorks TSN IP.
tc qdisc replace dev eth1 parent 100:1 cbs \ idleslope 98 sendslope -91 \ hicredit 153 locredit -1389 offload 1
Now that traffic classes in Linux have been mapped to hardware queues in the DornerWorks TSN IP, we need a way to tell the difference between packets of different traffic classes as they enter and exit the system. Packets of different traffic classes are differentiated by the three bit Port Control Protocol (PCP) field of the 802.1Q tag section of a frame header. Packets going out of hardware queues with mapped qdiscs must be tagged with a PCP that corresponds to the traffic class it’s associated with. In order for a receiving system to differentiate these packets on an interface a VLAN with appropriate configuration needs to be added to the interfaces. Using the ip tool a vlan can be added to an interface which will map traffic classes to PCP fields.
The below example adds a vlan named eth1.5 to interface eth1, the vid portion of the 802.1Q tag is set to 5, the egress PCP value is mapped 1 to 1 to traffic classes using the egress-qos-map command in the format from:to, in the example it can be seen that traffic classes 0-3 are being mapped to the corresponding PCP values.
ip link add link eth1 name eth1.5 type vlan id 5 egress-qos-map 3:3 2:2 1:1 0:0
Systems that include TSN are often complex and require complicated configuration and management. Using Linux utilities can streamline the configuration by offering COTS tools that allow easy configurability.
The DornerWorks TSN IP is highly configurable and integrates easily with Linux utilizing DornerWorks PetaLinux BSP and drivers. DornerWorks is available to partner with you to help you find the best solution to your TSN hardware and software needs. Schedule a meeting with our team when you are ready to turn your ideas into reality.
References:
