Deterministic Networking (DetNet) Data Plane: MPLSEricssonMagyar Tudosok krt. 11.BudapestHungary1117balazs.a.varga@ericsson.comEricssonMagyar Tudosok krt. 11.BudapestHungary1117janos.farkas@ericsson.comLabN Consulting, L.L.C.lberger@labn.netMalis Consultingagmalis@gmail.comFuturewei Technologiessb@stewartbryant.comjouni.nospam@gmail.comDetNet
This document specifies the Deterministic Networking (DetNet) data plane
when operating over an MPLS Packet Switched Network. It leverages
existing pseudowire (PW) encapsulations and MPLS Traffic Engineering
(MPLS-TE) encapsulations and mechanisms. This document builds on the
DetNet architecture and data plane framework.
Introduction
Deterministic Networking (DetNet) is a service that can be offered by a
network to DetNet flows.
DetNet provides a capability for the delivery of data flows with
extremely low packet loss rates and bounded end-to-end delivery
latency.
General background and concepts of DetNet can be found in the DetNet
architecture .
The purpose of this document is to describe the use of the MPLS
data plane to establish and support DetNet flows.
The DetNet architecture models the DetNet-related data plane
functions as being decomposed into two sub-layers: a service sub-layer and a
forwarding sub-layer. The service sub-layer is used to provide
DetNet service functions, such as protection and reordering. At the
DetNet data plane, a new set of functions (Packet Replication, Elimination
and Ordering Functions (PREOF)) provide the tasks specific to the service
sub-layer. The forwarding sub-layer is used to provide
forwarding assurance (low loss, assured latency, and limited
out-of-order delivery). The use of the functionalities of the DetNet
service sub-layer and the DetNet forwarding sub-layer require
careful design and control by the Controller Plane in addition to the
DetNet-specific use of MPLS encapsulation as specified by this
document.
This document specifies the DetNet data plane operation and the on-wire
encapsulation of DetNet flows over an MPLS-based Packet Switched Network
(PSN) using the service reference model. MPLS encapsulation
already provides a solid foundation of building blocks to enable the DetNet
service and forwarding sub-layer functions.
MPLS-encapsulated DetNet can
be carried over a variety of different network technologies that can
provide the level of service required for DetNet. However, the specific
details of how DetNet MPLS is carried over different network technologies
are out of scope for this document.
MPLS-encapsulated DetNet flows can carry different types of
traffic. The details of the types of traffic that are carried in
DetNet are also out of scope for this document. An example of IP
using DetNet MPLS sub-networks can be found in . DetNet MPLS may use an associated controller
and Operations, Administration, and Maintenance (OAM) functions
that are defined outside of this document.
Background information common to all data planes for DetNet
can be found in the DetNet data plane framework .
TerminologyTerms Used in This Document
This document uses the terminology established in the DetNet
architecture and
the DetNet data plane framework . The reader is
assumed to be familiar with these documents, any terminology
defined therein, and basic MPLS-related terminologies in
.
The following terminology is introduced in this document:
F-Label
A DetNet "forwarding" label that identifies the Label Switched Path (LSP) used to
forward a DetNet flow across an MPLS PSN, i.e., a hop-by-hop
label used between Label Switching Routers (LSRs).
S-Label
A DetNet "service" label that is used between DetNet nodes that
implement the DetNet service sub-layer functions. An S-Label
is used to identify a DetNet flow at the DetNet service
sub-layer at a receiving DetNet node.
A-Label
A special case of an S-Label, whose aggregation properties are known only at
the aggregation and deaggregation end points.
d-CW
A DetNet Control Word (d-CW) that is used for sequencing information
of a DetNet flow at the DetNet
service sub-layer.
Abbreviations
The following abbreviations are used in this document:
CoS
Class of Service
CW
Control Word
DetNet
Deterministic Networking
LSR
Label Switching Router
MPLS
Multiprotocol Label Switching
MPLS-TE
Multiprotocol Label Switching Traffic Engineering
MPLS-TP
Multiprotocol Label Switching Transport Profile
OAM
Operations, Administration, and Maintenance
PE
Provider Edge
PEF
Packet Elimination Function
PRF
Packet Replication Function
PREOF
Packet Replication, Elimination and Ordering Functions
POF
Packet Ordering Function
PSN
Packet Switched Network
PW
Pseudowire
QoS
Quality of Service
S-PE
Switching Provider Edge
T-PE
Terminating Provider Edge
TSN
Time-Sensitive Networking
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED",
"MAY", and "OPTIONAL" in this document are to be interpreted as
described in BCP 14
when, and only when, they appear in all capitals, as shown here.
Overview of the DetNet MPLS Data PlaneLayers of DetNet Data Plane
MPLS provides a wide range of capabilities that can be utilized
by DetNet. A straight-forward approach utilizing MPLS for a
DetNet service sub-layer is based on the existing pseudowire (PW)
encapsulations and utilizes existing MPLS-TE
encapsulations and mechanisms.
Background on PWs can be found in , , and .
Background on MPLS-TE can be
found in and .
The DetNet MPLS data plane representation is illustrated in
.
The service sub-layer includes a DetNet Control Word (d-CW) and
an identifying service label (S-Label). The DetNet Control Word
(d-CW) conforms to the Generic PW MPLS Control Word (PWMCW)
defined in . An aggregation label (A-Label) is
a special case of S-Label used for aggregation.
A node operating on a received DetNet flow at the DetNet service
sub-layer uses the local context associated with a received S-Label,
i.e., a received F-Label, to determine which local DetNet
operation(s) are applied to that packet.
An S-Label may be taken from the platform label space
, making it unique and enabling DetNet
flow identification regardless of which input interface or
LSP the packet arrives on. It is important to note that S-Label
values are driven by the receiver, not the sender.
The DetNet forwarding sub-layer is supported by zero or more
forwarding labels (F-Labels). MPLS-TE
encapsulations and mechanisms can be utilized to provide a
forwarding sub-layer that is responsible for providing resource
allocation and explicit routes.
DetNet MPLS Data Plane Scenarios illustrates
a hypothetical DetNet MPLS-only network
composed of DetNet-aware MPLS-enabled end systems operating over a
DetNet-aware MPLS network. In this figure, the relay nodes are PE
devices that define the MPLS LSP boundaries, and the transit nodes
are LSRs.
DetNet end systems and relay nodes understand the particular needs
of DetNet flows and provide both DetNet service and forwarding
sub-layer functions. In the case of MPLS, DetNet service-aware nodes add, remove,
and process d-CWs, S-Labels, and F-Labels as needed.
DetNet MPLS nodes provide functionality analogous to T-PEs when
they sit at the edge of an MPLS domain and S-PEs when they are
in the middle of an MPLS domain; see .
In a DetNet MPLS network, transit nodes may be DetNet-service-aware or
DetNet-unaware MPLS Label Switching Routers
(LSRs). In this latter case, such LSRs would be unaware of the
special requirements of the DetNet service sub-layer but would
still provide traffic engineering functions and the QoS
capabilities needed to ensure that the (TE) LSPs meet the service
requirements of the carried DetNet flows.
The application of DetNet using MPLS supports a number of
control and management plane types. These types support certain MPLS
data plane deployments. For example, RSVP-TE, MPLS-TP, or MPLS Segment
Routing (when extended to support resource allocation) are all valid
MPLS deployment cases.
illustrates how an end-to-end MPLS-based
DetNet service is provided in more detail.
In this figure, the
Customer Edge (CE1 and CE2) are able to send and receive
MPLS-encapsulated DetNet flows, and R1, R2, and R3 are relay nodes in the
middle of a DetNet network. The 'X' in the end systems and relay
nodes represents potential DetNet compound flow packet replication and
elimination points. In this example, service protection is supported
utilizing at least two DetNet member flows and TE LSPs. For a
unidirectional flow, R1 supports PRF, and R3 supports PEF and POF. Note
that the relay nodes may change the underlying forwarding sub-layer,
for example, tunneling MPLS over IEEE 802.1 TSN or simply
over interconnected network links.
X
- Optional service protection (none, PRF, PREOF, PEF/POF)
DFx
- DetNet member flow x over a TE LSP
MPLS-Based DetNet Data Plane SolutionDetNet over MPLS Encapsulation Components
To carry DetNet over MPLS, the following is required:
A method of identifying the MPLS payload type.
A method of identifying the DetNet flow(s) to the processing element.
A method of distinguishing DetNet OAM packets from DetNet data packets.
A method of carrying the DetNet sequence number.
A suitable LSP to deliver the packet to the egress PE.
A method of carrying queuing and forwarding indication.
In this design, an MPLS service label (the S-Label) is similar to a
pseudowire (PW) label and
is used to identify both the
DetNet flow identity and the MPLS payload type satisfying
(1) and (2) in the list above.
OAM traffic discrimination
happens through the use of the Associated Channel method described in
.
The DetNet sequence number is carried in
the DetNet Control Word, which also carries the Data/OAM discriminator. To
simplify implementation and to maximize interoperability, two sequence
number sizes are supported: a 16-bit sequence number and a 28-bit
sequence number. The 16-bit sequence number is needed to support
some types of legacy clients. The 28-bit sequence number is used in
situations where it is necessary to ensure that, in high-speed networks,
the sequence number space does not wrap whilst packets are in flight.
The LSP used to forward the DetNet packet is not restricted regarding any
method used for establishing that LSP (for example, MPLS-LDP, MPLS-TE,
MPLS-TP , MPLS Segment Routing
, etc.). The F-Label(s) and the
S-Label may be used alone or together to indicate the required queue
processing as well as the forwarding parameters. Note that the possible
use of Penultimate Hop Popping (PHP) means that the S-Label may be the only
label received at the terminating DetNet service.
MPLS Data Plane Encapsulation illustrates a DetNet data plane MPLS
encapsulation. The MPLS-based encapsulation of the DetNet flows
is well suited for the scenarios described in
.
Furthermore, an end-to-end DetNet
service, i.e., native DetNet deployment (see ), is also possible if
DetNet end systems are capable of initiating and terminating
MPLS-encapsulated packets.The MPLS-based DetNet data plane encapsulation consists of:
A DetNet Control Word (d-CW) containing sequencing information for
packet replication and duplicate elimination purposes, and the OAM
indicator.
A DetNet service label (S-Label) that identifies a DetNet flow at
the receiving DetNet service sub-layer processing node.
Zero or more DetNet MPLS forwarding label(s) (F-Label) used to
direct the packet along the Label Switched Path (LSP) to the next
DetNet service sub-layer processing node along the path. When
PHP is in use, there may be no F-Label in the
protocol stack on the final hop.
The necessary data-link encapsulation is then applied prior to
transmission over the physical media.
DetNet Control Word and DetNet Sequence Number
A DetNet Control Word (d-CW) conforms to the Generic PW MPLS Control
Word (PWMCW) defined in . The d-CW formatted
as shown in MUST be present in all
DetNet packets containing App-flow data.
This format of the d-CW was created in order to (1) allow larger sequence number
space to avoid sequence number rollover frequency in some applications
and (2) allow sequence numbering systems that include the value zero
as a valid sequence number, which simplifies implementation.
(bits 0 to 3)
Per ,
MUST be set to zero (0).
Sequence Number (bits 4 to 31)
An unsigned value implementing the DetNet sequence number. The
sequence number space is a circular one with no restriction on the
initial value.
A separate sequence number space MUST be maintained by
the node that adds the d-CW for each DetNet App-flow, i.e., DetNet service.
The following Sequence Number field lengths MUST be supported:
0 bits
16 bits
28 bits
The sequence number length MUST be provisioned on
a per-DetNet-service basis via configuration, i.e., via the
Controller Plane described in .
A 0-bit Sequence Number field length indicates that there is no
DetNet sequence number used for the flow. When the length is zero,
the Sequence Number field MUST be set to zero (0) on all packets
sent for the flow.
When the Sequence Number field length is 16 or 28 bits for a flow,
the sequence number MUST be incremented by one for each new App-flow
packet sent. When the field length is 16 bits, d-CW bits 4 to 15
MUST be set to zero (0). The values carried in this field can wrap,
and it is important to note that zero (0) is a valid field value.
For example, where the sequence number size is 16 bits, the sequence
will contain: 65535, 0, 1, where zero (0) is an ordinary
sequence number.
It is important to note that this document differs from , where a sequence number of zero
(0) is used to indicate that the sequence number check algorithm is
not used.
The sequence number is optionally used during receive processing,
as described below in Sections and .
S-Labels
A DetNet flow at the DetNet service sub-layer is identified by
an S-Label. MPLS-aware DetNet end systems
and edge nodes, which are by definition MPLS ingress and egress
nodes, MUST add (push) and remove (pop) a DetNet
service-specific d-CW and S-Label. Relay nodes MAY
swap S-Label values when processing a DetNet flow, i.e., incoming and
outgoing S-Labels of a DetNet flow can be different.
S-Label values MUST be provisioned per DetNet service via
configuration, i.e., via
the Controller Plane described in
.
Note that S-Labels provide identification at the downstream
DetNet service sub-layer receiver, not the sender. As such,
S-Labels MUST be allocated by the entity that controls the service
sub-layer receiving a node's label space and MAY be allocated from
the platform label space .
Because S-Labels are local to each node, rather than being a
global identifier within a domain, they must be advertised to their
upstream DetNet service-aware peer nodes (i.e., a DetNet MPLS end
system or a DetNet relay or edge node) and interpreted in the
context of their received F-Label(s).
In some PREOF topologies, the node performing replication will be
sending to multiple nodes performing PEF or POF and may need to
send different S-Label values for the different member flows for the
same DetNet service.
An S-Label will normally be at the bottom of the label stack once
the last F-Label is removed, immediately preceding the d-CW.
To support OAM at the service sub-layer level, an OAM Associated Channel
Header (ACH)
together with a Generic Associated Channel Label (GAL) MAY be used in place of
a d-CW.
Similarly, an Entropy Label Indicator (ELI) and Entropy Label (EL) MAY be carried below
the S-Label in the label stack in
networks where DetNet flows would otherwise receive ECMP treatment. When
ELs are used, the same EL value SHOULD be used for all of the packets
sent using a specific S-Label to force the flow to follow the same path.
However, as outlined in , the use of ECMP for DetNet
flows is NOT RECOMMENDED. ECMP MAY be used for non-DetNet flows within a
DetNet domain.
When receiving a DetNet MPLS packet, an implementation MUST identify
the DetNet service associated with the incoming packet based on the
S-Label. When a node is using platform labels for S-Labels, no
additional information is needed, as the S-Label uniquely identifies
the DetNet service. In the case where platform labels are not used, zero
or more F-Labels proceeding the S-Label MUST be used together with
the S-Label to uniquely identify the DetNet service associated with a
received packet.
The incoming interface MAY also be used together with
any present F-Label(s) and the S-Label to uniquely identify an
incoming DetNet service, for example, in the case where PHP is used.
Note that the choice to use the platform label space
for an S-Label or an S-Label plus one or more F-Labels to identify
DetNet services is a local implementation choice, with one caveat. When one
or more F-Labels, or the incoming interface, is needed together with an
S-Label to uniquely identify a service, the Controller Plane must ensure that
incoming DetNet MPLS packets arrive with the needed information
(F-Label(s) and/or the incoming interface) and provision the needed
information. The provisioned information MUST then be used to
identify incoming DetNet service based on the combination of S-Label
and F-Label(s) or the incoming interface.
The use of platform labels for S-Labels matches other pseudowire
encapsulations for consistency, but there is no hard requirement in
this regard.
Implementation details of PREOF are out of scope for
this document.
defines equivalent replication and elimination-specific aspects,
which can be applied to PRF and PEF.Packet Replication Function Processing
The Packet Replication Function (PRF) MAY be supported
by an implementation for outgoing DetNet flows. The use of the PRF
for a particular DetNet service MUST be provisioned via configuration,
i.e., via the Controller Plane described in
. When replication
is configured, the same App-flow data will be sent over multiple
outgoing DetNet member flows using forwarding sub-layer LSPs.
An S-Label value MUST be configured per outgoing member flow.
The same d-CW field value MUST be used on all outgoing member flows for
each replicated MPLS packet.
Packet Elimination Function Processing
Implementations MAY support the Packet Elimination Function (PEF)
for received DetNet MPLS flows. When supported, use of the PEF
for a particular DetNet service MUST be provisioned via configuration,
i.e., via the Controller Plane described in
.
After a DetNet service is identified for a received DetNet MPLS packet,
as described above, if PEF is configured for that DetNet service,
duplicate (replicated) instances of a particular sequence number MUST be
discarded.
The specific mechanisms used for an implementation to identify which
received packets are duplicates and which are new is an
implementation choice. Note that, per ,
the Sequence Number field length may be 16 or 28 bits, and the
field value can wrap. PEF MUST NOT be used with DetNet flows configured
with a d-CW Sequence Number field length of 0 bits.
An implementation MAY constrain the maximum number of sequence numbers
that are tracked on either a platform-wide or per-flow basis.
Some implementations MAY support the provisioning of
the maximum number of sequence numbers that are tracked on
either a platform-wide or per-flow basis.
Packet Ordering Function Processing
A function that is related to in-order delivery is the Packet Ordering Function
(POF). Implementations MAY support POF. When
supported, use of the POF for a particular DetNet service MUST be
provisioned via configuration, i.e., via the Controller Plane
described by
.
Implementations MAY require that PEF and POF be used in combination.
There is no requirement related to the order of execution of the PEF and POF in an implementation.
After a DetNet service is identified for a received DetNet MPLS
packet, as described above, if POF is configured for that DetNet
service, packets MUST be processed in the order
indicated in the received d-CW Sequence Number field, which may not
be in the order the packets are received.
As defined in , the Sequence Number field length
may be 16 or 28 bits, the sequence number is incremented by one (1) for each new
MPLS packet sent for a particular DetNet service, and the field
value can wrap. The specific mechanisms used for an implementation
to identify the order of received packets is an implementation
choice.
An implementation MAY constrain the maximum number of out-of-order
packets that can be processed on either a platform-wide or per-flow
basis. The number of out-of-order packets that can be processed also
impacts the latency of a flow.
The latency impact on the system resources needed to support a
specific DetNet flow will need to be evaluated by the Controller Plane
based on that flow's traffic specification. An example traffic
specification that can be used with
MPLS-TE can be found in .
DetNet implementations can use flow-specific requirements (e.g., maximum
number of out-of-order packets and maximum latency of the flow) for
configuration of POF-related buffers. POF implementation details
are out of scope for this document, and POF configuration parameters
are implementation specific. The Controller Plane identifies and
sets the POF configuration parameters.
F-Labels
F-Labels support the DetNet forwarding sub-layer. F-Labels
are used to provide LSP-based connectivity between DetNet service
sub-layer processing nodes.
Service Sub-Layer-Related Processing
DetNet MPLS end systems, edge nodes, and relay nodes may operate at
the DetNet service sub-layer with understanding of DetNet services and their
requirements. As mentioned earlier, when operating at this layer,
such nodes can push, pop, or swap (pop then push) S-Labels. In all
cases, the F-Label(s) used for a DetNet service are always replaced, and
the following procedures apply.
When sending a DetNet flow, zero or more F-Labels MAY be pushed
on top of an S-Label by the node pushing an S-Label. The
F-Label(s) to be pushed when sending a particular DetNet service MUST
be provisioned per outgoing S-Label via configuration, i.e., via the
Controller Plane discussed in .
F-Label(s) can also provide context for an S-Label. To
allow for the omission of F-Label(s), an implementation SHOULD
also allow an outgoing interface to be configured per S-Label.
Note that when PRF
is supported, the same App-flow data will be sent over multiple
outgoing DetNet member flows using forwarding sub-layer
LSPs. This means that an
implementation may be sending different sets of
F-Labels per DetNet member flow, each with a different S-Label.
When a single set of F-Labels is provisioned for a particular
outgoing S-Label, that set of F-Labels MUST be pushed after
the S-Label is pushed.
The outgoing packet is then forwarded, as
described below in . When a single
outgoing interface is provisioned, the outgoing packet is then
forwarded, as described below in .
When multiple sets of outgoing F-Labels or interfaces are provisioned for
a particular DetNet service (i.e., for PRF), a copy of the outgoing packet, including
the pushed member flow-specific S-Label, MUST be made per F-Label set and outgoing
interface. Each set of provisioned F-Labels are then pushed onto
a copy of the packet. Each copy is then forwarded, as described
below in .
As described in the previous section, when receiving a DetNet MPLS flow, an implementation
identifies the DetNet service associated with the incoming packet based
on the S-Label. When a node is using platform labels for
S-Labels, any F-Labels can be popped, and the S-Label uniquely
identifies the DetNet service. In the case where platform labels are
not used, incoming F-Label(s) and, optionally, the incoming interface MUST also be provisioned for a
DetNet service.
Common F-Label Processing
All DetNet-aware MPLS nodes process F-Labels as needed to meet
the service requirements of the DetNet flow or flows carried in
the LSPs represented by the F-Labels. This includes normal
push, pop, and swap operations. Such processing is essentially
the same type of processing provided for TE LSPs, although the
specific service parameters, or traffic specification, can
differ. When the DetNet service parameters of the DetNet flow or
flows carried in an LSP represented by an F-Label can be met by
an existing TE mechanism, the forwarding sub-layer processing
node MAY be a DetNet-unaware, i.e., standard, MPLS LSR. Such
TE LSPs may provide LSP forwarding service as defined in, but
not limited, to the following: , , , , , , and .
More specifically, as mentioned above, the DetNet forwarding
sub-layer provides explicit routes and allocated resources, and
F-Labels are used to map to each. Explicit routes are
supported based on the topmost (outermost) F-Label that is
pushed or swapped and the LSP that corresponds to this label.
This topmost (outgoing) label MUST be associated with a
provisioned outgoing interface and, for non-point-to-point
outgoing interfaces, a next-hop LSR. Note that this
information MUST be provisioned via configuration or the
Controller Plane. In the previously mentioned special case
where there are no added F-Labels and the outgoing interface is
not a point-to-point interface, the outgoing interface MUST
also be associated with a next-hop LSR.
Resources may be allocated in a hierarchical fashion per each LSP
that is represented by each F-Label. Each LSP MAY be
provisioned with a service parameter that dictates the
specific traffic treatment to be received by the traffic
carried over that LSP. Implementations of this document MUST
ensure that traffic carried over each LSP represented by one or more
F-Labels receives the traffic treatment provisioned for that
LSP. Typical mechanisms used to provide different treatment to
different flows include the allocation of system resources
(such as queues and buffers) and provisioning of related
parameters (such as shaping and policing) that may be found in
implementations of the Resource ReSerVation Protocol (RSVP) and RSVP-TE . Support can also be
provided via an underlying network technology, such as IEEE 802.1
TSN .
The specific mechanisms selected by a DetNet node to ensure DetNet
service delivery requirements are met for supported DetNet
flows is outside the scope of this document.
Packets that are marked in a way that do not correspond to
allocated resources, e.g., lack a provisioned F-Label, can
disrupt the QoS offered to properly reserved DetNet flows by
using resources allocated to the reserved flows. Therefore,
the network nodes of a DetNet network:
MUST defend the DetNet QoS by discarding or remarking (to
an allocated DetNet flow or noncompeting non-DetNet flow)
packets received that are not associated with a completed
resource allocation.
MUST NOT use a DetNet allocated resource, e.g., a queue or
shaper reserved for DetNet flows, for any packet that does
match the corresponding DetNet flow.
MUST ensure a QoS flow does not exceed its allocated
resources or provisioned service level.
MUST ensure a CoS flow or service class does not impact the
service delivered to other flows. This requirement is
similar to the requirement for MPLS LSRs that CoS LSPs do
not impact the resources allocated to TE LSPs, e.g., via
.
Subsequent sections provide additional considerations related
to CoS (), QoS (), and
aggregation ().
OAM Indication
OAM follows the procedures set out in with the
restriction that only Virtual Circuit Connectivity Verification
(VCCV) type 1 is supported.As shown in Figure 3 of , when the first nibble of
the d-CW is 0x0, the payload following the d-CW is normal user
data. However, when the first nibble of the d-CW is 0x1, the
payload that follows the d-CW is an OAM payload with the OAM
type indicated by the value in the d-CW Channel Type field.The reader is referred to for a more detailed
description of the Associated Channel mechanism and to the
DetNet work on OAM for more information about DetNet OAM.
Additional considerations on DetNet-specific OAM are subjects
for further study.
Flow Aggregation
The ability to aggregate individual flows and their associated
resource control into a larger aggregate is an important technique
for improving scaling of control in the data, management, and control
planes. The DetNet data plane allows for the aggregation of DetNet flows
to improved scaling. There are two methods of supporting flow
aggregation covered in this section.
The resource control and management aspects of aggregation
(including the configuration of queuing, shaping, and policing) are
the responsibility of the DetNet Controller Plane and are out of scope for this
document. It is also the responsibility of the Controller
Plane to ensure that consistent aggregation methods are used.
Aggregation via LSP Hierarchy
DetNet flows forwarded via MPLS can leverage MPLS-TE's existing
support for hierarchical LSPs (H-LSPs); see . H-LSPs are
typically used to aggregate control and resources; they may also be
used to provide OAM or protection for the aggregated LSPs. Arbitrary
levels of aggregation naturally fall out of the definition for
hierarchy and the MPLS label stack . DetNet nodes that
support aggregation (LSP hierarchy) map one or more LSPs (labels)
into and from an H-LSP.
Both carried LSPs and H-LSPs may or may not
use the Traffic Class (TC) field, i.e., L-LSPs (Label-Only-Inferred-PSC LSPs)
or E-LSPs (EXP-Inferred-PSC LSPs , which were renamed to "Explicitly TC-encoded-PSC LSPs" in ). Such nodes will need to
ensure that individual LSPs and H-LSPs receive the traffic
treatment required to ensure the required
DetNet service is preserved.
Additional details of the traffic control capabilities needed at a
DetNet-aware node may be covered in the new service definitions
mentioned above or in separate future documents. Controller Plane
mechanisms will also need to ensure that the service required on
the aggregate flow are provided, which may include the discarding
or remarking mentioned in the previous sections.
Aggregating DetNet Flows as a New DetNet Flow
An aggregate can be built by layering DetNet flows, including both
their S-Label and (when present) F-Labels, as shown below:
Both the aggregation label, which is referred to as an A-Label, and the individual flow's S-Label
have their MPLS S bit
set indicating the bottom of stack, and the d-CW allows the PREOF
to work. An A-Label is a special case of an S-Label, whose
properties are known only at the aggregation and deaggregation
end points.
It is a property of the A-Label that what follows is a d-CW
followed by an MPLS label stack. A
relay node processing the A-Label would not know the underlying
payload type, and the A-Label would be processed as a normal
S-Label. This would only be known to a node that was a peer of the
node imposing the S-Label. However, there is no real need for it to
know the payload type during aggregation processing.
As in the previous section, nodes supporting this type of
aggregation will need to ensure that individual and aggregated
flows receive the traffic treatment required to ensure the
required DetNet service is preserved. Also, it is the Controller
Plane's responsibility to ensure that the service required on
the aggregate flow is properly provisioned.
Service Sub-Layer Considerations
The internal procedures for edge and relay nodes related to PREOF are
implementation specific. The order of a packet elimination or
replication is out of scope for this specification.
It is important that the DetNet layer is configured such that a
DetNet node never receives its own replicated packets. If it were
to receive such packets, the replication function would make the
loop more destructive of bandwidth than a conventional unicast
loop. Ultimately, the TTL in the S-Label will cause the packet to
die during a transient loop, but given the sensitivity of applications
to packet latency, the impact on the DetNet application would be
severe.
To avoid the problem of a transient forwarding loop, changes to an
LSP supporting DetNet MUST be loop-free.
Edge Node Processing
A DetNet edge node operates in the DetNet forwarding sub-layer
and service sub-layer. An edge node is responsible for matching
incoming packets to the service they require and encapsulating
them accordingly. An edge node may perform PRF, PEF, and/or POF.
Details on encapsulation can be found in ; details on PRF can be found in ; details on PEF can be
found in ; and
details on POF can be found in
.
Relay Node Processing
A DetNet relay node operates in the DetNet forwarding sub-layer and service sub-layer.
For
DetNet using MPLS, forwarding-related processing is performed on the F-Label. This
processing is done within an extended forwarder function. Whether an
incoming DetNet flow receives DetNet-specific processing depends
on how the forwarding is programmed. Some relay nodes may be DetNet
service aware for certain DetNet services, while, for other DetNet services,
these nodes may perform as unmodified LSRs that only understand
how to switch MPLS-TE LSPs, i.e., as a transit node;
see . Again, this is entirely up to
how the forwarding has been programmed.
During the elimination and replication process, the
sequence number of an incoming DetNet packet MUST be preserved and
carried in the corresponding outgoing DetNet
packet. For example, a relay node that performs both PEF and PRF
first performs PEF on incoming packets to create a compound
flow. It then performs PRF and copies the App-flow data and the
d-CW into packets for each outgoing DetNet member flow.
The internal design of a relay node is out of scope for this
document. However, the reader's attention is drawn to the need to
make any PREOF state available to the packet processor(s) dealing
with packets to which PREOF must be applied and to
maintain that state in such a way that it is available to the
packet processor operation on the next packet in the DetNet flow
(which may be a duplicate, a late packet, or the next packet in
sequence).
Forwarding Sub-Layer ConsiderationsClass of Service
Class of Service (CoS) and Quality of Service (QoS) are terms that
are often used interchangeably and confused with each other.
In the context of DetNet, CoS is used to refer to mechanisms that provide
traffic-forwarding treatment based on non-flow-specific traffic classification,
and QoS is used to refer to mechanisms that provide traffic-forwarding
treatment based on DetNet flow-specific traffic classification.
Examples of existing network-level CoS mechanisms include
Differentiated Services (Diffserv), which is enabled by the IP header Differentiated Services
Code Point (DSCP) field and MPLS label
Traffic Class field and at
Layer 2 by IEEE 802.1p Priority Code Point (PCP).
CoS for DetNet flows carried in PWs and MPLS is provided using
the existing MPLS Differentiated Services (Diffserv) architecture
. Both E-LSP and L-LSP MPLS Diffserv
modes MAY be used to support DetNet flows. The Traffic Class
field (formerly the EXP field) of an MPLS label follows the
definition of and . The Uniform, Pipe, and Short Pipe
Diffserv tunneling and TTL processing models are described in and and MAY be used for MPLS LSPs
supporting DetNet flows. MPLS Explicit Congestion Notification (ECN)
MAY also be used, as defined in ECN and updated by .
Quality of Service
In addition to explicit routes and packet replication and
elimination (described in above),
DetNet provides zero congestion loss and bounded latency and
jitter. As described in , there are different
mechanisms that may be used separately or in combination to
deliver a zero congestion loss service. This includes
QoS mechanisms at the MPLS layer, which may be combined
with the mechanisms defined by the underlying network layer, such
as IEEE 802.1 TSN.
QoS mechanisms for flow-specific traffic
treatment typically include a guarantee/agreement for the
service and allocation of resources to support the service.
Example QoS mechanisms include discrete resource allocation,
admission control, flow identification and isolation, and
sometimes path control, traffic protection, shaping, policing, and
remarking. Example protocols that support QoS control include the
Resource ReSerVation Protocol (RSVP)
and RSVP-TE . The existing MPLS mechanisms defined
to support CoS can also be used to
reserve resources for specific traffic classes.
A baseline set of QoS capabilities for DetNet flows carried in PWs
and MPLS can be provided by MPLS-TE
. TE LSPs can
also support explicit routes (path pinning). Current service
definitions for packet TE LSPs can be found in "Specification of
the Controlled-Load Network Element Service" ,
"Specification of Guaranteed Quality of Service" , and "Ethernet Traffic Parameters"
. Additional service
definitions are expected in
future documents to support the full range of DetNet services.
In all cases, the existing label-based marking mechanisms defined
for TE LSPs and even E-LSPs are used to support the identification
of flows requiring DetNet QoS.
Management and Control Information Summary
The specific information needed for the processing of each DetNet
service depends on the DetNet node type and the functions being
provided on the node. This section summarizes this information based on the DetNet
sub-layer and if the DetNet traffic is being sent or received. All
DetNet node types are DetNet forwarding sub-layer aware, while all
but transit nodes are service sub-layer aware. This is shown in .
Much like other MPLS labels, there are a number of alternatives
available for DetNet S-Label and F-Label advertisement to an
upstream peer node. These include distributed signaling
protocols (such as RSVP-TE), centralized label distribution via a
controller that manages both the sender and the receiver using the
Network Configuration Protocol (NETCONF) and YANG, BGP, the Path Computation
Element Communication Protocol (PCEP), etc., and hybrid combinations of the
two. The details of the Controller Plane solution required for
the label distribution and the management of the label number
space are out of scope for this document.
Particular DetNet considerations and requirements
are discussed in .
Conformance language is not used in the summary, since it applies to
future mechanisms, such as those that may be provided in signaling
and YANG models, e.g., .
Service Sub-Layer Information Summary
The following summarizes the information that is needed (on a per-service
basis) on nodes that are service sub-layer aware and transmit DetNet
MPLS traffic:
App-flow identification information, e.g., IP information as
defined in . Note that this information is not needed on DetNet
relay nodes.
The sequence number length to be used for the service. Valid
values include 0, 16, and 28 bits. 0 bits cannot be used when PEF or
POF is configured for the service.
If PRF is to be provided for the service.
The outgoing S-Label for each of the service's outgoing DetNet
(member) flows.
The forwarding sub-layer information associated with the output
of the service sub-layer. Note that when PRF is
provisioned, this information is per DetNet member flow. Logically,
the forwarding sub-layer information is a pointer to further details
of transmission of DetNet flows at the forwarding sub-layer.
The following summarizes the information that is needed (on a per-service basis) on nodes that are service
sub-layer aware and receive DetNet MPLS traffic:
The forwarding sub-layer information associated with the input
of the service sub-layer. Note that when PEF is
provisioned, this information is per DetNet member flow. Logically,
the forwarding sub-layer information is a pointer to further details
of the reception of DetNet flows at the forwarding sub-layer or
A-Label.
The incoming S-Label for the service.
If PEF or POF is to be provided for the service.
The sequence number length to be used for the service. Valid
values included 0, 16, and 28 bits. 0 bits cannot be used when PEF or
POF are configured for the service.
App-flow identification information, e.g., IP information as
defined in . Note that this information is not needed on DetNet
relay nodes.
Service Aggregation Information Summary
Nodes performing aggregation using A-Labels, per , require
the additional information summarized in this section.
The following summarizes the additional information that is needed on
a node that sends aggregated flows using A-Labels:
The S-Labels or F-Labels that are to be carried over each
aggregated service.
The A-Label associated with each aggregated service.
The other S-Label information summarized above.
On the receiving node, the A-Label provides the forwarding context
of an incoming interface or an F-Label and is used in subsequent
service or forwarding sub-layer receive processing, as
appropriate. The related additional configuration that may be
provided is discussed elsewhere in this section.
Forwarding Sub-Layer Information Summary
The following summarizes the information that is needed (on a per-forwarding-sub-layer-flow basis) on nodes that are
forwarding sub-layer aware and send DetNet MPLS traffic:
The outgoing F-Label stack to be pushed. The stack may include
H-LSP labels.
The traffic parameters associated with a specific label in
the stack. Note that there may be multiple sets of traffic
parameters associated with specific labels in the stack, e.g.,
when H-LSPs are used.
Outgoing interface and, for unicast traffic, the next-hop
information.
Sub-network-specific parameters on a technology-specific
basis. For example, see .
The following summarizes the information that is needed (on a
per-forwarding-sub-layer-flow basis) on nodes that are forwarding
sub-layer aware and receive DetNet MPLS traffic:
The incoming interface. For some implementations and
deployment scenarios, this information may not be needed.
The incoming F-Label stack to be popped. The stack may include
H-LSP labels.
How the incoming forwarding sub-layer flow is to be handled,
i.e., forwarded as a transit node or provided to the service
sub-layer.
It is the responsibility of the DetNet Controller Plane to
properly provision both flow identification information and
the flow-specific resources needed to provide the traffic
treatment needed to meet each flow's service requirements.
This applies for aggregated and individual flows.
Security Considerations
Detailed security considerations for DetNet are cataloged in
, and more general security considerations
are described in . This section
exclusively considers security considerations that are specific to the DetNet
MPLS data plane. The considerations raised related to MPLS networks
in general in are equally applicable to
the DetNet MPLS data plane.
Security aspects that are unique to DetNet are those whose aim is to
protect the support of specific QoS aspects of DetNet, which are
primarily to deliver data flows with extremely low packet loss rates
and bounded end-to-end delivery latency.
Achieving such loss rates and bounded latency may not be possible
in the face of a highly capable adversary, such as the one
envisioned by the Internet Threat Model of BCP 72 that can
arbitrarily drop or delay any or all traffic. In order to
present meaningful security considerations, we consider a
somewhat weaker attacker who does not control the physical links
of the DetNet domain but may have the ability to control a
network node within the boundary of the DetNet domain.
An additional consideration for the DetNet data plane is to maintain
integrity of data and delivery of the associated DetNet service traversing
the DetNet network.
Application flows can be protected through whatever means are
provided by the underlying technology. For example, encryption may be
used, such as that provided by IPsec for IP
flows and/or by an underlying sub-network using MACsec
for IP over Ethernet
(Layer 2) flows.
MPLS doesn't provide any native security services to account for
confidentiality and integrity.
From a data plane perspective, this document does not add or modify any
application header information.
At the management and control level, DetNet flows are identified on a
per-flow basis, which may provide Controller Plane
attackers with additional information about the data flows (when
compared to Controller Planes that do not include per-flow identification).
This is an inherent property of DetNet that has security
implications that should be considered when determining if DetNet is
a suitable technology for any given use case.
To provide uninterrupted availability of the DetNet
service, provisions can be made against DoS attacks and delay
attacks. To protect against DoS attacks, excess traffic due to
malicious or malfunctioning devices is prevented or mitigated
through the use of existing mechanisms, for example, by policing and
shaping incoming traffic. To prevent DetNet packets from
having their delay manipulated by an external entity, precautions need
to be taken to ensure that all devices on an LSP are those intended to
be there by the network operator and that they are well behaved. In
addition to physical security, technical methods, such as authentication
and authorization of network equipment and the instrumentation and
monitoring of the LSP packet delay, may be used. If a delay attack is
suspected, traffic may be moved to an alternate path, for example,
through changing the LSP or management of the PREOF configuration.
IANA ConsiderationsThis document has no IANA actions.ReferencesNormative ReferencesInformative ReferencesDetNet Data Plane: IP over MPLSDetNet Data Plane: MPLS over IEEE 802.1 Time Sensitive Networking (TSN)Deterministic Networking (DetNet) Security ConsiderationsIEEE Standard for Local and metropolitan area
networks-Media Access Control (MAC) SecurityIEEEIEEE Standard for Local and metropolitan area networks--
Frame Replication and Elimination for ReliabilityIEEEAcknowledgements
The authors wish to thank ,
, , ,
, , , , ,
, , , and for their
various contributions to this work.
Contributors
The editor of this document wishes to thank and acknowledge the
following person who contributed substantially to the content of this
document and should be considered a coauthor:
LabN Consulting, L.L.C.dfedyk@labn.net