The Ultra Ethernet Consortium’s UE Specification v1.0 has profoundly enhanced the building of Ethernet networks for cutting-edge AI and HPC applications. Next-generation AI and HPC deployments have already adopted many of the v1.0 spec techniques.
Today, we’re excited to share that the Ultra Ethernet Consortium (UEC) is building on that success by releasing the UE Specification v1.1. This new revision introduces many updates, including:
- Broad improvements to tackle scale-up networking
- UE Transport enhancements for application compatibility and faster bulk transfer
- In-Network Collectives that offload compute to the network
- Congestion Signaling (CSIG) for congestion control and telemetry
- Programmable Congestion Management for the most flexible congestion control
Scale-up Networking Support
Most substantially, the v1.1 release includes multiple optimizations to support scale-up networking. One major focus is reducing packet sizes to make small packets more efficient. A reduced inter-packet gap decreases the number of idle bytes that must be inserted between packets, saving as many as 11 bytes.
The Unified Forwarding Header (UFH), a new network header that overlays on top of a traditional Ethernet header, saves 20 bytes compared to a traditional IP packet. At the same time, it maintains full IP routing functionality and supports full wire interoperability and simultaneous operation with traditional Ethernet.
The new “Unified Local” profile targets scale-up endpoints and includes a simplified transport protocol that supports write, read, and atomic operations. This protocol uses optimized headers that minimize packet overhead, while remaining wire-compatible with all Ultra Ethernet Transport (UET) standards.
UET Enhancements
A central goal of v1.1 is to reduce migration and deployment costs by preserving application-layer compatibility and semantics. UEC created a Verbs mapping for UE (UET Verbs) that supports the AI Base profile and introduces two new transport-layer features.
The first feature provides more flexible memory mapping, allowing implementations to use memory-region resources more efficiently across logical services within an application. The second introduces synchronization groups, enabling UE applications to replicate the in-order responder completion delivery semantics of existing Verbs. Together, these changes mean that existing applications can be ported to UE with fewer modifications.
In-Network Collectives (INC)
Traditional all-reduce algorithms require sending data into the network twice: The initial values are first reduced, then the reduced values are broadcast to all nodes. To produce a 1 GB reduced result using this traditional approach, each endpoint must send 2 GB of data. With INC, an endpoint needs to send only 1 GB of data, yielding up to a 2x communication speedup. Network switches reduce the data as it is received and broadcast the reduced data back to the endpoints.
In addition to reducing bandwidth consumption, INC can also significantly reduce the latency of small collective operations. Broadcast methods such as trees require log n hops through the network, with each hop adding latency. By using network broadcast, INC reduces this per-hop incremental latency cost, thus improving collective performance and dramatically boosting the overall performance of both HPC and AI applications.
Congestion Signaling (CSIG)
Congestion Signaling (CSIG) defines a compact 4- or 8-byte Layer-2 tag attached to packets in the network. As packets move through the network, switches update this tag with high-fidelity information about bottleneck congestion experienced by the packet along the path. Endpoints receiving the tag learn the magnitude of the congestion and the exact location where it was experienced. With this precise in-band telemetry, congestion control algorithms can adjust more accurately to the network’s actual state.
UE’s congestion control algorithms (NSCC and RCCC) now both consume CSIG telemetry. NSCC leverages CSIG-reported queueing latency at a path’s bottleneck hop, which enhances the fairness of bandwidth sharing. RCCC uses CSIG measurements of either network delay or queue depth to account for network congestion when granting credits to senders. Beyond congestion control, CSIG’s fine-grained data enables sophisticated network telemetry that helps operators understand, diagnose, and optimize their networks. More details on CSIG’s role in both congestion control and telemetry can be found in the ACM SIGCOMM paper.
Programmable Congestion Management (PCM)
Looking beyond today’s supported congestion control algorithms, UEC recognizes a need for customizable congestion control that end users can program to fit their use cases. Programmable Congestion Management (PCM) provides a unified language and set of APIs that NIC vendors expose to users. With these APIs, NIC vendors can develop new and optimized congestion control and load balancing algorithms, tailored to a user’s workload and compiled to run on endpoints. The standardized PCM APIs allow these custom algorithms to be ported to any NIC with PCM support. More details about PCM’s philosophy, design, use cases, and applications can be found in this paper.
UEC Collaboration
UEC is working with OCP and IEEE to help bring these v1.1 technologies to fruition. The OCP ESUN 1.0 specification includes many of the technologies described in the UE Specification v1.0. We’re now actively working with OCP to bring v1.1 scale-up technologies to the next ESUN specification. Additionally, we’re collaborating with OCP SUE-T to build out expanded endpoint support. Finally, UEC is partnering with the IEEE to further standardize CSIG.
We’re very excited about the advances in the v1.1 specification. We encourage you to read it here. And as ever, UEC welcomes new members who would like to contribute ideas to make Ethernet networks even better.
Written by Tom Emmons
Chair, UEC Technical Advisory Committee