Congestion control for AI Ethernet: a fix that helps in simulation and one place where it still loses

Training a large AI model sends huge bursts of data between thousands of chips at once. When the network jams, the slowest transfer sets the speed of the whole job. Networking vendors have been designing new congestion control for this. We worked in a public simulator of the Ultra Ethernet design (uet-htsim), with an existing published method called MNSCC as our starting point. We have no connection with Cisco or any other vendor.

A one-line change cut the typical completion time by 3 to 37 percent in five simulated traffic setups, and it lost slightly on incast. It is a simulation result with novelty not checked.

Before measuring, we wrote down the hypotheses and the pass criteria. Our first idea, to adapt how much the controller smooths its delay signal, failed in two designs. Neither worked on the incast pattern, where many senders hit one receiver. We recorded that as a negative result.

In round three we looked at why one design sometimes lost badly on a single seed. In one case the slowest flow had delay below target, yet 72% of its packets were marked as congested. The marking threshold (40 KB of queue) sits below the target queue (about 100 KB), so a mark says only that the queue is above 40 KB. The controller then cut its window about 36% every round trip. That explains this case from traces. It is not a proof for all cases.

The fix, frozen before testing on fresh seeds: cut the window only if the median delay is also at or above target. Nothing else changed.

Source: REPORT3.md, validation on seeds not used in design. Labels are the simulator's own scenario names (ele = cross-traffic setups at 16 and 8 paths, failed8 = with failed links); their full definitions are in the original report. 20 paired seeds in total: 5, 5, 3, 3 and 2 per setup. No single seed lost in these cross-traffic setups. That is true only for these 20 seeds.

Incast (3 seeds)

Mean completion 2530 against 2455 for the adaptive baseline (+3.1%) with local placement, and 2507 against 2450 (+2.3%) with spread placement. Max completion about equal (+0.3% and +0.4%). The plain NSCC method is still best on incast mean (2382 and 2444). Our frozen criterion was plus or minus 2%, so this FAILED.

Worst single seed on incast

+11.8%, down from +18% before the fix.

• Simulation only: an idealized fat-tree with 128 or 1024 nodes and 2 to 5 seeds per setup.
• MNSCC is a published method, not ours. Whether our variant is new is unverified. The closest published ideas we found are APSCC and STrack.
• The remaining incast loss may come from the size of the cuts. That is a hypothesis from traces.
• Not a breakthrough. No hardware and no vendor involved.