The move to 400G per lane

Electrical lanes inside switches are going from 200 to 400 Gb/s. USFP, XPO's roadmap and every co-packaged design are bets on how optics keep up.

Every switch chip talks to its ports through SerDes: circuits that send data as fast serial signals over copper traces, one lane at a time. The lane rate sets the pace for everything around it. Lanes of 100 Gb/s gave the industry 800G ports; 200G lanes give the 1.6T ports AI clusters are buying now. The next step is 400 Gb/s per lane.

What doubles and what doesn't

A chip with the same number of lanes at twice the rate has twice the bandwidth: 512 lanes of 400G is a 204.8T switch, against 102.4T for today's 512 lanes of 200G. At the same port speed it has twice as many ports, and that's the point. A switch with more ports (a higher radix) can connect more endpoints before you need another layer of switches.

What doesn't double is the room on the front of the box. A 1RU switch fits about 32 OSFP ports across its face, the OSFP MSA's own reference design (OSFP MSA). At 1.6T each, that's 51.2T per rack unit. To put a 204.8T chip's bandwidth on the front panel you need four times the bandwidth per opening, which is exactly what USFP's 16 lanes of 400G give: 6.4T in roughly the space of a 1.6T OSFP.

How the power claim adds up

Cisco and NVIDIA say USFP is about 1.3 times as power-efficient as 200G-per-lane optics. NAND Research reports the reasoning behind it: a 400G SerDes lane moves twice the data for roughly 1.5 times the power of a 200G lane, and 2 divided by 1.5 is 1.33 (NAND Research). Fewer, faster lanes means less energy per bit, if the 400G SerDes hit that power target. Nobody has measured a USFP module yet, because none exists.

The silicon isn't here yet

No one has announced a 400G-per-lane switch chip. At the same OCP Global Summit where USFP was launched, Cisco's new chips were the 102.4T Silicon One G300 and the 51.2T G302, both 200G per lane, and NVIDIA's Spectrum-6 also runs at 200G per lane (NAND Research). USFP is a module designed ahead of the chips that will need it, which is normal: OSFP-XD was specified long before most switches used it. It does mean any USFP timeline depends on the 400G SerDes timeline.

Four ways to get there

Put the optics in a bigger pluggable (XPO), a denser pluggable (USFP), a socket next to the chip (Open CPX) or inside the chip package itself (co-packaged optics). Each trades serviceability, cooling and power differently.

  • OSFP 1.6T
  • OSFP-XD 3.2T
  • USFP 6.4T to 12.8T
  • Open CPX 7.2T
  • XPO 12.8T to 25.6T
One-direction capacity per module (per socket for Open CPX) at the lane rate each format has announced. The pale bar is the announced roadmap: USFP at 32 lanes, XPO at 400G per lane. Sources: the OSFP MSA, Cisco, Open CPX MSA, Arista and NAND Research.