Unified AI Networking

Traditional networking stacks (such as InfiniBand and RoCE) address parts of this challenge, but none deliver a truly end-to-end, workload-aware, topology-aware, multi-data center fabric.

zFABRIC, part of the zSUITE Platform, is designed as the AI-native networking fabric that unifies all three scaling dimensions into a single, software-defined control plane.

As AI systems grow to trillion-parameter scale, GPU clusters must support three simultaneous capabilities:

1. Scale-Up: deterministic, high-bandwidth intra-node/cabinet communication
2. Scale-Out: predictable, lossless data center wide networking
3. Scale-Across: multi data center connectivity for GW-class AI factories

Existing networking technologies, including InfiniBand and RoCE, solve fragments of today’s scaling challenges but stop short of full unification. zFABRIC, within the zSUITE Platform, delivers an AI-native fabric that brings together all three scaling dimensions under a single, software-defined control plane.

zFABRIC unifies:

• Scale-Up: GPU topology discovery and optimization
• Scale-Out: AI-tuned Ethernet with deterministic performance
• Scale-Across: Cross–data center routing and orchestration
• Optional Optical Circuit Switching (OCS) for long-lived AI flows

zFABRIC transforms GPU networks from static transport layers into an intelligent, dynamic resource optimized explicitly for AI training and inference workloads.

1. AI Scaling: The Three Dimensions Reframed by zFABRIC

Modern AI infrastructure expands along three axes:

zFABRIC binds these three layers into a unified operational fabric.

2. zFABRIC for Scale-Up: Making GPU Topology First-Class

Scale-up networks (NVLink / NVSwitch / UALink / PCIe) are essential for:

• Tensor Parallelism (TP)
• Expert Parallelism (MoE)
• High-frequency memory semantic operations.

‍2.1 Industry Background

Scale-Up fabrics feature:

• Sub-100ns latency
• TB/s-level bandwidth
• Tightly coupled hardware architectures

However, training frameworks often treat GPU topologies as opaque, causing:

• Suboptimal ring-tree collective construction
• Uneven communication patterns
• Poor MFU (Model FLOPs utilization)

2.2 zFABRIC’s Role in Scale-Up

(A) zFABRIC Topology Service

Automatically learns:

• NVLink graph
• NVSwitch cluster boundaries
• PCIe/host affinity
• NUMA mappings

(B) zFABRIC Intranode Scheduler

Uses real-time telemetry to determine:

• Ideal TP/EP placements
• GPU pairing for fused attention blocks
• Path selection inside SuperNodes (NVL72, MI300 pods, Gaudi racks)

(C) ZCCL Topology-Aware Collectives

zFABRIC’s collective library builds optimized rings and trees based on:

• Available GPU paths
• Predicted congestion
• Link utilization

Scale-Up becomes visible and programmable, not a hidden hardware detail.

3. zFABRIC for Scale-Out: Deterministic AI Ethernet Fabric

Scale-Out connects thousands of GPUs across a data hall. Challenges include:

• PFC head-of-line blocking
• Unpredictable tail latency
• Multi-tenant interference
• Complex manual tuning (ECN, buffer profiles, DCQCN)

zFABRIC replaces fragile manual tuning with an autonomous AI fabric.

3.1 zFABRIC DCQCN+ - Advanced AI Congestion Control

Enhances RoCEv2 with:

• Latency-driven congestion windows
• Dynamic queue carving
• Per-job virtual lanes
• Adaptive flow pacing

Benefits:

• 40–70% lower tail latency
• Near-zero packet loss
• Improved AllReduce throughput

3.2 zFABRIC Fabric Controller

Provides an intent-based network operating plane:

• Automated switch tuning (ECN thresholds, buffer allocation)
• Consistent queue profiles across thousands of ports
• Real-time congestion heatmap
• Proactive congestion mitigation

3.3 ZCCL (zFABRIC Collective Communication Library)

Designed for AI training traffic:

• Topology-aware
• Congestion-aware
• Supports heterogeneous NIC/GPU environments

Outperforms NCCL/RCCL in large-scale clusters by 5–18%.

4. zFABRIC for Scale-Across: Multi–Data Center AI Fabric

AI factories increasingly span multiple data centers due to:

• Power delivery limits
• Cooling constraints
• Land availability
• Resiliency requirements

Standard networking lacks:

• WAN-aware collective routing
• Bandwidth slicing across regions
• Job-aware cross-DC scheduling

zFABRIC introduces a global fabric controller that extends training workflows across multiple regions.

4.1 zFABRIC DCI Fabric (ZDF)

zFABRIC’s scale-across layer includes:

• Cross-DC transport abstraction
• Distance-aware rate adaptation
• Latency-based path selection
• Cross-region collective orchestration (AllReduce, MoE routing)

4.2 Global Fabric Controller (GFC)

Handles:

• Multi-DC routing policies
• Bandwidth quotas for tenants
• Cross-site job placement (power, cooling, network availability)
• Global topology discovery

This allows AI operators to treat multiple data centers as one logical GPU pool.

5. Optional Layer: zFABRIC + OCS for AI Optical Super-Fabrics

Although ZFABRIC does not require OCS, it integrates OCS as a performance accelerator for clusters with large, predictable flows. AI workloads produce:

• Long-lived AllReduce paths
• MoE expert dispatch routes
• Inter-stage activation transfers
• Large checkpoint flows

These are ideal for circuit-switched optics.

5.1 Why zFABRIC Integrates OCS

OCS provides:

• Bufferless switching (no queuing)
• Ultra-low energy consumption
• Scalable port counts
• Ideal characteristics for elephant flows

zFABRIC’s OCS Integration Layer can:

• Pre-provision optical circuits for training phases
• Assign circuits based on ZCCL’s upcoming collective schedule
• Dynamically re-optimize circuits during job transitions
• Ensure strict bandwidth/QoS for multi-tenant workloads

This transforms the optical network into an active component of the AI training engine.

6. zFABRIC VS Traditional Fabrics

7. The Road Ahead: zFABRIC for Global AI Factories

AI factories will soon consist of:

• Millions of GPUs
• Multi-GW power envelopes
• Regional compute clusters
• Cross-DC MoE training paths

zFABRIC is built for this scale:

• Autonomous congestion control
• Global path orchestration
• Topology-aware collective engines
• Optional OCS acceleration
• Compatibility with heterogeneous GPUs and NICs

zFABRIC becomes the “AI-era network OS” that coordinates compute across nodes, clusters, and regions.

8. Conclusion

zFABRIC delivers the first unified AI networking fabric spanning:

• Scale-Up (deterministic intra-node fabrics)
• Scale-Out (AI-tuned deterministic ethernet)
• Scale-Across (multiple data center orchestration)
• Optional OCS acceleration for elephant flows