This article explores the pros and cons of point-to-point versus structured cabling in AI clusters, particularly in light of increasing bandwidth demands. It distinguishes between training and inference GenAI clusters, emphasizing the high computing and storage needs of training clusters and the importance of rapid deployment and operational efficiency. The article highlights the impact of Return Loss (RL) on network signal quality in fiber connections, especially in high-bandwidth environments. Additionally, it compares copper and fiber in terms of latency, noting that while copper has lower latency, it is limited in distance. The next blog will cover deployment, installation, power, cooling, and sustainability aspects.

The last couple of months there has been a lot of noise about the expected boom of 400Gb, 800Gb
and 1.6Tb in the next 2-3 years. Yet it seems only yesterday we made the jump of 40Gb to 100Gb.
Similar with latency where requirements increased from us to ms, yet latency in some of my latest
project, related to the gateways to the cloud, was in ms. And I thought we were quite advanced in
these things, was I so wrong or behind in my assumptions?
Figure 1: 2021 Lightcounting study on transceiver speed market growth
I think there are a few aspects currently making noise that need to be put in the right perspective.
Indeed there is advanced requirement around AI with increased bandwidth demand and low
latency expectations. But is this going to impact every aspect of the data center?
First of all the AI clusters will become the brain of the IT and you will still need the customer facing
applications that will run in your DC or cloud environment. Secondly not all applications have a
need for AI, e.g. the application responsible for paying your wages once a month, does not
necessarily has to be AI driven. Third there is the difference between training and inference, where
the amount of AI clusters needed to train models is a multiple of the hardware needed to apply
the inference. All this will impact the amount of AI hardware needed, so were are the sudden
expectations of massive 800G and beyond transceiver sales over the next couple of years come
from.