US (enterprise/kubarnetes) datacenters also average 5% gpu and 8% cpu utilization. Much of datacenter buildout is for “reserving” capacity, even if existing infrastructure could accomodate "spot rental"or “serverless” (let google/aws cram your work request into the machine of their choice) to get 6x+ more “tokens”.

The reason you can’t buy RAM anymore is that “projections” are 16gw+ of AI deployment in US this year requires 70% of RAM to be for AI. 5gw is a practical ceiling for projects currently in active development. NVIDIA not only is growing its undelivered inventory at huge rates ($30B latest), its customers have $150B in “Construction in process” inventory as they aren’t getting transformers and utility hookups to finish/power on their datacenters. The circular financing by NVIDIA is just forcing their customers to shift unused GPU inventory into their warehouses. It eventually leads to less new sales/manufacturing of their GPUs, and then hopefully, RAM price normalization.

Here in malaysia there are a lot of projects that havent yet been started. My customers are involved to the scale of largest project in company history. After an exhilirating boom, now its just meh. Apparently the hyperscales are having trouble finding tenants beyond the big tech.

😬