Last week was all about inference in AI and new players emerging as forces to be reckoned with in the space. For the last few years, the entire industry has been obsessed with training'stacking thousands of H100s to teach a ghost how to speak. But this week, the vibe shifted. We are moving from a world where we spend billions to create intelligence, to one where we spend billions to serve it. Leading the charge is BaseTen, who just announced a monster $300M round at a ~$5B valuation. Interestingly, NVIDIA is writing the check. BaseTen isn't trying to build the model; they are building the plumbing. Their bet is that inference is the new 'cloud computing''a utility that needs to be boring, reliable, and infinitely scalable. They are effectively saying: 'You bring the weights, we'll handle the nightmare of GPU orchestration.' While BaseTen handles the macro-infrastructure, two other players emerged this week to handle the micro-optimization. First up, we have RadixArk. If you've been...
learn more
