In partnership with

In this edition:

1. The building blocks of a GPU datacenter future, in map form

2. Thoughts on investing in compute infra + thesis

MADE POSSIBLE BY

Startup PromoTix ($48M in traction) is saving the events industry

PromoTix is solving the event industry’s challenges around high ticket fees and low attendance. Ticketmaster and competitors charge up to 40% of the ticket price to book, deterring guests who can’t afford the added cost. Combined with a crowded marketing space, events struggle.

PromoTix is raising funds to expand. Already profitable, with 656k users and $48M in sales in its first 30 months, PromoTix has low-fee and no-fee SaaS pricing, as well as patented marketing tools that drive attendance.

See investment terms

FOLLOW THE MONEY

Building the Future of Compute

Clearly, a lot goes into building, managing, and utilizing a datacenter.

The map I’ve put together isn’t comprehensive; the groupings are thematic and may evolve with market entrants.

Some thoughts

• A common question: GPUs are quickly evolving. What happens when a datacenter is loaded with GPU version x and version x+1 releases a year later? Are those GPUs a sunk cost?

  • While frontier hardware is rapidly advancing, customers still seek good-enough, discounted solutions. Nvidia’s latest enterprise GPU is the H100, with the B100 expected soon (delayed). Yet, the A100 (released in mid-2020) is still in circulation and actively-sought in compute pools.

  • Datacenter hardware has always had limited lifespans. Servers last 3-4 years, storage devices last 4-6 years, networking equipment lasts 5-7 years, and cooling lasts much longer. Cooling—particularly water cooling—is an explosive topic in R&D.

Hosting and providers: Why include Canonical/RedHat? What’s Nvidia doing in this list? Why is this category useful at all?

Canonical supports the open-source operating system Ubuntu, a distribution of Linux. Most servers run on Ubuntu, and server maintenance is the job of a department.

Overlook the software component of GPU hardware at your peril. Engineering teams will often rewrite entire kernels just to optimize GPU workloads. Nvidia’s greatest hold on the GPU world is CUDA, an interface allowing applications to easily utilize graphics hardware. While developers have worked with CUDA for years, Nvidia has been developing lower-level software to bridge GPUs in enterprise settings. They know the power of vendor lock-in. For this reason alone, they belong in the software category.

On HuggingFace—I see them as an emerging GitHub for machine learning. GitHub’s value proposition to developers cannot possibly be understated. Code, organizations, workflows, rulesets, containers, and artifacts are just the beginning. HuggingFace has always been extremely developer-friendly. Now, they’re leaning even further towards pain-alleviation with storage and hosting.

I hope to update this map over time. Please share any thoughts with me at [email protected].

THESIS

Investing in Infra

Some of the biggest GPU clouds are raising swaths of cash—naming names is unnecessary. Let’s start with themes before moving into a final thesis.

Overspending in CapEx

Last week I outlined my perspective on the supposed “overspending” in AI infrastructure.

• Why it’s happening: Initial demand overload, power games amongst oligopolistic incumbents (see: The Truth About GPU Overspending).

• Why it matters: Those trying to time the market are weary of investing at high valuations, from within a “bubble.” (And, never forget the Robinhood traders well-versed in buying at the peak.)

• Why it doesn’t matter: Believe in the future of compute and invest in a vision of the world at that point. Timing is unimportant.

Relationships

• Given the scale of the first-mover advantage in this market, companies with strong provider partnerships are well-poised to dominate infrastructure. Such partnerships, i.e. frequent handshakes with Nvidia, allow cloud companies to settle into their customers’ stacks early. In a market of desperation, companies need service providers, and new entrants will have to work harder to sell themselves going forward.

Lifecycle Integration

• The most competitive GPU cloud providers offer more than compute services. They focus on several aspects of the AI model lifecycle: gathering, annotation, selection, training, tuning, demonstration, inference, feedback, re-training/re-tuning.

• Earlier in this edition, we discussed Nvidia’s software efforts. The best cloud companies work with providers directly to maximize hardware optimization and stay ahead of the efficiency innovation curve.

• Microsoft Azure, Google Cloud, and Amazon AWS have deep vendor lock-in given the sheer depth of their services. Take Google, for example. Connect to storage buckets from code directly. Push code to GitHub, triggering a workflow to Cloud Build, building a container and sending it to Cloud Artifacts. Start a routine cleanup on Cloud Run Jobs, before deploying to the web with Cloud Run, pulling from Cloud Artifacts. A forehead tattoo emblazoned with “Google” is in order.

• GPU cloud providers hoping to compete with the big three have an incredible opportunity today. Customers are dissatisfied with hyperscalers’ allocations, pricing, uptime, development timelines, and support. Yet, hundreds of billions of dollars are going into monolithic compute apparatuses. The clock is ticking.

The Global Opportunity

• The Middle East is on a trajectory to invest trillions of dollars into local compute infrastructure. Energy, to them, is almost free. Similar eastern countries, particularly in cold climates, have an incredible opportunity to power compute for the rest of the world.

• The portability of AI training: The beauty of a remote training job is the measly output size. Throughput—data shared between GPUs working side-by-side—is gargantuan in comparison, but an entirely separate issue. This simple fact unlocks the ability to run training tasks abroad with no concern for latency.

• The best GPU clouds are already working with foreign countries to establish compute outposts around the world, taking advantage of local energy dynamics. Frankly, these countries are much less capable of facilitating complex, large-scale datacenter buildouts alone.

Reservations

• A convenient dynamic of the infra industry is the forecastability of revenue. The biggest customers (Fortune 500s and large model trainers like Google, OpenAI, and xAI) have to reserve compute. Thus, compute providers are clued into the primary composition of their incoming revenue.

• Simply put, the best GPU cloud providers are already trusted by the biggest spenders.

Of course, there’s plenty more to this topic, but these themes should suffice for the moment. They, along with thoughts from the first section and from last week’s edition on compute overspending, lead nicely into a thesis.

ABOUT RTF

Return the Fund 🚀 

• One startup a week poised for 10x growth; market deep dives with actionable insights for builders and investors.

• Technical breakdowns of advanced new tech to empower informed decision-making

• Find your next prospect, product, job, customer, or partner. 🤝 Written by undercover pioneers of the field; trusted by builders and investors from Silicon Valley to NYC. 💸 

Last week, we dove into the supposed “overspending” in AI infrastructure.

As you know, we’re hell-bent on uncovering future unicorns cruising under the radar. Preeminent companies are lean, quiet, and driven before reaching their watershed moments. By the time people start talking about them, it’s too late.

You won’t get editions from us very often. Weekly at best. Two reasons:

1. We’re balancing full-time VC/startup work with Return the Fund.

2. We prioritize depth, insight, and value. This is not a daily news publication… We hope that when you do get an email from us, it’s dense, valuable, actionable, and worth saving.

Thanks for reading today’s RTF. Feel free to reach out to us at [email protected]. 🤝 

Psst: None of our company picks are ever sponsored. All research and opinions are completely held by the Return the Fund team.