skip to Main Content

Ad Interim: Provisional Thoughts on AI, Issue #3

Toward a Roadmap for Infrastructure 5.0

Welcome to the third edition of Ad Interim: Provisional Thoughts on AI. In these letters we document the rapidly-changing AI landscape and lay out how Inner Loop’s Digital Infrastructure investment thesis is evolving and expanding to meet these new opportunities. These letters come out approximately quarterly, and we welcome your engagement and feedback on these topics any time.

Significant technologists have compared the emergence of AI to some of the biggest developments in human history. AI has been said to be as important as electricity, and more profound than fire. With these claims coming from eminently credible sources, it would seem there is little room remaining for a Solo GP of a Seed VC fund to stake out a unique bull case on AI. And yet in Ad Interim Issue #3, I attempt to do exactly that.

In the second issue of this letter (October 2023), I stated my belief that AI does indeed represent a technology platform shift, at least as significant as Cloud Computing, and perhaps as impactful as the Internet itself. In today’s letter I will explain why I believe the future of Digital Infrastructure investing is driven by the convergence of three distinct but interrelated developments. The software that supports these three simultaneous platform shifts I (provisionally) refer to as Infrastructure 5.0.

While other investors and tech leaders are certainly tracking and commenting on parts of these developments, there is a key aspect that I do not think is getting enough attention in today’s GPT-driven Gen AI frenzy. Nor do I see enough investors addressing the combination and interplay between these three interrelated trends. This is what I set out to do in today’s letter.

Neither Artificial Nor Intelligent

I began these letters in April 2023 with some light-hearted skepticism about the “intelligence” component of the current generation of LLMs. I also count myself as a skeptic that we are on the verge of anything like Artificial General Intelligence (AGI). On the whole, I don’t expect to see AGI in my lifetime, nor do I expect it to enter the investing universe of any Inner Loop fund. I feel this view is somewhat validated by Sam Altman’s recent comments that he no longer sees scaling LLMs as a path to AGI, and that he now believes other breakthroughs will be needed to reach that goal.

Like the inaptly named Holy Roman Empire, the very phrase Artificial Intelligence continues to fall flat for me. The systems generally available to the public today strike me as neither Artificial nor Intelligent. They are certainly useful and impactful, and indeed represent a significant break from the classical technology stack we all interact with every day.

There are three main underlying innovations which enable today’s AI systems: the efficient management of mind-bogglingly large data sets in software, the efficient processing of those data sets in GPUs that utilize a parallel-processing architecture, and the advent of transformer-based deep learning models. These three innovations are no more artificial or intelligent than any other part of the information technology stack broadly in use today. But it is a new stack, which is why I believe we need to think very broadly about the implications of this platform shift.

The term “AI” to refer to this tripartite innovation is now firmly established. I cannot buck it, and so I embrace it. But when I do, for my own purposes, I am more comfortable thinking of AI as “Augmented Intelligence” or better yet “Accelerated Inference”. In the first case, Augmented Intelligence emphasizes that most decisions and actions taken with AI will involve a human in the loop, either before or after the insight is generated. The machines can propose ideas or constructs we hadn’t quite considered, or efficiently increase the scope of human intelligence. “Accelerated Inference” reminds me that what these systems do at their core is a very large, very efficient regression. They are fitting a curve to the data we have provided, sometimes in specific domains, sometimes more broadly.

Naming conventions aside, I do continue to believe that the wide availability of powerful Accelerated Inference capabilities will have profound effects on our digital and physical worlds for decades to come.

Defining Infrastructure 5.0 (more on this next time)

I do not want to dedicate the space in this letter to fully flesh out the term Infrastructure 5.0. In brief, I have defined six major phases of enterprise computing, beginning with the mainframe. I assigned the mainframe as stage zero, in part because these monolithic systems did not create much opportunity for third-party infrastructure software, the main sector that Inner Loop invests in.

Continuing through PCs (1977), Client-Server (1980), Internet (1994), and Cloud Computing (1999), I defined the four major platform shifts in enterprise computing and the independent infrastructure companies that supported them, from Computer Associates (founded 1976) and Quest Software (1987) to Crowdstrike ($72 billion market cap) and ServiceNow ($161 billion).

I now firmly believe that AI is driving a fifth major platform shift in enterprise computing that will be at least as profound as any other, and perhaps more so. I define the infrastructure software opportunities to support this shift as Infrastructure 5.0.

However, I further believe that Infrastructure 5.0 means more than just finding out how enterprises will use LLMs and supporting them on that journey. This platform shift will be much more profound than that. Infrastructure 5.0 is about supporting three simultaneous technology trends that, combined, will re-define most of the established world of information technology.

The Three Trends Driving Infrastructure 5.0

Inner Loop’s view is that there are three distinct transformations happening in the global computing stack simultaneously, the confluence of which creates the opportunity for what we call Infrastructure 5.0. Despite the intense focus on AI by the VC, tech commentariat, and futurist communities since the public launch of ChatGPT in November 2022, we have not yet seen anyone else describe and integrate these three trends the way we see them. In order of how far along they are (more to less) the three trends we identify are:

1) Digital Transformation Accelerates and Incorporates AI

The first trend is the ongoing Digital Transformation of work and life, while integrating newly available AI tools into its software, workflow, and tech stack. Investing in the infrastructure for this ongoing Digital Transformation has been the core thesis of Inner Loop since our pilot fund in 2019, the basis for my personal angel investing 2015-2018, and the primary recurring theme for my venture investing career going back to my start at Bessemer Venture Partners in 2001. I have described this investing thesis as Digital Infrastructure, defined mainly as Cybersecurity, Cloud Technologies, and Data Science. It underpins the vast bulk of the 36 investments Inner Loop has made from its funds since 2019.

The first slide of the Inner Loop pitch deck has been the 2023 quote from Orlando Bravo, founder of Thoma Bravo with $117 billion under management, that “Digital Transformation is Just at its Infancy.” The Inner Loop thesis has long been that if that sentiment is correct, then the tech world will be investing in and absorbing as-yet unlaunched infrastructure products for the rest of my investing career.

Many of these products in recent years have included an aspect of AI to them, but this focus has clearly increased since the ChatGPT shockwave. As I have enumerated in prior issues, most of Inner Loop’s Digital Infrastructure companies are now finding ways to integrate these newly available tools into their own products, accelerating their ability to add value to their customers. This trend is clearly continuing, in that effectively all new start-ups in Digital Infrastructure are finding ways to be AI-first innovators, integrating these models into their product designs from day one.

Given the increasing power of Accelerated Inference, new Digital Infrastructure products are increasingly more impactful and more intuitive than ever. This is increasing their value in customer’s networks and in my opinion, will inflect upward the growth rate of these industries, creating ever more opportunities to disrupt incumbents and establish new dominant platforms. Inner Loop believed that Digital Infrastructure was one of the best opportunities to deploy venture capital even before modern AI tools were broadly available. I now view this opportunity – a combined $500 billion market, growing at double digit rates – as a generational investment opportunity.

For over a year, Inner Loop has been steering more and more toward being an AI-first investor. The first issue of this newsletter launched in April 2023. The reason for this is merely because I believed that in order to fully pursue our traditional Digital Infrastructure thesis, it would be necessary to be conversant in how start-ups and customers in this space would be using AI going forward.

Jeetu Patel, EVP and GM of Security and Collaboration at Cisco, expressed this same sentiment recently in relation to Cisco’s $28 billion acquisition of Splunk. Compressed for brevity, Patel said: “In order to be a world-class security company, … you have to be really good at AI.” Since I believe this is equally true for world-class Digital Infrastructure investors, Inner Loop undertook a broader commitment to AI in late 2022.

For much of 2023, this is how I described Inner Loop’s approach to AI. It is a new catalyst to drive demand for Digital Infrastructure and a new tool available to founders to build valuable, disruptive Digital Infrastructure companies. I believe a great Seed VC practice can be built around this single theme alone for the next decade. More recently, however, it has crystallized for me that the full opportunity is bigger even than that, due to two other simultaneous trends.

2) Ubiquitous Deployment of AI Requires All-New Infrastructure Layers

The second trend is the coming deployment of new infrastructure to support delivering AI everywhere, efficiently, and at scale. This trend has not been overlooked by VCs and some established firms are building whole new practices around it. Menlo Ventures calls this AI Infrastructure, or the Modern AI Stack, and has just made a new GP hire to pursue the opportunity more fully. Mayfield calls this Cognitive Plumbing, and draws direct analogies to the venture opportunities that arose from building the plumbing layers for the Web (1994+) and the Cloud (1999+). Both of these visions resonate with me and closely map to how Inner Loop has begun to pursue opportunities in AI Infrastructure.

The Market Maps for AI Infrastructure from Menlo and Mayfield indicate just how early we are in this process.

It is obvious that they are significantly sparser than the more established sectors that Inner Loop invests in: Cybersecurity, Cloud Tech, and Data Science. The market maps for those sectors below are from Momentum Cyber, Redpoint, and First Mark VC, respectively.

There is already a land-grab underway in AI Infrastructure, and a $30m Solo GP Seed Fund is not well-positioned to participate in land-grabs. However, the market map for Cybersecurity 25 years ago looked more like the AI Infrastructure one above, and 1999 was not the last chance to make good Cybersecurity seed investments. Indeed, the sector produced a whole generation of strong seed-stage investments for the next quarter century. I largely expect AI Infrastructure to play out in a similar way, and I am positioning Inner Loop to engage with this opportunity over the long term.

While the AI Infrastructure land-grab swirls around us, Inner Loop will stay focused on the same type of start-up and deal structure we have always pursued. We won’t be interested in building tech for its own sake, even if there is strong indication that advanced AI tech could be snapped up by platform companies in the short term. We will seek to engage with AI infrastructure companies that solve existing or near-horizon customer needs in ways that generate revenue in early years. Consistent with our traditional investment goals, we want to participate in investment rounds of $2-8m that enable provable milestones over 18-30 months, increase enterprise value, and qualify the company for larger rounds of venture capital.

In today’s AI Infrastructure world, this does not describe the majority of investment rounds that are happening. However, Inner Loop strongly believes that this is a generationally long game, and opportunities to build great companies on rational milestones in this space will become available to us for decades to come. We are laying the groundwork now to do the work we have always done in Digital Infrastructure, now in the AI Infrastructure space, for multiple fund cycles going forward.

3) The Underlying Compute is Changing, and a New Stack Needs New Infrastructure

The third trend driving Infrastructure 5.0 is one that we think has not gotten nearly enough attention in the VC or tech worlds so far. It is the re-invention of the core compute infrastructure underpinning the modern technology world. Put another way, what disruptions and opportunities will the first architectural change in computing in over 75 years create?

Recent predictions of other core tech stack changes have fallen flat. Some investors and futurists have suggested that blockchain/crypto or quantum computing could represent the next breakthrough in computing architectures and fundamentally change how humans store and process information. Inner Loop has been consistently skeptical of both of these theses, and has repeatedly declined to back any crypto or quantum company for the past nine years.

However, the GPU revolution is not speculative. It is happening in front of our eyes in full view and with public reporting. Jensen Huang (CEO of Nvidia) has painted a clear picture of where the global computing world is headed. We only need to think through the eventual implications of that in order to see the full scope of the Infrastructure 5.0 opportunity.

Roughly combining Inner Loop’s second and third trends, Huang has stated that two computing shifts are emerging at once: Accelerated Computing and Generative AI. Huang defines Accelerated Computing as the hybrid hardware layer of CPUs and GPUs, and the market trend toward that is itself accelerating. Huang’s keynote at the 2023 Nvidia developer conference is worth watching in its entirety to absorb the full scale of this opportunity.

As anyone following the public tech market knows, Nvidia sales have exploded upward in recent years. From just under $11 billion four years ago (fiscal year ending Jan 2020), Nvidia sales in the current fiscal year just ended are expected to reach over $54 billion. The average analyst forecast for fiscal 2025 (12-months forward from now) is over $86 billion. This is a 51% CAGR over five years, practically unheard of in the hardware space.

This growth has vaulted Nvidia over the past two years into the ranks of the tech mega-caps, alongside Microsoft, Apple, Alphabet, Amazon, and Meta. These are the six largest U.S. public companies, and the only six valued at over $1 trillion in market cap.

Largest U.S. public stocks by market cap, mid-day Tuesday, January 30, 2024.

Nvidia is close to displacing both TSMC and Intel as the largest revenue semiconductor producer globally.

Jensen Huang explains the opportunity for Accelerated Computing by estimating that there is $1 trillion worth of hardware deployed in global data centers, the vast majority of it based on CPUs. He believes that over four years, large portions of that will turn over in favor of a hybrid GPU/CPU architecture, suggesting the trends above will continue for some time.

However, this market share shift is very different from the standard ebbs and flows of tech competitors. The GPU architecture is a fundamentally different computing model than the CPU, which has been the workhorse of global compute since the demise of vacuum tubes. An entire stack for enterprise and cloud computing has been based on the CPU. I’m willing to believe that large portions of that stack will port gracefully to a new Accelerated Computing architecture. But I also believe that changing out the hardware will create new bottlenecks, new inefficiencies to solve, and new opportunities throughout the stack.

This is the part of Infrastructure 5.0 that I do not see enough investors paying attention to. I find this surprising because the Founder and CEO of the world’s highest-valued tech hardware company is explicitly telling us where we are headed. AI-first software is changing the canonical tech stack from the top down, and GPU-based Accelerated Computing is changing it from the bottom up. A $500 billion market of infrastructure software has been built to support the traditional stack of: CPUs, large (but not impossibly large) data, and classical application software. If the new stack will be: GPU/CPU hybrids, impossibly large data sets, and Accelerated Inference-based applications, then large portions of that $500 billion of infrastructure software revenue could be up for grabs.

Imagining a New World of Ubiquitous AI

The history of technology teaches us that killer apps demand a new infrastructure build cycle, which then enables new experimentation and prompts the creation of new previously-unconsidered applications. This is a tech build cycle in Accelerated Inference infrastructure and applications that we are undertaking now.

I believe we are now building a world whereby tech that has been available for the last decade only to the Hyperscalers (Google, Alphabet, Facebook, Apple, Microsoft) will become as ubiquitous and as democratized as the Internet. As that happens, use cases we aren’t imagining yet will become commonplace, as occurred at the advent of the Internet, Mobile, and Cloud Computing platform shifts. Software has not finished eating the world, but Accelerated Inference based software will soon be influencing an enormous share of all economic value.

I describe this new opportunity as “Hyperscaling-as-a-Service”, powered by Infrastructure 5.0. I will be adding detail to this vision in future issues of this letter and elsewhere.

Over the past year, I have become increasingly convinced that this will be the investment theme that drives Inner Loop Capital through the rest of my investing career. I sometimes think of this as my Last Roadmap. It’s a privilege to be learning and exploring an investment theme as massive as this. It’s an honor to collaborate with Founders, Friends and Limited Partners who want to access and shape this new world.

Justin Label

Managing Director

Inner Loop Capital

Back To Top