View the summary slides:

Everything as an API

Backstory

Long-gone are the days of the technology monolith. The technology stack that was owned completely by a single company. Fully closed-source. Pay up or hit the highway. Microsoft. Oracle. Adobe.

Today the average company has technology woven into every facet of the business. From the basics of finding customers or making a website to heavy machine learning analytics and payment processing. This complexity makes it impossible to build everything yourself, and why would you? Simply buy a world-class solution off-the-shelf and move on. Even a startup will have 20-50 different tools that they pay for monthly. Letting someone else solve everything but your core specialty enables focus on only the most valuable tasks. This abstraction of busywork has made it easier than ever to start a company.

Solution

In a world of increasing complexity, interoperability becomes the new ideal. How can similar processes be bundled up so that they each don't steal away attention? The answer has pretty clearly emerged as APIs (application programming interfaces). An API can be thought of as a pathway that provides the core value of a software product, whatever that may be. They can be very small and specific, or large complex beasts that perform major operations. They run almost everything on the internet and, arguably, are just getting started.

An API takes the form of a standard black box system:

Request → [operation] → Result

Requests are made to an endpoint (URL) that is publicly known, such as example.com/api/login. This lets a server know what operation it is expected to perform, in this case login. Requests typically have data that are sent along with them, such as a username and password.

I won't belabor this point as there is a wealth of information available on APIs, but they are very important to the future when everything is virtual and not on exclusively the web. APIs allow services to talk to each other efficiently.

As an interface, they provide business actions without exposing the actual logic behind what is going on. To process payments all you have to add to your website is the Stripe API, to send a text you can use Twilio, to launch a missile from a drone you call the [redacted] API. Easy.

A truism in technology is that value is often delivered through abstraction. If you can make someone's life easier by handling complexity for them, you are saving them time and money. So APIs are the way we connect services, sell abstraction B2B, and merge with the physical world.

Software is often described as either bundling or unbundling. Where it delivers value through abstraction. An API has the special ability that it allows for both bundling and unbundling*.* You can use one part of an API to perform a single operation, or roll many APIs together to provide a completely new business value. (This is also the fundamental concept behind the no-code movement.)

💵 CHAPTER 2 — The Metaverse

Right now there are some standards for APIs that are well known, but yet they still range greatly in end-design. We are at most halfway to interoperability. What happens when we allow many services to work together? Well, that is where the metaverse springs to life. What is the metaverse? It sounds like some wacky silicon valley term, but don't lump it in with other buzzwords just yet. I define a metaverse as a persistent system where everything is able to understand everything else. A virtual world of real-time shared experiences offering radically lower structural limitations. A place that builds off of real-life abilities, yet greatly expands potentials. If that is hard to imagine, then you are on the right track, as this doesn't exist today. But it will. People have been thinking about this concept for years, but the actual construction is just starting now.

There have been a few popular discussions on this topic lately, for instance a great podcast by Patrick O'Shaunassey and Matthew Ball discussing the How Epic Games is making bets on the Unreal Engine and to gain early marketshare into this space.

https://open.spotify.com/episode/3RKopMLwVWwvOQZ6Y8Ts7w?si=4cjzOnKTSwSsr3JCU8RSZQ

While I recommend listening to the full episode, a high-level summary goes like this:

• Epic created the Unreal Engine, an impressive piece of 3D graphics software that allows for virtual worlds to be modeled. It powers many major games, such as Fortnite.

  From their website – "Unreal Engine is the world’s most open and advanced real-time 3D creation tool. Continuously evolving to serve not only its original purpose as a state-of-the-art game engine, today it gives creators across industries the freedom and control to deliver cutting-edge content, interactive experiences, and immersive virtual worlds."

• They are making it cheap for other developers to use their engine, not just for building games, but to lay the groundwork for more open-world use cases (transactions, events, exploration).

• Fortnite, the large-scale shooter game, has been used to host concert and screen movies.

While I genuinely loved the discussion, I want to play the devils advocate for a moment. To me the frequent association of Fortnite as being version one of the metaverse does it a disservice. As soon as you map a future concept to current known, such as Uber for X, people tend to stop thinking bigger. This holds true for any not-yet-ready topic and largely is why so many people have different takes on blockchain tech. When talking about a breaking technology the truly valuable potentials are often only realized when thinking beyond what exists today. This is uncomfortable, time consuming, and often avoided. When people argue that we have metaverses here now, my knee-jerk reaction is to disagree. A product that people use for tangential or unintended reasons is still just a product. Having 10 games and apps that all communicate and work together on the Unreal Engine? That sounds like the v1 metaverse. Just like with the internet, there will be many generations of metaverses as the concept matures. /rant

What are the main features?

Let's break down the core attributes of the metaverse. While the definition of what qualifies as the metaverse can be argued for days, the basic concepts will hold everywhere. To build a virtual world, you need to make sure it can be widely accessed, stays consistent in experience, and doesn't artificially limit what can be made.

Said another way it needs to be accessible, persistent, verifiable, and interoperable. (get ready to hear these words more in the coming years). These properties are dependent on one another.

1. Access

If you are building a model of the world in bits, there is no reason to artificially limit access to it. This is a problem of language, laws, and expectations. How can you deliver a consistent experience that everyone has a chance to participate in? This is one reason decentralized systems (blockchains) are frequently mentioned when discussing the metaverse. No one entity controls them, which makes it harder to ban or control. ie. Bitcoin is worth the same amount in every country.

2. Persistence

The internet today is a hodgepodge of live and asynchronous activities. The web doesn't depend on the physical time to deliver its contents. The metaverse on the other hand will never sleep. It will have the concept of time baked-in. This means that the current state of the virtual world will always be changing. In this environment it becomes very important to make sure the right things persist across time. If you buy something in the metaverse, you need to make sure that it will still be there next time you want to use it. You need to be you no matter how you log on, where you are, or what actions you take. Persistence is not only personal, it needs to be spatial. In a virtual world, gathering in at a specific place needs to be identifiable and able to be revisited over long periods of time. This is a challenge considering how often people update software.

3. Verification

People in real life can prove who they are. They have birth certificates, bank accounts, SSNs, and unique physical characteristics. Identity forms the foundation for the multitude of trust-based activities we take for granted. You can loan a friend your car because you know who they are and how to find them. In a digital-only world this is a hard problem. Related to this is the concept of digital uniqueness, where something is provably finite. This is called non-fungiblity (a mouthful for sure), and it provides the ability to truly 'own' a one-of-a-kind digital good. This concept is often the headline feature of blockchains.

4. Interoperability

How can you make a system widely accessible when it will require depending in thousands of individual resources? When you buy a product in real life, you can use it anywhere, and in any way you please. How can you take a digital good from one place to another? How can companies around the world build applications and experiences that can be work together and not reinvent the wheel? Objects in the metaverse should work when you interact with them. This is a classic problem faced by video games, that will get worse when there is no storyline to follow. A truly open world could require unending engineering effort.

Putting the pieces together

If this still seems too abstract, you are not alone. No one knows what the metaverse will end up looking like, we can only make our best guesses about the characteristics that fill form it. Note that I did not include any mention of graphics, look and feel, or hardware platforms when describing the metaverse. That is because I consider the most pure interpretation to be a data layer, with the final rendering performed by any device a user wishes. You should be able to have a text-only metaverse experience. Right now there are no wrong answers.

The easiest way to picture this is to think of the current internet, which is also a data layer. The metaverse is basically an evolved web, where operations happen in shared time. That is to say that there is no refresh, no resets , it just keeps running for everyone all the time. The best analogy I have is when you are done playing a video game you power it off (time stops), when you are done with your day in real life you go to sleep (time keeps running).

The metaverse will effectively be a network of interconnected services that allow for users to perceive and take actions in shared-time.

Platforms are often referred to as metaverses as they provide the summation of services (lowest level of single point abstraction)

Hardware

To connect to the metaverse we can use any type of hardware, from a cell phone to AR glasses. The likely starting point will be similar to video games, with computers and consoles providing rich experiences. Mobile devices, like the Nintendo switch or phones will provide on-the-go access to the world.

Platform

The platform is what the metaverse actually looks like to the user. This will likely be a game engine that can display complex graphics, such as Unreal or Unity (more on these later). The platform layer will also handle user interactions with metaverse services, such as displaying products, handling payments, storing and displaying data, etc. These services will be what most people will name when referencing the metaverse, as they are the obvious visual representation.

What does this enable?

The benefits from a well structured, common data layer are innumerable. With exceptions, if you can do something in real life, you should be able to do it in the metaverse. Everything from exploring new cultures and meeting people to operating a business. You should be able to have a virtual meeting with colleagues, take an engaging tour of another planet, or walk (or fly) into a subreddit.

The metaverse will be the king of strong network effects, where every person that joins makes it better. This beneficial flywheel effect will work very well to help it exponentially gain traction as it grows. Unlike social networks, a virtual world will have tangible benefits beyond just communication. You will be able to use the metaverse alone and still find it interesting, useful, and entertaining.

This is the next evolution of how the content will be produced and consumed. We used to be purely content consumers (movies), then we became one-to-many content creators (Instagram), now we are entering a period where we will participate in the content. This is naturally a more engaging environment for most people, where they will be encouraged to be proactive in what they consume, how they discuss it, and what actions they take.

The multi-metaverse

Until now we have discussed how the metaverse would look if there were only one, idyllic virtual world where everyone interacts. This is the world consumers want; a shared-experience environment where everything just works. Sadly, this is far from realistic.

Owning distribution in a market is incredibly valuable. Today, companies like Apple and Google own the access points to major app stores and profit handsomely from doing so. This inescapable incentive for control will push every major company to create their own version of a metaverse. Unless someone makes a big bet on open-source development, there is almost no way that we don't have dozens of entrants vying for a slice of this pie.

Why is this bad? The issue lies in the unprecedented scope a metaverse. Imagine a major company that has the ability to place a fee on every aspect of your digital life. This anti-consumer potential is why many people are hesitant to start building. But competition is a good thing, right? The problem of having many independent multiverses is in developing and enforcing standards to ensure the core features are upheld. Fragmentation makes everything harder to understand, develop, and instill strong conviction. As shown below, with a multiple metaverses both users and service providers will have to model their lives in multiple worlds. Without a standardized protocol for communication between these virtual spaces, development would be painful and expensive.

This situation has strong parallels with how cryptocurrencies went through a boom and bust period. Where intense interest and promise was met with unending complexity and reinventing the wheel.

Edit: this piece by Matt Ball also does a great job at breaking down the metaverse in a similar fashion. I wish I had read it before writing this essay!

Cool, but how is this about AI?

The more data that exists, the more machine learning is required to make sense of it. Creating, monitoring, and profiting from the metaverse will involve many types of systems to be in place. Let's look at some opportunities where AI is likely to drive incredible value for those that build it.

🏗️ CHAPTER 3 — Building the Metaverse

The translation layer

While the game-engine opportunities have been fairly well covered, I personally think the more interesting infrastructure concept revolves around a generic translation layer.

Web standards are not that well followed. Modern development is more fragile than people admit. There will be mission critical activities done inside the verse. It and the services supporting it can’t just crash for a day or two. There is no way that we can force every developer to learn the details of Unreal Engine, and Unity, and <insert every other multiverse>. To unlock the full value of the metaverse we need to be able to turn messy inputs into sensible outputs. A system for abstracting how to do something into simply what is being done.

Example–

You are 'in' the metaverse and you pick up a phone to send an SMS message. That phone should work. It should connect to the physical world.

In the real world in order for your phone to work you would need to subscribe to a carrier service, get a number, buy a phone, activate it, and make sure people associate that number with your name. In the virtual environment, that phone should simply be hooked up to APIs that can fulfill your request. For example Twilio, or anyone else could be the end processor of the message. It shouldn't matter. And this is the opportunity. To build the service that makes it invisible to the end-user what service actually fulfills any given request.

This is a hard problem. Actually, quite a few hard problems. There are likely hundreds of thousands of APIs out there that each have unique operations they perform, oddball structures they are formatted into, and hard-to-read documentation. Enter natural language processing (NLP), ML systems that are much better at handling text information than a human ever could.

This would translate user actions in actual actions, without specific programs being developed. Without user training. Can you imagine the value that could unlock? I can't.

This is easy to just postulate without any deeper thought, but I want to stress that in theory it should be possible right now. It just requires significant work.

It would require:

• An NLP model trained to recognize user intent and break down inputs into actions. (imagine Alexa, but with deeper software knowledge)

• A bidding market where service providers offer their bids to fulfill a request. (much like how display ads work)

• A universal digital payment method. (Bitcoin)

• An NLP model that is trained on the API documentation of all the service providers in the bidding market.

• A cache that can store all common user actions.

Even a simpler version of just the last bullet could provide serious value, offering a service such as:

Companies (each with an API) ~> [some learning AI] ~> standardized interoperable formats

Someone please try to make this. it represents a classic 'big swing'; what venture capital was truly meant to enable. (If this is you please email me)

Follow-up edit: Over the course of writing this essay this concept quickly became the most fascinating aspect of the metaverse to me. I dove deep into this and developed an MVP implementation of the 'Interpret' section using OpenAI's GPT-3. It works surprisingly well. This validation warrants follow-up work which I am currently dropping other projects to pursue. This is well beyond the scope of this essay, so if you are interested please email or DM on Twitter.

Before leaving this section it is worth thinking about how a system like this would impact brand strength. In the metaverse, users might not need to care what notes app they use or which brand of cell phone has the fastest specs. These features could simply be skins on-top of commodity APIs. On the flip side, if the metaverse gets implemented in a less-generic way, we will likely be presented with a never-ending stream of in-app purchases.

Infinite content dilemma

Going a different direction, an unending 'game' that models life also poses interesting content problems. With high-quality original content (HBO, Netflix) costing hundreds of millions and taking years to make, how can we possibly produce enough to satisfy an always-on world full of players? With synthetic media. Content that is created by machines without any human intervention. While I have never seen synthetic media that is good enough to qualify as TV-grade, the rate of improvement is staggering. I am lucky enough to have an inside look at this ramp up by working at Generated Photos, a company that is producing content using GANs (generative adversarial networks). As you can see from the images below, captured over one year, the realism gap is closing. From first-hand knowledge I can tell you that in the last 6 months the most interested parties have been major game companies. There is no reason to assume that the metaverse will look like a kids game (a la Fortnite), even in early versions. I personally would love to work in a Mandalorian-esque environment with real faces rather than wacky avatars.

Three generations of 100% AI-generated images. Fall '19 → Winter '19 → Spring '20

Non-player characters

What happens in a virtual world at night when most people log out? Will it seem like a ghost town? How can the user experience be kept consistent across so many potential interactions? This will likely fall to another AI system, natural language processing, to take in the slack. Not the stupid 'bots' that you see in video games today, but open-ended creative personalities. We already have NLP characters that can have articulate conversations of their own making, the next step forward here will be in making characters that have defined personality traits and stay in canon. This will take a significant amount of processing power ($$$), but should be possible within the next year. By using modern language models like GPT-3 we can provide details to characters that would be impossible to add by human creation.

More specifically, we can start with a seed of truth and grow an entire persona using AI. I have worked on this myself and it works well. Text generation language models, such as transformers, need to be supplied context in order to know how to make sense of incoming requests. In order to produce a character that stays consistent it is required to 'set the stage' for the system. This means that a certain number of events are created and pinned to the characters identity in a database. When an interaction happens with the character a search algorithm is quickly run to select the ideal context to supply to the text model. From this context a character has nearly infinite possibilities in which to fills in all the gaps when discussing with a human.

TL;DR - We are very close to having virtual characters that are able to hold entertaining and engaging conversations.

UGC

In a real-time world, a user simply participating will provide a basic form of user generated content. The next generation of media creators will have the potential to work with brands or directly offer content to a near infinite base of potential customers.

In the very first variants of the metaverse we will see traditional content that can be consumed in a virtual world. Interactive concerts, movies, podcasts, exercise, education... all of it. Existing social media companies will have a fast on-ramp to get started here.

Content Discovery

With a real-time world it will be a difficult problem to surface content for users as they are playing. This will certainly be done with complex algorithms, much like TikTok uses. Content discovery will cross-over from social apps to practical applications in the metaverse, as people natually gravitate toward the medium with the highest information density. Think text chat → voice → video call → interactive shared experience. This means that the problem set is stunningly large, and will be only possible to solve by individual companies. In a way this will likely hamper the metaverse into looking more like a VR app store in the early days, until more generalized search and user data collection methods are created.

The data opportunity

Think about how the data collected from the internet has shaped our culture and created unparalleled fortunes. This current level of data collection is a tiny sliver of what will be possible in the metaverse. Everything will be data. Every action you take, every purchase you make, EVERYTHING. The potential for analyzing this data has no bounds. From predicting crime to underwriting credit. Better suggestions for content, products, friends, relationships, jobs, education, ... this is the deepest rabbit hole to go down.

With mountains of new data being emitted services will need to scale up to storage, security, sales, and analysis. Major companies like Tencent are already positioning themselves to have access to the companies building the first blocks of the metaverse. This could give them a massive data advantage later on. This is analogous to how Tesla has a lead in self-driving simply because of the proprietary data they have collected and can now use. Investing in companies that are building the first blocks of this world takes a generalized approach.

Data as the new UBI?

The metaverse could reasonably pose the first opportunity for 'data as personal property' to gain widespread adoption. As the value of this data will be very high, individual users will desire to profit from allowing companies to access their information. Giving people a way to earn a living in the metaverse will likely be one of the fastest ways to drive adoption. Just like in the real world, people will seek entertainment, profit, status, and comfort in the metaverse. Building programs and infrastructure that support these basic human traits has proven to be the winning strategy.

💵 CHAPTER 4 — Investing in the Metaverse

Early money

The great value is created in the early days of a disruption: the dot com boom, mining multiple bitcoin in a day, 'theres an app for that'. Today, the metaverse is just emerging in its most nascent form, video games. We aren't far from having a functioning v1 metaverse running inside a game engine. I expect to see a strong platform candidate emerge in the next 1-2 years, with full-featured metaverses taking shape in ~5 years. If you started a venture fund today, by the time the first vintage closed the early benefits of the metaverse will have already been captured.

Macro Trends

Vertical Businesses

As infrastructure begins to be more accessible, businesses will be able to create increasingly specific niche software that to serve customers. This simply means that every type of business will be able to have a tailored software solution instead of taking a general piece of software (say Excel) and customizing it to match your needs. Less custom engineering, more no-code.

Monopolies under threat → Acquisitions

Just as with any other major change, such as the introduction of the internet, many existing companies will be left behind. Old-world business models such as search engine page rank, web display ads, and simple UGC will be at risk

It has shown to be hard for the major tech corporations to keep up with innovation. With such lucrative potentials for the metaverse, established brands stand more to lose than others. This should incentivize them to make strategic acquisitions earlier, rather than let upstarts gain majority share.

While most large companies will actively work on the metaverse, often is their core revenue drivers that are put in harms way from the start.

Apple - at risk: services revenue

Facebook - at risk: connection graph value, data uniqueness

Google - at risk: web search (page rank) and display advertising

Revenue capture shifts from platform to creator

The metaverse should lower the barriers for people looking to earn money. You don't have to be famous to sell digital goods. The cost for Nike to produce a pair of shoes for a virtual character are the close to anyone elses. As a digital money system, there will be more direct transfers of wealth. Digital fame will have incredible value. Creators could well be the super-companies of the future. Building and interacting with an audience will almost certainly be more common.

Public Market

Near-term hardware and software

While we have discussed the 'verse as a data layer, it is important to realize that most people will interact with the it through visual effects. In this regard video game engines have the expectation to capture market share through their ability to render complex environments. Both Epic Games and Unity have made it clear that they are moving into the metaverse. These engines have multiple practical applications beyond the metaverse, such as interactive movie production, making them highly interesting investments. Specifically, the upcoming Unreal Engine 5 holds promise to make it much easier to create large, intricate environments.

Compute

The is no escaping that the metaverse will be computationally demanding. As discussed already, we know that stunning amounts of data will need to be generated, stored, analyzed, and kept safe. Right now experiences in games such as Fortnite are practically limited by compute regarding the number of people that can actually share in a simultaneous experience. It isn't hard to imagine a full metaverse becoming the largest consumer of compute ever. This means that investing in cloud service providers –now– offers a non-zero investment opportunity into the metaverse.

Identity

Who owns identity? Right now companies such as Google, Apple, Facebook, Microsoft, Amazon and others own billions of identities along with social graphs. Payments and commerce shops Paypal, Stripe, Shopify, Venmo, and CashApp all have access into the financial side. These lists are too long to call a winner on just yet. If there emerges a leader that can bundle these weak identifies into one stronger variant that would be a very attractive opportunity. This bundling could also be formed by an outsider, for instance in the form of a blockchain token.

Visual Hardware and Spatial Computing

The experience of the metaverse will only be as good as the device that can render it. Many people think of virtual or augmented reality headsets when speaking of the metaverse, the most likely near-term interfaces. Facebook is making high-profile bets here with Oculus, with initiatives that directly approach both working and playing in a virtual environment.

Apple said to be working on AR glasses, though they will likely not support engaging experiences required for immersion. Currently Microsoft and Magic Leap are the leaders in immersive AR, often dubbed mixed reality, with the Hololens and Lightwear. These devices are early interpretations of what users will likely wear to interact with the metaverse. Unsurprisingly, both of these devices are made to rendering the Unreal Engine and Unity.

https://youtu.be/eAagtcAup0o?t=250

https://youtu.be/5_bVkbG1ZCo?t=28

!https://s3-us-west-2.amazonaws.com/secure.notion-static.com/47d38184-5e56-42dc-abda-2256b41133b2/Untitled.png

Private Market

Regarding early-stage investment, the purest way to gain exposure to the metaverse will be through companies taking known behaviors and directly applying them to the virtual world. API-first virtual education, payments, communities, content creation, or social media. Zoom, Twitch, and even Youtube Live all have aspects that resemble the metaverse. Creating multiplayer technology to extend services such as these should naturally transfer to early iterations of the metaverse and produce rich acquisition targets.

It is also worth gaining some exposure to high-risk bets on companies looking to become the leaders in creating the large infrastructure elements for the metaverse. These are your protocol developers, synthetic media companies, NLP model producers,

Dark horse: Open Source

Who will be the first company or fund to go all-in on the metaverse; backing the creation of an major open-source universal data layer? If an effort like this was started early enough it could front-run the entry of major tech companies. Similar to how bitcoin is still the dominant cryptocurrency, after being first AND open source.

Aside: The open source community is powered by git which interestingly is accessible, verifiable, interoperable, and persistent. The git wiki reads: "[git] is designed for coordinating work among programmers, but it can be used to track changes in any set of files. Its goals include speed, data integrity, and support for distributed, non-linear workflows". Though it does not have the concept of real-time, perhaps this is the true minimum viable metaverse?

Going forward

Hopefully this helped shed some light on the potential value to be unlocked the next decade. This topic is a large part of why I am creating this site today. AI and data science are interwoven into every facet of our digital future. It is hard to imagine that gaining more operational knowledge in these areas will prove to be a waste of time. If you found this article interesting, I encourage you to start thinking more about APIs, NLP, and how they can all work together. The shift won't look like an overnight virtual world, you have time to help shape the tools that build value.

This essay could be 20x the length and still not cover all the topics it should. Instead I will leave you with a few question to keep you thinking.

• How do we avoid the pitfalls faced by blockchain? How do we get the media companies on board right away, and leading with the killer app.

• How can partnerships be created to reduce fragmentations?

• How could the metaverse enable remote work?

Ethical topics to ponder:

• How will this virtual society be structured?

• Will the metaverse be global? How does nationalism impact a universal data layer?

• In a virtual world, who makes or enforces laws?

• How can a virtual world be structured so that the gains won't just be captured by large companies?

• How much will rent on data persistence be?

• Does digital gentrification pose a serious threat?

Editable diagrams are available at figma.com/@tyler