Replacing Search?: Why Google Won’t Give Up in the ChatGPT vs. Gemini Battle

It’s more fun if you read these first:

Microsoft Bing, emerging as Google’s challenger with ChatGPT

A new paradigm in search: What does it cost to run ChatGPT?

One out of three Koreans has tried ChatGPT. Since the launch of ChatGPT-3 in 2020, interest has been so high that everyone has at least once searched for “AI stocks,” and worries have poured out that AI will replace people’s jobs, or that AI will rule the world like in The Matrix or Terminator.

For some, it’s an entertaining thought experiment; others shrug it off as premature and don’t bother paying attention…

Today, let’s talk about this massive wave of technological change and the war being waged among the tech giants.

Regardless of whether AI ends up ruling the world 100 years from now, if you’re an investor living in the present, it only makes sense to pay attention to a technological shift that’s drawing the world’s full attention.

At the very least, this boom will last longer than the metaverse craze, and the Ponzi schemes that spin off from it—like the Luna/UST collapse—will be far fewer. This is a real technological shift.

- Bill Gates, Reddit

When we think of AI, the first thing that comes to mind is, of course, OpenAI’s ChatGPT.

It gives remarkably good answers on all kinds of topics, and it feels so human that you might wonder, “Is there an actual person in there?” Building an AI that human-like has required an enormous amount of money.

So what exactly makes it so expensive? Are we paying the wages of part-timers frantically Googling answers inside the ChatGPT black box?

If the most expensive part of making sushi is the fish, then for AI the big cost is training.

You probably have a specific image in mind when you hear the word “training.” That image is exactly why “training an AI model” is such an apt expression—no other word captures it quite as well.

Training an AI model means repeatedly feeding data into an algorithm, checking the results, and adjusting the model’s output. Through this process, the algorithm being trained starts to recognize patterns, make reasonably good decisions, and gradually reduce its errors.

Let’s think about ChatGPT. What kind of AI is ChatGPT?

ChatGPT is an AI that chooses the most suitable word.

Imagine training a model like this:

Using basic logic, we can tell right away that the word that goes in the blank is “right.”

At the very beginning of training, before it has fully learned the patterns, the model will spit out random words. For example, instead of turning left, it might turn trampoline, turn swimming, or turn puppy.

After repeated data input and some learning, it will gradually start to insert words that actually make sense. Words that can come before “turn.” Things like “the other side,” or words that indicate a place…

But there is still error. That’s because the blank is preceded by extra information like “left” and “instead of.”

The iterative process continues until the model outputs the correct word, “right.” That is essentially how an AI model is trained.

Conceptually, that’s the idea. Unfortunately, the technical report does not cover the key details of the training method, so we will stick to just what we need to understand and move on.

- The ‘AI company’ backed by the world’s most astonishing tech firms

If you keep grinding away like this, you eventually end up with an AI model. Even at a glance, though, you can tell this grind is incredibly hard, in reality it’s even harder, and it requires a truly massive amount of data. So how much did it all cost?

The training cost for ChatGPT-3 was $4,300,000.

And the training cost for ChatGPT-4 was a staggering $78,400,000, far beyond that.

But there is a model that had more than twice the money poured into it than GPT-4: Gemini Ultra.

Gemini Ultra is an AI model developed by Google DeepMind. It is the successor to the previously announced “Bard,” which was rebranded under the new name and has been continuously developed since.

Gemini Ultra’s total training cost was $191,400,000, which is 2.44 times more than GPT-4.

They went ahead and built that Gemini Ultra, taking on more than twice the spending of ChatGPT-4. So how does it actually perform?

Unfortunately, the reviews have been quite poor both at home and abroad.

Let’s compare it using a few of the standard benchmarks for large language models (LLMs).

According to The Strategy Deck:

MMLU 5-shot

GPT-4: 86.4% / Gemini Ultra: 83.7%

MMLU: A benchmark that measures how broadly and deeply a model can understand a wide range of topics within a language.

DROP F1 score

GPT-4: 80.9% / Gemini Ultra: 82.4%

DROP: A benchmark for understanding narrative details and reasoning within a text.

HellaSwag 10-shot

GPT-4: 95.3% / Gemini Ultra: 74.4%

HellaSwag: A benchmark for commonsense reasoning about real-world dynamics, and cause-and-effect relationships.

The only area where it clearly beats ChatGPT-4 is in certain reasoning tasks, while it lags significantly in basic commonsense.

The video also points out that hallucinations are a serious issue. Here, AI hallucination refers to false information generated by large language models (LLMs) like ChatGPT or Gemini that is not grounded in real data.

When you ask the ChatGPT-4o model about a made-up incident called the “AWARE Lab coin fraud case,” it responds that it cannot verify such an event. That is exactly what you would expect, because AWARE considers trust its most important value.
In contrast, in the video, Gemini goes so far as to fabricate details about an event that never happened and presents them as an answer (2:09).

To be fair, it performs somewhat better when you ask questions in English, but it is hard to deny that it still needs a lot of improvement.

If you held even a single share of Alphabet (Google), this would have been infuriating to read.

But once you understand how Alphabet actually makes money, it starts to make sense. There’s a backstory behind this apparent waste of money.

It’s because the search market is under threat. Let’s try Googling something. Not something simple like the capital of the United States or tomorrow’s weather, but a random question like “What is the average height of buildings in the United States?”

If you type that into Google as is, you’ll see things like “List of skyscrapers in the United States,” “Top 10 tallest buildings in the U.S.,” and “Heights of skyscrapers” at the top of the page.

If you ask ChatGPT the same question, it will say something like, “It varies greatly depending on the type and location of the building,” then go on to explain that “there are low-rise residential buildings of just a few dozen meters, while many downtown buildings exceed 200 meters,” and that “you would need to refer to data from the U.S. Geological Survey (USGS) for a classification system and detailed statistics on building heights.”

It may not be the exact answer the questioner had in mind, but it is the kind of answer they actually need. It solves one of the biggest pain points of search: the overwhelming volume of information. It naturally leads to the thought: “Wouldn’t it be more convenient for people to use ChatGPT instead of Google?”

Google controls 90% of the search market. And 77.8% of Alphabet’s revenue comes from advertising, most of which is generated through its search engine—Google. If people start turning to ChatGPT instead of Google when they want to look something up, Alphabet’s advertising revenue will inevitably take a meaningful hit.

If you’ve read the previous article, this should click right away.

Remember when Microsoft’s CEO said this?

You might think he was just taking a cheap shot at a rival. But if you look closely, you’ll see that Alphabet’s advertising share of revenue has been declining since 2020, the year GPT-3 was released. Total revenue has been growing every year, yet advertising’s share of that revenue has been shrinking. That suggests Alphabet agrees with what Microsoft said and is already building long-term strategies around that view.

Believe it or not, this may be coincidence—but if we interpret it, it points to something familiar: AI is slowly reshaping how companies think about revenue strategy in the search market.

Once again, 77.8% of Alphabet’s revenue comes from advertising via its search engine. Think about how 70% of the human body is water. For Alphabet (Google), losing advertising revenue is like our bodies losing water: eventually you go into dehydration. It may not be a problem today, but if you can clearly see a future 5 or 10 years out where dehydration sets in, you start preparing a prevention strategy now.

Here’s a natural question. Even if AI gives good answers, doesn’t it still need to use a search engine to scrape the information users need? If Google charged a fee for that process, couldn’t it at least partially offset the decline in ad revenue?

1. ChatGPT to Google: Honestly… I don’t really need you

First, ChatGPT doesn’t fetch information in the way most of us imagine.

GPT-3 was trained on data up to 2021, and GPT-4 on data up to April 2023.

On the surface, it looks like it’s searching on behalf of humans, but technically, that does not mean it is actually performing a search in our place. It simply combines “words that are likely to fit under the underline” based on data, using an algorithm trained on massive amounts of information.

Our hypothesis was built on the premise that ChatGPT needs Google. If ChatGPT says it doesn’t need Google… that effectively means Google has nothing to hold on to.

2. Birds of a feather: GPT sides with Bing

Of course, users demand better technology every day. For now, LLMs may not strictly need search capabilities, but the latest paid versions of GPT already offer browsing. In other words, they really can stand in for search.

Here’s the catch: ChatGPT does its browsing through Microsoft’s Bing.

That’s because OpenAI, the creator of ChatGPT, has a deep alignment of interests with Microsoft.

In 2023, Microsoft invested more than $10 billion in OpenAI, acquired a 49% stake, agreed to share AI intellectual property, and became its exclusive cloud service provider.

Through this partnership, OpenAI uses Microsoft’s cloud computing platform Azure to train and deploy its AI models.

In March 2024, Microsoft went even further, announcing plans to invest close to ten times that amount—$100 billion—to build new data centers.

Even if the day comes when a search engine becomes indispensable for ChatGPT, as long as Bing is in the picture instead of Google, there will be no business card for Google to hand over.

So was Google ultimately forced to pour money into building Gemini because LLMs like ChatGPT will replace search engines?

To briefly recap:

- Rishi Jaluria, RBC Capital Markets equity analyst

Stepping back to look at the forest rather than the trees, the reason Microsoft and Google are pouring money into AI like there’s no tomorrow is this:

It’s not simply to bolt AI onto their search engines. In the face of a massive technological shift, it’s more accurate to see this as a race to secure a natural strategic position—and to build the infrastructure needed for it.

Google Meet, Google Calendar, Google Docs, Gmail…

MS Teams, One Calendar, MS365, Outlook…

Any technology that captures the public’s attention will inevitably see the money that poured in disappear once that attention fades.

In the end, the companies that make money are the ones that have an ecosystem.

Now we can finally start to understand what Satya Nadella, the CEO of Microsoft, really meant when he said that the gross margin of the search market will fall forever.

This is not limited to the search market. It is a bit broader than that.

Development, research, design—everyday tasks that are embedded in our work and create added value all require search. Going forward, “LLM” will take the place of “search” in that sentence.

This is exactly where Google’s sense of crisis begins, as it currently dominates most of the search market. The moment your boss says, “Why are you searching when you could just use an AI copilot?” will be the moment Google dies. In our minds, “Google” is synonymous with “search.”

But when it comes to AI copilots, who comes to mind? There is still no product that has firmly taken that spot in our minds.

This suggests that Microsoft pouring money into OpenAI, attaching Bing to GPT’s browsing, and adding Copilot to Word and Excel, and Alphabet working hard to develop Gemini, are ultimately all competing along the same line.