Breaking News
Investing Pro 0
Last Call for Cyber Monday! Save Now on Claim 60% OFF

For tech giants, AI like Bing and Bard poses billion-dollar search problem

Published Feb 22, 2023 06:06AM ET Updated Feb 22, 2023 06:51PM ET
Saved. See Saved Items.
This article has already been saved in your Saved Items
 
© Reuters. FILE PHOTO: The logo for Google LLC is seen at the Google Store Chelsea in Manhattan, New York City, U.S., November 17, 2021. REUTERS/Andrew Kelly
 
INTC
-2.15%
Add to/Remove from Watchlist
Add to Watchlist
Add Position

Position added successfully to:

Please name your holdings portfolio
 
MSFT
-1.16%
Add to/Remove from Watchlist
Add to Watchlist
Add Position

Position added successfully to:

Please name your holdings portfolio
 
GOOGL
-0.51%
Add to/Remove from Watchlist
Add to Watchlist
Add Position

Position added successfully to:

Please name your holdings portfolio
 
ACN
+1.49%
Add to/Remove from Watchlist
Add to Watchlist
Add Position

Position added successfully to:

Please name your holdings portfolio
 
MS
+1.97%
Add to/Remove from Watchlist
Add to Watchlist
Add Position

Position added successfully to:

Please name your holdings portfolio
 
GOOG
-0.45%
Add to/Remove from Watchlist
Add to Watchlist
Add Position

Position added successfully to:

Please name your holdings portfolio
 

By Jeffrey Dastin and Stephen Nellis

MOUNTAIN VIEW, Calif. (Reuters) - As Alphabet (NASDAQ:GOOGL) Inc looks past a chatbot flub that helped erase $100 billion from its market value, another challenge is emerging from its efforts to add generative artificial intelligence to its popular Google Search: the cost.

Executives across the technology sector are talking about how to operate AI like ChatGPT while accounting for the high expense. The wildly popular chatbot from OpenAI, which can draft prose and answer search queries, has "eye-watering" computing costs of a couple or more cents per conversation, the startup's Chief Executive Sam Altman has said on Twitter.

In an interview, Alphabet's Chairman John Hennessy told Reuters that having an exchange with AI known as a large language model likely cost 10 times more than a standard keyword search, though fine-tuning will help reduce the expense quickly.

Even with revenue from potential chat-based search ads, the technology could chip into the bottom line of Mountain View, Calif.-based Alphabet with several billion dollars of extra costs, analysts said. Its net income was nearly $60 billion in 2022.

Morgan Stanley (NYSE:MS) estimated that Google's 3.3 trillion search queries last year cost roughly a fifth of a cent each, a number that would increase depending on how much text AI must generate. Google, for instance, could face a $6-billion hike in expenses by 2024 if ChatGPT-like AI were to handle half the queries it receives with 50-word answers, analysts projected. Google is unlikely to need a chatbot to handle navigational searches for sites like Wikipedia.

Others arrived at a similar bill in different ways. For instance, SemiAnalysis, a research and consulting firm focused on chip technology, said adding ChatGPT-style AI to search could cost Alphabet $3 billion, an amount limited by Google's in-house chips called Tensor Processing Units, or TPUs, along with other optimizations.

More costs for Google with AI https://www.reuters.com/graphics/AI-ECONOMICS/AI-ECONOMICS/klvygnkrrvg/Ai-cost-to-Alphabet.jpg

What makes this form of AI pricier than conventional search is the computing power involved. Such AI depends on billions of dollars of chips, a cost that has to be spread out over their useful life of several years, analysts said. Electricity likewise adds costs and pressure to companies with carbon-footprint goals.

The process of handling AI-powered search queries is known as "inference," in which a "neural network" loosely modeled on the human brain's biology infers the answer to a question from prior training.

In a traditional search, by contrast, Google's web crawlers have scanned the internet to compile an index of information. When a user types a query, Google serves up the most relevant answers stored in the index.

Alphabet's Hennessy told Reuters, "It's inference costs you have to drive down," calling that "a couple year problem at worst."

Alphabet is facing pressure to take on the challenge despite the expense. Earlier this month, its rival Microsoft Corp (NASDAQ:MSFT) held a high-profile event at its Redmond, Washington headquarters to show off plans to embed AI chat technology into its Bing search engine, with top executives taking aim at Google's search market share of 91%, by Similarweb (NYSE:SMWB)'s estimate.

A day later, Alphabet talked about plans to improve its search engine, but a promotional video for its AI chatbot Bard showed the system answering a question inaccurately, fomenting a stock slide that shaved $100 billion off its market value.

Microsoft later drew scrutiny of its own when its AI reportedly made threats or professed love to test users, prompting the company to limit long chat sessions it said "provoked" unintended answers.

Microsoft's Chief Financial Officer Amy Hood has told analysts that the upside from gaining users and advertising revenue outweighed expenses as the new Bing rolls out to millions of consumers. "That's incremental gross margin dollars for us, even at the cost to serve that we're discussing," she said.

And another Google competitor, CEO of search engine You.com Richard Socher, said adding an AI chat experience as well as applications for charts, videos and other generative tech raised expenses between 30% and 50%. "Technology gets cheaper at scale and over time," he said.

A source close to Google cautioned it's early to pin down exactly how much chatbots might cost because efficiency and usage vary widely depending on the technology involved, and AI already powers products like search.

Still, footing the bill is one of two main reasons why search and social media giants with billions of users have not rolled out an AI chatbot overnight, said Paul Daugherty, Accenture (NYSE:ACN)'s chief technology officer.

"One is accuracy, and the second is you have to scale this in the right way," he said.

MAKING THE MATH WORK

For years, researchers at Alphabet and elsewhere have studied how to train and run large language models more cheaply.

Bigger models require more chips for inference and therefore cost more. AI that dazzles consumers for its human-like authority has ballooned in size, reaching 175 billion so-called parameters, or different values that the algorithm takes into account, for the model OpenAI updated into ChatGPT. Cost also varies by the length of a user's query, as measured in "tokens" or pieces of words.

One senior technology executive told Reuters that such AI remained cost-prohibitive to put in millions of consumers' hands.

"These models are very expensive, and so the next level of invention is going to be reducing the cost of both training these models and inference so that we can use it in every application," the executive said on condition of anonymity.

For now, computer scientists inside OpenAI have figured out how to optimize inference costs through complex code that makes chips run more efficiently, a person familiar with the effort said. An OpenAI spokesperson did not immediately comment.

A longer-term issue is how to shrink the number of parameters in an AI model 10 or even 100 times, without losing accuracy.

"How you cull (parameters away) most effectively, that's still an open question," said Naveen Rao, who formerly ran Intel Corp (NASDAQ:INTC)'s AI chip efforts and now works to lower AI computing costs through his startup MosaicML.

In the meantime, some have considered charging for access, like OpenAI's $20 per month subscription for better ChatGPT service. Technology experts also said a workaround is applying smaller AI models to simpler tasks, which Alphabet is exploring.

The company said this month a "smaller model" version of its massive LaMDA AI technology will power its chatbot Bard, requiring "significantly less computing power, enabling us to scale to more users."

Asked about chatbots like ChatGPT and Bard, Hennessy said at a conference called TechSurge last week that more focused models, rather than one system doing everything, would help "tame the cost."

For tech giants, AI like Bing and Bard poses billion-dollar search problem
 

Related Articles

Add a Comment

Comment Guidelines

We encourage you to use comments to engage with other users, share your perspective and ask questions of authors and each other. However, in order to maintain the high level of discourse we’ve all come to value and expect, please keep the following criteria in mind:  

  •            Enrich the conversation, don’t trash it.

  •           Stay focused and on track. Only post material that’s relevant to the topic being discussed. 

  •           Be respectful. Even negative opinions can be framed positively and diplomatically. Avoid profanity, slander or personal attacks directed at an author or another user. Racism, sexism and other forms of discrimination will not be tolerated.

  • Use standard writing style. Include punctuation and upper and lower cases. Comments that are written in all caps and contain excessive use of symbols will be removed.
  • NOTE: Spam and/or promotional messages and comments containing links will be removed. Phone numbers, email addresses, links to personal or business websites, Skype/Telegram/WhatsApp etc. addresses (including links to groups) will also be removed; self-promotional material or business-related solicitations or PR (ie, contact me for signals/advice etc.), and/or any other comment that contains personal contact specifcs or advertising will be removed as well. In addition, any of the above-mentioned violations may result in suspension of your account.
  • Doxxing. We do not allow any sharing of private or personal contact or other information about any individual or organization. This will result in immediate suspension of the commentor and his or her account.
  • Don’t monopolize the conversation. We appreciate passion and conviction, but we also strongly believe in giving everyone a chance to air their point of view. Therefore, in addition to civil interaction, we expect commenters to offer their opinions succinctly and thoughtfully, but not so repeatedly that others are annoyed or offended. If we receive complaints about individuals who take over a thread or forum, we reserve the right to ban them from the site, without recourse.
  • Only English comments will be allowed.
  • Any comment you publish, together with your investing.com profile, will be public on investing.com and may be indexed and available through third party search engines, such as Google.

Perpetrators of spam or abuse will be deleted from the site and prohibited from future registration at Investing.com’s discretion.

Write your thoughts here
 
Are you sure you want to delete this chart?
 
Post
Post also to:
 
Replace the attached chart with a new chart ?
1000
Your ability to comment is currently suspended due to negative user reports. Your status will be reviewed by our moderators.
Please wait a minute before you try to comment again.
Thanks for your comment. Please note that all comments are pending until approved by our moderators. It may therefore take some time before it appears on our website.
 
Are you sure you want to delete this chart?
 
Post
 
Replace the attached chart with a new chart ?
1000
Your ability to comment is currently suspended due to negative user reports. Your status will be reviewed by our moderators.
Please wait a minute before you try to comment again.
Add Chart to Comment
Confirm Block

Are you sure you want to block %USER_NAME%?

By doing so, you and %USER_NAME% will not be able to see any of each other's Investing.com's posts.

%USER_NAME% was successfully added to your Block List

Since you’ve just unblocked this person, you must wait 48 hours before renewing the block.

Report this comment

I feel that this comment is:

Comment flagged

Thank You!

Your report has been sent to our moderators for review
Continue with Google
or
Sign up with Email