Respond to prefers-color-scheme dark in app web views

Follow section front redirects in Spectrum

Enable NLP source in PCS for load testing

Switch off the News Alerts Survey Widget when we have enough responses

Poll to determine if new LUF updates are available

Allows the winning A/B test variant to be turned on without Spectrum deployment

Show infinite scrolling sections in Assembler MultiView

Show option for NLP article summaries in Action Bar

Show NLP summaries collection recirc on article pages

A general feature flag for Recipe Features that aren't yet released

Feature flag for media team features that aren't yet released

Use new Orca video player for preroll plugin in Spectrum

Show the snow tracker (instead of the heat tracker) on Spectrum

Google’s new ‘Find My’ device network is useful but a stalking risk

Apple apologizes for iPad ad after blowback 

Hack targeting hospital chain Ascension is impacting patient care

I used to dismiss my mom’s advice. Then I found a note she wrote before she died.  

Photos of his mom at lunch are a balm people didn’t know they needed

Dog hit by car loses leg, is adopted by firefighter who helped her

Narrative of Trump snoozing in court takes hold — much to his annoyance

Trump might not go to jail, but this trial is a close second

Trump team throws out GOP plan and builds a ‘leaner’ 2024 operation 

Israel’s shameful ban on Al Jazeera

Courtroom photo ban adds to hurdles for Trump trial journalists

Checks, not sex, and other takeaways from Trump’s New York hush money trial

The case for eating at Italian gas stations

Cruise ship drags dead whale into New York, prompting investigation

5 big travel lessons from an accomplished conservationist

Russia unleashes new attacks on Ukraine’s Kharkiv border region

Malaysia’s appetite for oil and gas puts it on collision course with China

Israel orders new evacuations in Rafah as U.N. says more than 150,000 have fled

Carolyn Hax: Why is he trying to sabotage his partner’s very public success?

Ask Amy: What do I say about my former walking buddy who yelled at me?

Miss Manners: Journalist posts on social media about a typo at another outlet

Kristi Noem’s dog killing is pure Southern gothic

Claire Messud turns her family’s history into a masterpiece

Joseph Epstein, conservative provocateur, tells his life story in full

Bishop vanished. His species can still be saved.

In photos: Extreme solar storm brings spectacular northern lights

Late-night hosts mock Trump sleeping in court

The urgent air defense lessons of April 16 

Are smartphones destroying childhood? It’s complicated.

In the campaign against Donald Trump, do the ends justify the means?

Nats win opener at Fenway behind Patrick Corbin and a lockdown bullpen

Jayden Daniels makes Commanders debut at rookie minicamp: ‘He’s a stud’

To get outs with his fastball, Trevor Williams put his ego aside

US-POLITICS-BIDEN

Big Tech news and how to take control of your data and devices

See more coverage in your recommendations

Technology

Tips delivered to your inbox twice weekly

The Tech Friend newsletter

Business and Tech alerts

Gabby Jones

AI chatbots have a problem: They lose money on every chat.

The enormous cost of running today’s large language models, which underpin tools like <a href="https://www.washingtonpost.com/technology/2022/12/06/what-is-chatgpt-ai/" target="_blank">ChatGPT</a> and <a href="https://www.washingtonpost.com/technology/2023/04/13/microsoft-bing-ai-chatbot-error/" target="_blank">Bard</a>, is limiting their quality and threatening to throttle the global <a href="https://www.washingtonpost.com/business/interactive/2023/artificial-intelligence-tech-rapid-advances/" target="_blank">AI boom</a> they’ve sparked.

Their expense, and the limited availability of the <a href="https://www.washingtonpost.com/technology/2023/05/25/nvidia-ai-stock-gpu-chatbots/" target=_blank>computer chips</a> they require, are also constraining which companies can afford to run them and pressuring even the world’s richest companies to turn chatbots into moneymakers sooner than they may be ready to.

“The models being deployed right now, as impressive as they seem, are really not the best models available,” said Tom Goldstein, a computer science professor at the University of Maryland. “So as a result, the models you see have a lot of weaknesses” that might be avoidable if cost were no object — such as a propensity to spit out <a href="https://www.washingtonpost.com/technology/2022/07/16/racist-robots-ai/" target=_blank>biased results</a> or <a href="https://www.washingtonpost.com/technology/2023/04/05/chatgpt-lies/" target=_blank>blatant falsehoods</a>.

What happens when ChatGPT lies about real people?

The <a href="https://www.washingtonpost.com/technology/2023/06/04/ai-bubble-tech-industry-outlook/" target=_blank>tech giants staking their future on AI</a> rarely discuss the technology’s cost. OpenAI (the maker of ChatGPT), Microsoft and Google all declined to comment. But experts say it’s the most glaring obstacle to Big Tech’s vision of <a href="https://www.washingtonpost.com/technology/2023/05/07/ai-beginners-guide/" target=_blank>generative AI</a> zipping its way across every industry, slicing head counts and boosting efficiency.

The intensive computing AI requires is why OpenAI has held back its <a href="https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwjqkYTDvaX_AhXxEFkFHeVqAZEQFnoECAsQAQ&url=https%3A%2F%2Fwww.washingtonpost.com%2Ftechnology%2F2023%2F03%2F14%2Fgpt-4-has-arrived-it-will-blow-chatgpt-out-water%2F&usg=AOvVaw0NX98F2ytiF1NLMJ3utDsu" target=_blank>powerful new language model, GPT-4</a>, from the free version of ChatGPT, which is still running a weaker GPT-3.5 model. ChatGPT’s underlying data set was last updated in September 2021, making it useless for researching or discussing recent events. And even those who pay $20 per month for GPT-4 can send only 25 messages every three hours because it’s so expensive to run. (It’s also much slower to respond.)

Those costs may also be one reason Google has yet to build an AI chatbot into its flagship search engine, which fields billions of queries every day. When <a href="https://www.washingtonpost.com/technology/2023/03/21/bard-google-ai/" target=_blank>Google released its Bard chatbot</a> in March, it opted not to use its largest language model. Dylan Patel, chief analyst at the semiconductor research firm SemiAnalysis, estimated that a single chat with ChatGPT could cost up to 1,000 times as much as a simple Google search.

In a recent report on artificial intelligence, the Biden administration pinpointed the computational costs of generative AI as a national concern. The White House wrote that the technology is expected to “dramatically increase computational demands and the associated environmental impacts,” and that there’s an “urgent need” to design more sustainable systems.

Even more than other forms of machine learning, generative AI requires dizzying amounts of computational power and specialized computer chips, known as GPUs, that only the wealthiest of companies can afford. The intensifying battle for access to those chips has helped to make their leading providers into tech giants in their own right, giving them the keys to what has become the technology industry’s most prized asset.

Silicon Valley came to dominate the internet economy in part by offering services like online search, email and social media to the world free, losing money initially but eventually turning hefty profits on personalized advertising. And ads are probably coming to AI chatbots. But analysts say ads alone probably won’t be enough to make cutting-edge AI tools profitable anytime soon.

In the meantime, the companies offering AI models for consumer use must balance their desire to win market share with the financial losses they’re racking up.

The search for more reliable AI also is likely to drive profits primarily to the chipmakers and cloud computing giants that already control much of the digital space — along with the chipmakers whose hardware they need to run the models.

It’s no accident that the companies building the leading AI language models are either among the largest cloud computing providers, as with Google and Microsoft, or have close partnerships with them, as OpenAI does with Microsoft. Companies that buy those firms’ AI tools don’t realize they’re being locked into a heavily subsidized service that costs much more than what they’re currently paying, said Clem Delangue, CEO of Hugging Face, an open-source AI company.

OpenAI CEO Sam Altman indirectly acknowledged the problem at a Senate hearing last month, when Sen. Jon Ossoff (D-Ga.) warned that if OpenAI were to try to make ChatGPT addictive in a way that harms kids, Congress “will look very harshly” on it. Altman said Ossoff needn’t worry: “We try to design systems that do not maximize for engagement. In fact, we’re so short on GPUs, the less people use our products, the better.”

The expense of AI language models starts with developing and training them, which requires gargantuan amounts of data and software to identify patterns in language. AI companies also typically hire star researchers whose salaries can rival those of pro athletes. That presents an initial barrier to any company hoping to build its own model, though a few well-funded start-ups have succeeded — including Anthropic AI, which OpenAI alumni founded with financial backing from Google.

Then, each query to a chatbot like ChatGPT, Microsoft’s Bing or Anthropic’s Claude is routed to data centers, where supercomputers crunch the models and perform numerous high-speed calculations at the same time — first, interpreting the user’s prompt, then working to predict the most plausible response, one “token,” or four-letter sequence, at a time.

That sort of computational power requires GPUs, or graphics processing units, that were first made for video games but were found to be the only chips that could handle such heavy computer tasks as large language models. Currently, just one company, <a href="https://www.washingtonpost.com/technology/2023/05/25/nvidia-ai-stock-gpu-chatbots/" target=_blank>Nvidia</a>, sells the best of those, for which it charges tens of thousands of dollars. Nvidia’s valuation recently rocketed to $1 trillion on the anticipated sales. The Taiwan-based company that manufactures many of those chips, TSMC, has likewise soared in value.

“GPUs at this point are <a href="https://www.wsj.com/articles/the-ai-boom-runs-on-chips-but-it-cant-get-enough-9f76f554" target=_blank>considerably harder to get than drugs</a>,” Elon Musk, who recently purchased some 10,000 GPUs for his own <a href="https://www.washingtonpost.com/technology/2023/04/21/elon-musk-ai-startup/" target=_blank>AI start-up</a>, told a May 23 Wall Street Journal summit.

Those computing requirements also help to explain why OpenAI is no longer the nonprofit it was founded to be.

Started in 2015 with the <a href="https://openai.com/blog/introducing-openai" target="_blank">stated mission</a> of developing AI “in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return,” by 2019, it had <a href="https://www.washingtonpost.com/business/2019/07/22/microsoft-invests-billion-openai-pursue-artificial-intelligence-thats-smarter-than-we-are/" target="_blank">switched to a for-profit model</a> to attract investors, including Microsoft, which pumped in $1 billion and became OpenAI’s exclusive computing provider. (Microsoft has since <a href="https://www.washingtonpost.com/technology/2023/01/21/microsoft-office-chatgpt-openai/" target="_blank">poured in $10 billion more</a> and integrated OpenAI’s technology with Bing, Windows and other products.)

Exactly how much chatbots like ChatGPT cost to run is a moving target, as companies work to make them more efficient.

In December, not long after its launch, Altman estimated the cost of ChatGPT at “probably single-digits cents per chat.” That might not sound like much, until you multiply it by upward of 10 million users per day, as analysts have estimated. In February, SemiAnalysis calculated that <a href="https://www.semianalysis.com/p/the-inference-cost-of-search-disruption" target=_blank>ChatGPT was costing OpenAI</a> some $700,000 per day in computing costs alone, based on the processing needed to run GPT-3.5, the default model at the time.

Multiply those computing costs by the 100 million people per day who use <a href="https://www.washingtonpost.com/technology/2023/02/16/microsoft-bing-ai-chat-interview/" target="_blank">Microsoft’s Bing search engine</a> or the more than 1 billion who reportedly use Google, and one can begin to see why the tech giants are reluctant to make the best AI models available to the public.

The new Bing told our reporter it ‘can feel or think things’

“This is not a sustainable equation for the democratization or wide availability of generative AI, the economy or the environment,” said Sid Sheth, founder and CEO of d-Matrix, a start-up working to build more efficient chips for AI.

Google said in its February announcement of Bard that it would initially run on a “lightweight” version of the company’s LaMDA language model because it required “significantly less computing power, enabling us to scale to more users.” In other words, even a company as wealthy as Google wasn’t prepared to foot the bill of putting its most powerful AI technology into a free chatbot.

Perspective: What Google’s new AI gets right, wrong and weird.

The cost-cutting took a toll: Bard <a href="https://www.washingtonpost.com/technology/2023/02/08/google-bard-chatbot-microsoft/" target=_blank>stumbled over basic facts</a> in its launch demonstration, shearing $100 billion from the value of Google’s shares. Bing, for its part, went off the rails early on, prompting Microsoft to scale back both its personality and the number of questions users could ask it in a given conversation.

Such errors, <a href="https://www.washingtonpost.com/technology/2023/05/30/ai-chatbots-chatgpt-bard-trustworthy/" target=_blank>sometimes called “hallucinations,”</a> have become a major concern with AI language models as both individuals and companies increasingly rely on them. Experts say they’re a function of the models’ basic design: They’re built to generate likely sequences of words, not true statements.

Another Google chatbot, called Sparrow, was designed by the company’s DeepMind subsidiary to search the internet and cite its sources, with the goal of reducing falsehoods. But Google has not released that one so far.

ChatGPT ‘hallucinates.’ Some researchers worry it isn’t fixable.

Meanwhile, each of the major players is racing for ways to make AI language models cheaper.

Running a query on OpenAI’s new, lightweight GPT-3.5 Turbo model costs less than one-tenth as much as its top-of-the-line GPT-4. Google is <a href="https://www.reuters.com/technology/google-says-its-ai-supercomputer-is-faster-greener-than-nvidia-2023-04-05/" target=_blank>making its own AI chips</a>, which it claims are more efficient than Nvidia’s, as are start-ups like d-Matrix. And numerous start-ups are building on open-source language models, such as Meta’s LLaMA, so that they don’t have to pay OpenAI or Google to use theirs — even though those models don’t yet perform as well and may lack guardrails to prevent abuse.

The push for smaller, cheaper models marks a sudden reversal for the industry, said Maryland’s Goldstein.

“We spent the last four years just trying to make the biggest models we could,” he said. But that was when the goal was to publish research papers, not release AI chatbots to the public. “Now, just within the last few months, there’s been a complete turnaround in the community, and suddenly everyone’s trying to build the smallest model they can to control the costs.”

For consumers, that could mean the days of unfettered access to powerful, general-purpose AI models are numbered.

Microsoft is already experimenting with building advertisements into its AI-powered Bing results. At the Senate hearing, OpenAI’s Altman wouldn’t rule out doing the same, although he said he prefers a paid subscription model.

Both companies say they’re confident the economics will eventually pencil out. <a href="https://stratechery.com/2023/new-bing-and-an-interview-with-kevin-scott-and-sam-altman-about-the-microsoft-openai-partnership/" target="_blank">Altman told the tech blog Stratechery</a> in February, “There’s so much value here, it’s inconceivable to me that we can’t figure out how to ring the cash register on it.”

Yet critics note that generative AI also comes with costs to society.

“All this processing has implications for greenhouse gas emissions,” said Bhaskar Chakravorti, dean of global business at Tufts University’s Fletcher School. The computing requires energy that could be used for other purposes — including other computing tasks that are less trendy than AI language models. That “could even slow down the development and application of AI for other, more meaningful uses, such as in health care, drug discovery, cancer detection, etc.,” Chakravorti said.

Based on estimates of ChatGPT’s usage and computing needs, data scientist Kasper Groes Albin Ludvigsen estimated that <a href="https://towardsdatascience.com/chatgpts-electricity-consumption-7873483feac4" target=_blank>it may have used as much electricity</a> in January as 175,000 people — the equivalent of a midsize city.

For now, the tech giants are willing to lose money in a bid to win market share with their AI chatbots, Goldstein said. But if they can’t make them profitable? “Eventually you come to the end of the hype curve, and the only thing your investors are going to look at, at that point, is your bottom line.”

Still, Goldstein predicted many people and companies will find generative AI tools hard to resist, even with all their flaws. “Even though it’s expensive,” he said, “it’s still far less expensive than human labor.”

Nitasha Tiku contributed to this report.

National Magazine Award finalist, 2012

Stanford University, BA in Philosophy

Columbia Journalism School, MA in Political Journalism

Will Oremus writes about the ideas, products and power struggles shaping the digital world for The Washington Post. Before joining The Post in 2021, he spent eight years as Slate's senior technology writer and two years as a senior writer for OneZero at Medium. 

Will Oremus

washpost

ChatGPT has enormous hidden costs that could throttle AI development

the-washington-post

ChatGPT As Cyber Firms Warns That AI Bots Will Bring Nastier Phishing Scams

Bloomberg

The Washington Post

Help Desk

Artificial Intelligence

Internet Culture

Space

Tech Policy

Tech

prism://prism.query/two-sites,/technology,/politics/powerpost/the-technology-202?limit=20