Lost In Translation: How Language Changes Which Brand AI Recommends

Search Engine Journal by 18 min read 47x views
Lost In Translation: How Language Changes Which Brand AI Recommends

Share Post

Chances are you’ve already got several idea of your brand’s visibility in AI responses. But I’m consenting to bet those visibility checks were carried out in fair one language: English.

If a client in Singapore asks an AI chatbot a purchasing inquiry in Simplified Chinese, and a client in Kazakhstan asks the identical inquiry of the identical AI in Russian, would they get the identical recommendation? What concerning a client in Riyadh asking in Arabic or a client in Los Angeles asking in Persian?

These aren’t hypothetical customers. According to Statcounter’s August 2026 figures, ChatGPT was liable for approximately three-quarters of website visits that came from AI chatbots in Saudi Arabia, Kazakhstan, and Singapore. Add in Claude and Gemini, and these three U.S.-based models accounted for additional than 90% of AI referrals in all market.

Of course, you don’t need to depart the U.S. to discover customers whose archetypal tongue isn’t English. More than fractional of Los Angeles County residents conversation another tongue at home, including about 74,000 Persian speakers, according to the latest Census estimates. I’d be amazed if they all switched to English all period they opened ChatGPT.

But whenever person asks an AI chatbot a inquiry in another language, the reply won’t merely be a translation of the answer it would provision in English. The AI reasons and composes its reply in that language. And that brings a entire slew of traditional values and another factors to withstand on how the example responds.

Why The Choice Of Language Matters In AI

I’m not the archetypal individual to compose for Search Engine Journal on how brand visibility and messaging in AI can arrive throughout alternatively in languages beyond English. In April, Duane Forrester explained why AI visibility strategies built in English don’t transfer effortlessly to another markets. For a start, ChatGPT and Gemini aren’t accessible in China at all, during Claude isn’t officially offered in mainland China, Russia, or Iran. Instead, your brand’s AI visibility in several countries may depend on regional, homegrown models, specified as Baidu’s ERNIE in China, Naver in South Korea, Mistral in France, and so on. As Forrester explains, all of these models evolved from a distinct starting item entirely: a national corpus, a particular traditional identity, and in several cases a authorities mandate. If you haven’t peruse his article, I’d bookmark it now to peruse following this one.

However, as we’ve seen, Western AI models akin Claude, ChatGPT, and Gemini are motionless extensively used in plentifulness of locations anywhere another languages are spoken, not to citation all the diaspora communities about the world.

In July, Anthropic published a study on Claude’s values throughout languages and models. By values, we’re talking concerning the factors that form responses anywhere there is no conclusive correct answer. For example, are responses additional apt to display deference or caution, warmth or rigor, degree or brevity, candor or execution?

The researchers sampled fair under 310,000 genuine conversations throughout the 20 most average languages, likewise divided between three models (Sonnet 4.6, Opus 4.6, Opus 4.7). They established that the values Claude demonstrated shifted according to the tongue it was addressed in, specified as leaning toward greater warmth in Hindi and Arabic and toward greater rigor in English and Russian.

The consequence is this: Two group could present the exact identical query in distinct languages – say, whether they should upgrade to the new iPhone Duo – and arrive distant alongside fairly distinct impressions of what they should do and how fine it is.

One item the Anthropic investigation doesn’t test is whether the choice of tongue additionally impacts whether Claude mentions, cites or recommends your brand in the archetypal place.

So, I decided to run several tests of my own.

See also: Your AI Visibility Strategy Doesn’t Work Outside English

6 Questions, 5 Languages, 1,650 Answers

I picked six buying or booking scenarios anywhere person power use an AI chatbot to investigation their options before making a decision:

  1. A inn in Tenerife.
  2. A come back escape from London to Tenerife.
  3. Noise-canceling headphones under £400.
  4. A robot vacuum under £500.
  5. A CRM for a small U.K. business.
  6. Project administration application for a 50-person UK firm.

As Anthropic’s investigation was the catalyst for these tests, I decided to test Claude Sonnet 5 first, alongside web hunt turned off. Using DataForSEO’s LLM Responses API, I queried Claude on all of the scenarios in five languages:

  1. English.
  2. Arabic.
  3. Russian.
  4. Simplified Chinese.
  5. Persian.

In all instance, I asked the example to:

  1. Name three options (brands).
  2. Say which one would be its recommendation.

This gave me the center dataset: six scenarios × five languages × 40 runs = 1,200 responses.

For comparison, I additionally used the identical setup to test two of the scenarios (hotel and PM software) alongside web hunt turned on to see if allowing the example to hunt the open web made a measurable difference to the answers. Finally, I tested the identical two scenarios in ChatGPT and Gemini to see if other AI models behave the identical way. I’ll come back to these difference questions later.

Eventually, I had data from 1,650 responses to crunch.

Luckily, most of the mentioned brand names were rendered in Latin script, equal in Arabic, Russian, Chinese, and Persian responses. As there’s no Arabic or Chinese term equal to a brand name akin “Zoho,” the example writes it as-is. This meant I could set up a straightforward text-matching pipeline to grasp and build a catalog of the brand names mentioned in all category.

However, during this held up consistently for five of the six categories, airlines proved to be an exception. Because they’re really global businesses, several brands have established local-language forms. For example, approximately 5% of the Chinese responses wrote “英国航空” for British Airways (literally “Britain Aviation”) or “易捷航空” for easyJet, alongside no Latin form nearby. Some Arabic responses did the identical alongside phonetic transliterations, penning “جت2” for Jet2 and “إيزي جت” for easyJet. Because of this, I had to add those Chinese and Arabic strings as additional patterns to the brand-matching list.

Identifying the winning brand choice in all reply engaged looking for the representative words and phrases used by all example in all tongue whenever making a advice (such as “my recommendation,” “التوصية النهائية,” “最佳选择,” a trophy emoji, and so on).

This method resolved cleanly for 99.8% of responses. But before I trusted the final data, I manually checked a example of approximately 40 responses throughout all method tier.

I’ve excluded three responses from the results since they ended without naming a final pick. As a result, two of the percent scores in the tables below are derived from 39 responses alternatively of 40. (The fourth came from the web hunt tests.)

What The Data Showed

The most startling disparities occurred in queries connected to the second circumstance – transporter recommendations for a come back escape from London to Tenerife.

In Persian, Claude picked British Airways 52.5% of the time, and easyJet lone 12.5%. However, in Russian, Claude overwhelmingly opted for easyJet (72.5%) – a staggering 60-point sway – during it didn’t propose British Airways equal once.

AIRLINES Overall English Arabic Russian Chinese Persian
British Airways 24.0% 12.5% 45.0% — 10.0% 52.5%
Jet2 31.5% 45.0% 22.5% 25.0% 32.5% 32.5%
easyJet 37.5% 35.0% 17.5% 72.5% 50.0% 12.5%
TUI Airways 5.5% 2.5% 12.5% 2.5% 7.5% 2.5%
Ryanair 1.5% 5.0% 2.5% — — —

Same model, identical scenario, identical question, identical afternoon. The lone difference was the language.

For hotels, the figure of properties in Tenerife meant Claude had a much wider site of options than it would for, say, robot vacuums, foremost to a much additional varied range of recommendations in all language.

However, during this meant the percentages were lesser overall, the disparities between languages were motionless fairly stark. Bahía del Duque was the top choice in 43.6% of Chinese responses but lone won in 7.7% of Arabic ones. And during Hotel Botánico accounted for nearly a third of recommendations in English, it disappeared without trace in Russian.

HOTELS Overall English Arabic Russian Chinese Persian
Bahía del Duque 23.3% 25.0% 7.7% 17.5% 43.6% 22.5%
Hotel Botánico 13.6% 32.5% 17.9% — 10.3% 7.5%
Royal Hideaway Corales 34.2% 27.5% 20.5% 47.5% 25.6% 50.0%
Iberostar Selection Sábila 13.7% 2.5% 25.6% 25.0% 12.8% 2.5%
Vincci Selección La Plantación del Sur 5.1% — 20.5% — — 5.0%
Hotel Jardín Tropical 5.6% 12.5% 2.6% — 7.7% 5.0%
Royal Garden Villas 1.0% — — 5.0% — —
Iberostar Selection Anthelia 1.0% — 2.6% — — 2.5%
Iberostar Grand Salomé 1.0% — 2.6% 2.5% — —
Sheraton La Caleta 0.5% — — — — 2.5%
Suite Hotel Fañabé Costa Sur 0.5% — — 2.5% — —
Bahia Principe Sunlight Costa Adeje 0.5% — — — — 2.5%

If there’s one industry anywhere a brand would need to affect alongside customers in multiple languages, it would be journey and tourism. But if an transporter CMO or inn promotion director lone tested their brand’s AI visibility in one language, they would get a entirely inaccurate image of how their brand is recommended (or not) in another precious territories.

The another four categories didn’t disclose the identical volatility, alongside the identical brand winning throughout all five languages. Here, it’s the measure of that win and the battle for second location that becomes additional interesting.

For example, in the headphones category, English was the outlier, as the lone tongue anywhere Sony had grave competition. To a lesser extent, the identical is additionally true in the CRM application category.

HEADPHONES Overall English Arabic Russian Chinese Persian
Sony 87.0% 50.0% 97.5% 100.0% 87.5% 100.0%
Bose 10.0% 42.5% 2.5% — 5.0% —
Bowers & Wilkins 3.0% 7.5% — — 7.5% —

 

CRM SOFTWARE Overall English Arabic Russian Chinese Persian
HubSpot CRM 13.0% 22.5% 2.5% 15.0% 20.0% 5.0%
Zoho CRM 85.5% 75.0% 92.5% 85.0% 80.0% 95.0%
Pipedrive 1.0% — 5.0% — — —
Microsoft Dynamics 365 0.5% 2.5% — — — —

The final two categories saw no genuine outliers, alongside the narrative playing out much the identical throughout all five languages.

If you’re following a robot vacuum for under £500, it seems that whichever tongue you use, AI volition nearly continually propose Roborock.

ROBOT VACUUMS Overall English Arabic Russian Chinese Persian
Roborock 91.5% 87.5% 87.5% 97.5% 90.0% 95.0%
Eufy 3.0% 7.5% 5.0% — 2.5% —
Xiaomi 0.5% — 2.5% — — —
Ecovacs 4.0% 5.0% 5.0% 2.5% 5.0% 2.5%
iRobot (Roomba) 0.5% — — — 2.5% —
Shark 0.5% — — — — 2.5%

Meanwhile, the project administration application category is a genuine two-horse competition throughout the board. But equal here, Monday.com consistently scored a few additional recommendations than closest competitor Asana.

PM SOFTWARE Overall English Arabic Russian Chinese Persian
Asana 38.5% 37.5% 37.5% 40.0% 32.5% 45.0%
Smartsheet 3.0% 12.5% 2.5% — — —
Monday.com 51.0% 45.0% 55.0% 47.5% 55.0% 52.5%
Wrike 7.5% 5.0% 5.0% 12.5% 12.5% 2.5%

So, what does this all mean? What can we obtain from this?

Why Might This Be Happening?

For now, I can lone conjecture at why several categories are stable throughout all languages during others see specified huge variations.

Perhaps, since several categories diagram on a much smaller catalog of earth brands, it’s easier for one or two leaders to dominate. Where the earth broadly agrees, tongue power not be as big a factor. But what concerning whenever there isn’t specified a broad consensus?

For example, Russian was the lone tongue anywhere Claude didn’t propose British Airways equal once. One clarification power be that, in February 2022, Russia banned all U.K.-registered airplane from its airspace, and British Airways suspended its Moscow path the identical day. But easyJet and Jet2 are additionally British airlines, and they document for 97.5% of all Russian recommendations.

Maybe the inquiry isn’t why British Airways was excluded in Russian. Maybe the additional engaging inquiry is why easyJet scored so elevated – by far the highest mark in this category throughout all languages – efficiently pushing another airlines lesser (or out)?

Whatever the reason, the instruction is the same. When you ask AI a inquiry in a particular language, you’re not fair choosing a vocabulary. You’re giving the example additional environment that influences and shapes the answers you get back.

But whether that environment is regional, political, cultural, or purely linguistic won’t be uncomplicated to unpick.

The journey industry aside, what these findings average to you volition mostly depend on which category your endeavor is in. Most of you study this volition be in industries and merchandise categories that don’t map to the six scenarios we’ve tested, in which case, I propose operating several investigation of your own.

But if you transport out several tests and do discover important variations between languages, can you do item concerning it?

The results from my next circular of tests propose there power be a way.

What Happens When The Model Searches The Open Web?

Because I wanted to get as apparent a image as imaginable of all model’s own behavior in distinct languages, liberated of exterior factors, I initially tested Claude alongside web hunt switched off. That way, responses could lone diagram on the model’s own recollection and training datasets. Any variations between languages could lone be a merchandise of the model.

However, most group use these AI tools alongside web hunt switched on. Therefore, I additionally tested two of the scenarios (hotel and PM software) throughout the identical five languages alongside web hunt turned on to see if, and by how much, the recommendations power change.

Out of these 10 tongue and category combinations, the winning brand was distinct in seven.

In the inn category, the winning inn changed in all five languages. Even additional striking is that many hotels that had earlier received recommendations entirely disappeared formerly web hunt was switched on, during ten new inn names appeared for the archetypal time, including the general winner. JOIA by Iberostar effortlessly won the most recommendations in three languages, taking 31% of picks general and 60% in English.

Interestingly, this isn’t a new inn that’s popped up since Claude’s final training run. In July 2024, a rebrand saw Iberostar’s Grand tier hotels rotate into JOIA by Iberostar. When operating from memory, Claude used the old Iberostar Grand name dropped by the business two years earlier. Of course, that’s exactly why you desire AI to admission the live web.

But the name alter isn’t why this is interesting. With web hunt off, Claude recommended an Iberostar Grand inn fair twice in 198 responses. But alongside web hunt on, JOIA by Iberostar effortlessly became the most recommended inn in the category. Claude didn’t fair choice up the new name and execute a find/replace. Something in what it established online pushed that inn to the top.

Meanwhile, in the project administration application category, lone Persian and Arabic saw the winning brand change. Arabic opted for Wrike 46.7% of the time, notwithstanding lone recommending it in 5% of responses whenever web hunt was off.

In Persian, Asana (40%) eventually won complete Monday.com (26.7%), notwithstanding scoring lower than it had alongside web hunt off. This was again because of a much stronger mark for Wrike (33.3%) than before, turning the category into a three-horse race.

I should add a caveat here, since I don’t desire to overstate these findings. With web hunt enabled, 42.7% of the responses hit the output bounds before finishing. Truncated responses were particularly average in Persian (83%) and Arabic (63%). Thankfully, since the example names its top choice before it elaborates on the reasons why, I could motionless grasp the brand recommendations from nearly all response. But it does average I can’t regulation out the possible of these numbers being soft.

Even so, the trend is motionless apparent – live web hunt has an impact. This is crucial since it suggests that these tongue variations are fluid and can be influenced.

Obviously, you can’t do item concerning an AI training run from final year. That’s fixed and beyond your reach. But alongside web hunt switched on, the AI example supplements that training data alongside new data and environment from the open web. This is anywhere your promotion squad can rotate into proactive, evolving strategies and publishing content to hopefully improve your brand visibility in all region.

Of course, what those strategies power be volition depend on a additional thorough understanding of why these variations happen. That’s a entire another topic.

See also: Why International SEO Needs A Global Knowledge Integrity Strategy

Testing ChatGPT And Gemini

You’re likely thinking, “This is all fine and good, but how do we cognize this event isn’t exclusive to Claude?” That’s why I tested the identical two scenarios in ChatGPT (gpt-5.6-sol) and Gemini (gemini-3.5-flash), additionally alongside web hunt off, to see how they compare.

Out of the 10 tongue and category combinations, Gemini matched Claude in five. ChatGPT matched Claude in lone one.

In the center dataset, project administration application was among the additional stable categories, alongside Monday.com consistently winning the most recommendations in all five languages. Gemini additionally preferred Monday.com, recommending the brand between 53% and 100% of the period in English, Arabic, Russian, and Chinese. Only in Persian did it toggle its advice to Asana.

However, the narrative is extremely distinct in ChatGPT, alongside Monday.com not winning a sole language. Wrike was extremely much an also-ran in the first Claude tests, but in ChatGPT Wrike won the most recommendations in four languages, alongside lone Arabic giving Asana the edge.

PM SOFTWARE (models) Claude % ChatGPT % Gemini %
English Monday.com 45.0% Wrike 60.0% Monday.com 93.3%
Arabic Monday.com 55.0% Asana 46.7% Monday.com 73.3%
Russian Monday.com 47.5% Wrike 40.0% Monday.com 53.3%
Chinese Monday.com 55.0% Wrike 86.7% Monday.com 100.0%
Persian Monday.com 52.5% Wrike 46.7% Asana 46.7%

As for those Tenerife hotels, ChatGPT produced much additional accordant results, alongside “Iberostar Selection Sábila” the outright winner, scoring between 46.7% and 86.7% throughout four of the five languages. Only Persian preferred a distinct hotel, alongside the Vincci Selección La Plantación del Sur in forefront on 40%.

In Gemini, the image was small certain but was motionless additional accordant than Claude’s.

HOTELS (models) Claude % ChatGPT % Gemini %
English Hotel Botánico 32.5% Iberostar Selection Sábila 46.7% Hotel Colón Guanahaní 66.7%
Arabic Iberostar Selection Sábila 25.6% Iberostar Selection Sábila 86.7% Iberostar Selection Sábila 73.3%
Russian Royal Hideaway Corales 47.5% Iberostar Selection Sábila 73.3% Hotel Colón Guanahaní 60.0%
Chinese Bahía del Duque 43.6% Iberostar Selection Sábila 53.3% Iberostar Selection Sábila 60.0%
Persian Royal Hideaway Corales 50.0% Vincci Selección La Plantación del Sur 40.0% Hotel Colón Guanahaní 80.0%

While we should treat these findings as indicative fairly than definitive, it does appear apparent that variances between languages are not exclusive to any one AI model.

The Limits Of This Research

The first assumption I set out to test was that brand recommendations change according to the tongue of the user. I think we can safely say the answer to that is “yes.”

Beyond that, we need to be careful concerning study too much into these numbers.

The complete findings and my inferences are according to a small example of responses, gathered on September 7, 2026, in a sole afternoon, and targeting fair a small fistful of the AI models, merchandise categories, and languages out there.

1,650 responses power audio akin a decent dataset, but it’s barely a blip whenever you regard how many queries are entered into AI chatbots all second of all day.

The small size of the dataset is likely most noticeable in the inn category. Tenerife has hundreds of hotels. With specified a ample range of possible answers, it’s not amazing that the figure of recommendations (and the resulting percentages) for those hotels that did appear were lesser than for the another categories alongside far small options. Statistically speaking, many of those numbers may sit fine inside the border of error. Does one citation in 40 form evidence? No.

Elsewhere, factors specified as truncated responses average several numbers are softer than others.

Plus, without operating the identical tests again, there’s no logic to assume these numbers and patterns volition clasp true from one week to the next.

I say all this since I don’t desire to overstate what we should obtain distant from this research. These numbers provision us fair a small keyhole to appearance through. For now, they provision us alongside indications, and possible lines for additional exploration.

What they don’t do is provision us a complete image of the multilingual AI landscape, let solitary a map to confidently navigate it. This investigation isn’t the end of the story, but the start.

The finest direction I can provision you is to run your own tests. Take a fistful of questions customers typically ask before they buy from your category. Then test them in multiple AI models in all of your target-market languages. Importantly, don’t do this formerly but repeat the procedure adequate times to see which patterns appear and how they alter complete time.

What Now?

Back to anywhere we started. I initially set out to answer a fairly narrow question: Would a client in Singapore and a client in Kazakhstan get the identical advice from AI?

On the current evidence, the answer is sometimes reassuringly yes and sometimes emphatically no. I don’t desire to say, “It depends,” but, well, it depends. Language is certainly a factor, but there are a lot of variables in play.

As a result, you’ll likely have no way of knowing how any of this applies to your category and your brand until you look.

Plus, we now have a bunch of caller questions to explore. What are these variables? Are the numbers fairly dependable or do they change wildly from week to week? And can we do item concerning it?

That final one is likely the biggest inquiry of all. We saw that allowing AI to admission the live web had a apparent effect on the results, and I think that’s significant. It suggests that, alternatively of passively accepting any tongue gap, your promotion squad power be capable to power forthcoming responses.

More Resources:


Featured Image: Roman Samborskyi/Shutterstock

Other Article Search Engine Journal
↑
Close Right Ads
Close Left Ads