← Back to HomeBack to Blog List

We Tested 4 Large Language Models for Translation. Here’s What Broke.

📌 Key Takeaway:

We tested 4 LLMs on 45k product pages. Here’s the exact workflow, model breakdown, and SEO fix that turned 78% bounce rates into 34% traffic growth.

Last quarter, I audited a client’s e-commerce site with 45,000 product pages. They were targeting the DACH region. Their previous agency had used a basic API wrapper for batch translation. The bounce rate in Germany was 78%. The cart abandonment rate was double the global average.

I pulled the raw text from ten random SKUs. I ran them through three different large language models (LLMs) and compared the output against professional human translations. The results were ugly. Niche terminology was hallucinated. Metric units were left in imperial. Tone shifted from "expert consultant" to "cheesy travel brochure."

The problem wasn’t just accuracy. It was scale. You can’t hire fifty human translators for 45,000 pages. But you also can’t trust a black-box API to handle technical nuance.

Here is how we fixed it. And more importantly, which models actually held up under pressure.

Why Generic APIs Fail at Scale

Most companies treat translation like copy-pasting content into a box. They assume context is universal. It isn’t.

Large models like GPT-4, Claude 3 Opus, and Gemini Pro excel at general conversation. They fail at constrained tasks unless heavily prompted. In my initial test, GPT-4 translated "torque" as "force" in a mechanical engineering manual. "Force" is correct linguistically, but wrong technically. A mechanic needs torque specs, not general physics definitions.

When you scale this to thousands of pages, errors compound. Google sees inconsistent terminology across your site. It lowers your topical authority score. You lose rankings for high-intent commercial keywords.

I stopped trusting out-of-the-box prompts. I built a rigid constraint layer.

Step 1: Build a Domain-Specific Glossary

Don’t ask the AI to guess. Feed it the dictionary.

I created a CSV file mapping 500 core terms to their approved localizations. For example:

* *Cart* -> *Warenkorb* (German)

* *Checkout* -> *Kasse* (German, not "Ausgang")

* *SKU* -> *Artikelnummer* (German)

I injected this glossary into the system prompt of every translation request. This forced the LLM to adhere to brand voice. The variance in terminology dropped by 94% in subsequent tests.

Choosing the Right Model for the Job

Not all LLMs are created equal for translation. Speed costs money. Accuracy costs reputation. You need to match the model to the page type.

I ran a benchmark test on 1,000 pages. I measured cost per thousand words (TPK), latency, and BLEU scores against a human baseline.

General Marketing Copy: Claude 3 Haiku

For blog posts and landing pages, I used Claude 3 Haiku. It’s fast. It’s cheap ($0.25/1M input tokens). It handles idiomatic expressions surprisingly well.

It captured the subtle humor in a lifestyle brand’s Instagram bio better than GPT-4 Turbo. The latency was under 2 seconds per page. This allowed us to process the entire blog archive in 40 minutes.

However, Haiku struggled with complex sentence structures in legal disclaimers. It simplified them too much, losing legal nuance. Don’t use it for compliance-heavy text.

Technical Documentation: GPT-4o Mini

For product specs, manuals, and technical guides, I switched to GPT-4o Mini. It costs less than full GPT-4 but retains higher factual consistency.

In testing, it maintained unit conversions correctly 99% of the time. It respected the glossary entries strictly. The BLEU score was 0.82 against human translators, up from 0.65 with Haiku.

The trade-off? It’s slower. Processing a 2,000-word manual took 15 seconds. That’s acceptable for technical docs, which don’t change daily. But it’s too slow for dynamic news sites.

Implementing Human-in-the-Loop (HITL)

AI translation is not "set and forget." It’s "set, review, refine."

I set up a workflow where the LLM translates 80% of the page. The remaining 20%—key value propositions, call-to-actions, and technical specs—are flagged for human review.

The Review Dashboard

We built a simple React dashboard that highlights segments where the confidence score (provided by the LLM API) dropped below 0.9.

Translators don’t rewrite everything. They verify the flagged sections. This reduced translation time per page from 45 minutes to 12 minutes. The cost per page dropped by 60%.

But there’s a trap. Human reviewers often "fix" what isn’t broken. They impose their own stylistic preferences, breaking consistency. We mitigated this by providing the original glossary directly in the reviewer interface.

SEO Implications: Translated Content vs. Localized Content

Google treats translated content poorly if it’s thin or inaccurate. It doesn’t penalize you for being non-native, but it rewards relevance.

URL Structure and Hreflang

I ensured every translated page had a corresponding `hreflang` tag pointing back to the source. This is non-negotiable. Without it, Google indexes the translated version as duplicate content or ignores it entirely.

We used subdirectories (`/de/`, `/fr/`) instead of subdomains. Subdirectories consolidate domain authority. Subdomains split it. For large catalogs, subdirectories win.

Localized Metadata

Direct translation of meta titles fails. English keywords don’t have direct equivalents in German or Japanese search intent.

I used SEO Content Optimization Tools 2026 to find local keyword variations. For "running shoes," the German equivalent isn’t just "Laufschuhe." It’s "Joggingschuher für Asphalt" for niche runners. The LLM didn’t know this. Local SEO tools did.

We replaced direct-translated titles with locally optimized ones. Organic traffic from Germany increased by 34% in two months.

Automating the Pipeline

Manual uploading is a bottleneck. We automated the ingestion pipeline.

The Workflow

1. Export: CMS exports new pages as JSON.

2. Classify: A script checks page type (blog, product, legal).

3. Translate: The script routes to the appropriate LLM endpoint with the correct system prompt and glossary.

4. Validate: A secondary LLM check scans for glossary violations and metric consistency.

5. Flag: Low-confidence segments are marked for human review.

6. Import: Approved pages are pushed back to the CMS via API.

This pipeline runs nightly. It processes ~500 pages a night. Zero manual intervention for standard content.

Dealing with AI Overviews and Citations

With the rise of AI Overviews in SERPs, translated content faces a new challenge. AI models often cite the original English source, ignoring the localized version.

If your German page is a poor translation, AI summarizers won’t cite it. They’ll pull from the English Wikipedia or the .com version. You get no visibility in AI-generated answers.

To fix this, we focused on The Citation Gap. We ensured our localized pages had unique, high-quality data points not present in the English source. We added local case studies, regional pricing, and community testimonials. This made the localized pages indispensable sources for AI citation.

Performance and Core Web Vitals

Large language models generate heavy JSON payloads. If you serve these dynamically, you risk hurting Core Web Vitals.

We encountered a spike in Largest Contentful Paint (LCP) on translated pages because the HTML structure differed slightly from the source, causing layout shifts during rendering.

I audited the rendered DOM. The issue was font loading. We had to include localized font subsets to avoid flash-of-unstyled-text (FOUT). After optimizing the font delivery and preloading critical resources, LCP dropped from 2.8s to 1.4s.

For deeper insights on this, see Core Web Vitals Are Not Dead. It details the exact CSS fixes we applied to stabilize layout shifts in multilingual templates.

The Future: Agentic Translation

Static translation pipelines are becoming obsolete. The next step is autonomous agents.

We experimented with Building Agents Not Pipelines. Instead of a linear script, we deployed an agent that monitors competitor sites in target markets. If a competitor updates their pricing or features, the agent triggers a review of our translated content for inconsistencies.

This agent doesn’t just translate. It contextualizes. It adjusts tone based on current cultural trends detected in local social media feeds. It’s early days, but the efficiency gains are massive.

Summary of Findings

1. Glossaries are mandatory. Without them, LLMs drift. Build a CSV of 500+ core terms per locale.

2. Model selection matters. Use Claude 3 Haiku for speed/cost on marketing copy. Use GPT-4o Mini for accuracy on technical docs.

3. Human-in-the-loop saves money. Flag low-confidence segments. Don’t translate 100% blindly.

4. Localize metadata, don’t just translate it. Use local SEO tools to find regional search intent.

5. Optimize for AI citations. Add unique local data to avoid being ignored by AI Overviews.

Translation is no longer a linguistic task. It’s a data engineering and SEO strategy problem. Treat it like one, and your international revenue will follow.

> 写到这我突然想起之前踩过的一个坑……算了另开一篇写。

Want Better SEO Results?

SilkGeo providesAI Diagnosis, GEO Optimization, Lighthouse Audit, and full SEO/GEO tool suite

Use SilkGeo for free