Tencent Hy4 preview: 770B Parameters, 49B Active, 1M-Token Context — The Complete Guide (2026)

What Hy4 preview is: Tencent’s open flagship, shipped preview-first

Hy4 preview is the next-generation flagship of Tencent’s Hunyuan (混元) model family — Tencent’s answer to Alibaba’s Qwen, Zhipu’s GLM, and the DeepSeek line. "Preview" is the deliberate part: Tencent has settled into a rhythm of roughly one major model iteration every two months since February, and every release ships as a preview first, collects real-world feedback, and is followed by an official release. Hy4 preview is the preview step for the Hy4 generation — and unlike most previews, it is fully open from minute zero.

What makes the launch worth a guide rather than a headline is the combination you rarely see from a frontier-scale lab: the largest open-weight model Tencent has shipped, Apache 2.0 licensed, with serving support merged into vLLM on the same day. The model also came with a prebuilt Docker image, an SGLang image, and a working Ascend path for Chinese accelerators. Open-weight releases usually take weeks for the serving ecosystem to catch up; this one took hours.

The spec sheet

The lineage puts the jump in context — Hy3 to Hy4 preview roughly doubled active capacity and quadrupled the context window:

Architecture: the interesting bit is why it’s cheap to serve

Try to describe Hy4 preview as "just another big MoE" and you’ll miss the actual story, which is that every design choice exists to keep a 770B model affordable to run and to serve. The pieces matter individually, but together they are a coherent argument about how a Chinese lab can ship frontier-adjacent capability at prices US labs can’t match.

The MoE layout: 256 experts, top-8, one dense layer

The backbone is 78 layers: the first is a standard dense FFN, and the remaining 77 are MoE layers, each holding 256 routed experts plus one shared expert. Every token activates the top-8 routed experts along with the shared one — a roughly 16:1 sparsity ratio. That ratio is the difference between a model that needs a national grid and a model a serious team can actually put into production: you pay for 49B active parameters per token, not 770B.

Gated DeepSeek Sparse Attention with IndexCache

The attention design is where Hy4 preview gets interesting. It uses Gated DeepSeek Sparse Attention (Gated DSA) — the same sparse-attention family DeepSeek introduced — combined with IndexCache, a cross-layer sparse-index reuse scheme Tencent says was "inspired by DeepSeek and GLM." Instead of paying dense attention over the full 1M-token context, an indexer selects a top-k of 2,048 relevant tokens per query, and only 21 of the 78 layers compute their own sparse index while the other 57 reuse it. Think of a library where a card index is compiled once and shared across floors, rather than every floor re-cataloguing the building.

The headline number: with Gated DSA, the per-token attention cost stops scaling with the total context length the way dense attention does — which is what makes a 1M context window something you can actually serve on your own GPUs rather than a marketing slide.

iHC residuals and the built-in MTP layer

Two less-hyped choices round out the design. First, the residual pathway uses iHC (identity Hyper-Connections), giving the model four parallel residual streams instead of one — a technique for improving inter-layer information flow and training stability in very deep models (the README cites the approach as part of why the 78-layer stack trains and scales cleanly). Second, and importantly for serving: the checkpoint embeds a native MTP (multi-token prediction) layer of ~10B total parameters / 0.7B active, with draft depth three. That means speculative decoding is built in — vLLM and SGLang can draft tokens ahead using the model’s own MTP head, with no separate draft model to deploy. For a 770B model, not needing a second, smaller draft model is a genuinely big deal operationally.

Quantization story

Tencent published both BF16 and FP8 checkpoints on day one across Hugging Face, ModelScope, GitCode and CNB — and the official vLLM recipe targets the FP8 build, which is what makes the "one command, eight GPUs" story workable. At FP8 with 49B active parameters, this is a model that fits into serious-but-not-fantastical multi-GPU nodes, unlike the full-BF16 flagship tier from some rivals.

Benchmarks: what Tencent claims — and what we still don’t know

Here is the single most important sentence in this whole guide: every benchmark Tencent published for Hy4 preview is self-reported, run on Tencent’s own evaluation setup, and unreproduced by any independent lab. The model has been public only since August 28, 2026. Nothing on the public leaderboards reflects it. Read the numbers below as the ceiling Tencent believes it hit, not as established fact — and re-check before making deployment decisions.

The Hy3 → Hy4 preview jump (Tencent’s table)

Tencent calls DeepSWE’s 28.0 → 64.3 and the overall table the largest generation-over-generation gain it has measured. The headline row is Terminal-Bench 2.1 at 85.4 — the claim that puts an open-weight model at parity with the most expensive closed frontier models on a hard agentic-coding test — and it is exactly the claim most in need of independent confirmation.

The blind expert evaluation

Separately from leaderboard-style benchmarks, Tencent ran a blind side-by-side evaluation with 163 internal experts over 203 engineering tasks, scoring output quality from 1 to 4:

Hy4 preview edges both rivals on average score, and the win-rate split is revealing: against GLM-5.3 it wins 46.8% of pairings, ties 12.8%, and loses 40.4% — "ahead on average" hides that it loses two out of every five direct comparisons. That is the honest shape of the model: genuinely in the same class as GLM-5.3 and Kimi K3, not a clean sweep.

Against the closed frontier (Tencent’s chart)

The shape of the claim: SWE-bench Pro 65.7 vs 64.6 for both GPT-5.6 Sol and GLM-5.3; Terminal-Bench 2.1 tying Sol at 85.4; Toolathlon-Verified 74.1 "surpassing" Sol and Claude Opus 5; APEX-Agents 37.1, a hair behind Kimi K3’s 37.2. Notice what’s absent — no SWE-bench Verified, no HLE, no reasoning-heavy leaderboard publication, and no multimodal rows, consistent with a text-only preview.

Built for productivity — and for improving itself

Tencent positions Hy4 preview as a "built for productivity" model, trained partly on high-quality data co-created with Tencent’s own domain experts in software engineering, gaming, finance and security, and co-designed with its products (CodeBuddy, WorkBuddy, Yuanbao, ima). The concrete claims, per capability area:

• Software engineering — stronger understanding, planning, debugging and validation on long-context development tasks, plus better visual quality and interaction polish for front-end generation.
• Office & finance — deeper understanding of complex working environments, improved financial analysis, and the full data-analysis workflow from raw information to documents, spreadsheets and presentations.
• Game development — a playable prototype from a single natural-language request, working with game engines, refinable through multi-turn interaction.
• Scientific research — improved reasoning on AI R&D, molecular dynamics simulation, condensed-matter physics and fundamental mathematics.

The self-improvement loop: the model helped build itself

Two details in Tencent’s announcement deserve more attention than they got. First, Hy4 preview participated in its own development for the first time — it contributed to the automated optimization of training methods, data strategies, evaluation frameworks, and low-level operators, proposing approaches, running experiments, and feeding code, logs and feedback into the next round. Tencent calls this an "early-stage recursive self-improvement loop," which is remarkable language for a company release. Second, the model autonomously analyzed bottlenecks in its own inference system and ran multiple rounds of operator-fusion and communication optimization, improving end-to-end throughput by 31.8% against baseline with consistent gains across context lengths and concurrency levels. That is not future talk: that is a shipped model claiming to have made its own serving stack 32% faster.

Pricing, API, and the reasoning-mode landmine

The headline answer

Two clarifications, because both have burned people already. First, this is promotional-free flat pricing for a preview — Tencent has not announced discounts or off-peak windows for Hy4 (unlike DeepSeek’s tariff model, covered in our DeepSeek V4 guide). Second, preview pricing is explicitly subject to change before the official Hy4 release.

Endpoints

The compatibility story is unusually good for a Chinese open model: the API speaks OpenAI Chat Completions everywhere (most existing code just changes base_url and model ID), and TokenHub additionally advertises OpenAI Responses and Anthropic Messages compatibility — meaning Claude-shaped agent tooling can point at Hy4 preview with minimal changes, the same trick Z.ai offers with GLM-5.3.

Your first call

curl https://tokenhub.tencentmaas.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TENCENT_TOKENHUB_KEY" \
  -d '{
    "model": "hy4-preview",
    "messages": [
      {"role": "user", "content": "Explain Mixture-of-Experts in one sentence."}
    ],
    "temperature": 0.9,
    "top_p": 1.0
  }'

Official sampling guidance is temperature 0.9, top_p 1.0. Reasoning output arrives separately from the answer: the chain-of-thought comes back in reasoning_content and the final answer in content. If you’re using the Python OpenAI SDK, reasoning_content isn’t in its type definitions — grab it with getattr(msg, "reasoning_content", None).

Self-hosting: day-zero vLLM and SGLang support

The self-hosting story is the reason this guide exists in a year full of open-model releases: Hy4 preview launched with a ready-made vLLM container rather than just a checkpoint. The vLLM support (PR #54160) landed in the same window as the release, the official recipe targets vLLM 0.29.0+, and the official image is prebuilt. In practice, serving is one command rather than a weekend of kernel debugging:

docker run --gpus all -p 8000:8000 --ipc=host \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:hy4-preview \
  tencent/Hy4-preview-FP8 \
  --tensor-parallel-size 8 \
  --speculative-config '{"num_speculative_tokens":3,"method":"mtp"}' \
  --attention-backend FLASHMLA_SPARSE \
  --tool-call-parser hy_v4 \
  --reasoning-parser hy_v4 \
  --enable-auto-tool-choice \
  --served-model-name hy4-preview

Every flag maps to something real in the model:

• --attention-backend FLASHMLA_SPARSE — required, not optional. Gated DSA does not run correctly on the default dense backends; this flag is the official configuration.
• --speculative-config {"num_speculative_tokens":3,"method":"mtp"} — turns on the model’s built-in MTP layer for speculative decoding, three draft tokens ahead, with no separate draft model to deploy.
• --tool-call-parser hy_v4 and --reasoning-parser hy_v4 — tell the server how Hy4 preview emits tool calls and chain-of-thought, and --enable-auto-tool-choice lets the model decide when to call tools.

Day-zero support isn’t vLLM-only. SGLang shipped a prebuilt multi-architecture image (lmsysorg/sglang:hy4-preview) with NEXTN-style speculative decoding the same day, and the Ascend ecosystem landed 0-day support through vLLM-Ascend using W8A8-quantized weights across 16 Atlas 800I A3 NPUs — the domestic-silicon path matters given the export-control backdrop covered in Why China Is Winning the AI Race. Third-party tools with support on day one include Cline, OpenCode Go, and Vercel AI Gateway.

How Hy4 preview compares

vs GLM-5.3 — the direct rival, and what "ahead" actually means

These are the two most comparable open flagships shipping in the same August week, and the head-to-head is genuinely close. Tencent’s blind eval puts Hy4 preview at 2.99 vs GLM-5.3’s 2.92, with 46.8% wins but a 40.4% loss rate — "ahead on average," not dominant. On price, Hy4 preview is cheaper per token ($0.834/$2.501 vs $1.40/$4.40). The structural differences matter more than the averages: Hy4 preview’s Apache 2.0 weights are downloadable today, where GLM-5.3’s flagship weights remain withheld for safety evaluation (see our GLM-5.3 guide) — and Hy4 preview ships with day-zero serving tooling that Z.ai’s flagship still can’t offer because you can’t download it.

vs Kimi K3 — the closest margin on agentic coding

Kimi K3 (2.8T total) remains the bigger and more expensive model ($3/$15 vs $0.834/$2.501). On Tencent’s numbers they trade blows at the margin: APEX-Agents 37.1 vs K3’s 37.2, and the blind eval 2.99 vs 2.94 with Hy4 winning 51.2% of pairings. K3’s advantages are native vision and a longer head start; Hy4 preview’s are price, open weights on day one, and the serving story. If Tencent’s chart survives independent testing, this is the cheapest way into the frontier agentic-coding tier that exists today.

vs DeepSeek V4 — the efficiency war continues

DeepSeek’s V4 line is the other end of the same trend: MIT-licensed open weights, sparse attention, deliberately cheap inference. But note the architectural borrowings are directional: Hy4 preview’s attention is explicitly "Gated DeepSeek Sparse Attention," and its README credits DeepSeek and GLM as inspirations — the Chinese open-model ecosystem is now openly iterating on each other’s ideas at a pace no single lab can match, with all the strategic implications covered in our DeepSeek V4 guide.

vs the closed frontier

On Tencent’s chart, Hy4 preview’s SWE-bench Pro (65.7) and Terminal-Bench 2.1 (85.4) sit at parity with GPT-5.6 Sol (64.6 / 85.4), and Toolathlon-Verified 74.1 is claimed above Sol and Claude Opus 5. The measured-from-scratch verdict is still out: no independent lab has run these, the model has no published results on reasoning-heavy or multimodal leaderboards, and the same August week produced multiple reminders that vendor tables are ceilings, not facts. What is already true: at this price, with these weights, the open tier is no longer a "good enough" story — it is a cost-structure story.

Limitations — the honest list

Frequently asked questions

Sources & further reading