Skip to content

fix: use text_config.eos_token_id for HunyuanOCR v1.5#25514

Open
wendadawen wants to merge 1 commit into
ggml-org:masterfrom
wendadawen:fix/hunyuanocr-128k-eos
Open

fix: use text_config.eos_token_id for HunyuanOCR v1.5#25514
wendadawen wants to merge 1 commit into
ggml-org:masterfrom
wendadawen:fix/hunyuanocr-128k-eos

Conversation

@wendadawen

Copy link
Copy Markdown
Contributor

Fix incorrect eos_token_id in HunyuanOCR v1.5 conversion causing inference to never stop

Problem

HunyuanOCR v1.5 is an updated version of HunyuanOCR v1.0. After converting v1.5 with convert_hf_to_gguf.py, the GGUF eos_token_id is incorrectly written as 120020, while the model's real stop token is 120007 from text_config.eos_token_id.

As a result, when llama.cpp generates 120007 it is not recognized as a stop token, and the model degenerates into repetitive generation until truncated by max_tokens (finish=length).

Root cause: v1.5's config.json stores the real stop token in the nested text_config.eos_token_id, while the top-level eos_token_id=120020 is a different token. The parent class HunYuanModel._fix_special_tokens only reads the top-level value.

Fix

Override _fix_special_tokens in HunyuanVLTextModel to overwrite with hparams["eos_token_id"] (base.py already merges text_config.eos_token_id into hparams) after calling the parent:

def _fix_special_tokens(self):
    super()._fix_special_tokens()
    eos_id = self.hparams.get("eos_token_id")
    if eos_id is not None:
        self.gguf_writer.add_eos_token_id(eos_id)

Test

Tested v1.0 and v1.5 on 20 cases:

  • v1.5: all degenerated (finish=length) before fix; 20/20 normal stop after fix
  • v1.0: no change before/after fix (v1.0 has no text_config.eos_token_id, GGUF unaffected)

Model Weights

@wendadawen wendadawen requested a review from CISC as a code owner July 10, 2026 03:11
@wendadawen wendadawen marked this pull request as draft July 10, 2026 03:23
@wendadawen wendadawen marked this pull request as ready for review July 10, 2026 05:20
Signed-off-by: wendadawen <wendadawen@qq.com>
@wendadawen wendadawen force-pushed the fix/hunyuanocr-128k-eos branch from bb20964 to de8e1d5 Compare July 12, 2026 05:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant