Skip to content

test_gemma4_rocm_fp8_seams asserts EnvInt(name, 16) == 16 with the var unset — a tautology that made three non-existent knobs look real #845

Description

@localai-bot

What

tests/vt/test_gemma4_rocm_fp8_seams.cpp:69-71:

CHECK(EnvInt("VT_ATTN_DECODE_KV_SPLITS", 16) == 16);
CHECK(EnvInt("VT_ATTN_DECODE_SLIDE_SPLITS", 8) == 8);
CHECK(EnvInt("VT_ATTN_DECODE_SPLIT_WARPS", 12) == 12);

with the variables unset. That is default == default: it exercises the local helper's fallback return and would pass identically if the environment variables had never been invented. The test's own comment half-concedes it — "here we only document the recipe integers the lab pins (not process-wide defaults)".

Why it matters

It gives three env vars the appearance of being wired into the product when no production code reads them. VT_ATTN_DECODE_KV_SPLITS, VT_ATTN_DECODE_SLIDE_SPLITS and VT_ATTN_DECODE_SPLIT_WARPS appear only in this test file; git log -S puts their real consumers on dd1288095, a branch whose subject says "not for mudler merge".

This surfaced in PR #676, where a contributor recorded a measured KEEP recipe naming those knobs. From inside the repo they look real — a test asserts them and passes. The consequence reached a public page: docs/USAGE.md was about to instruct readers to set variables that do nothing on main.

This is the assertion-that-bounds-nothing class the project keeps hitting: a green test that would stay green under the defect it appears to guard.

Fix

Either delete the three assertions, or make them bind something real — set the variable and assert the parsed value differs from the fallback, which at least tests EnvInt. Better still, assert against the value a production call site actually reads, so the test fails when the knob is not wired up. If the knobs are lab-branch-only by design, the test should say so and assert nothing about the product.

Worth a sweep for the same shape elsewhere: EnvInt(name, X) == X with the variable unset is grep-able and is always either a tautology or a fallback test mislabelled as a contract test.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions