fix: key Tia results per dataset row - #1799
Open
jeffreyvanhees wants to merge 1 commit into
Open
Conversation
The result cache was keyed `Class::method` on both sides, so every row of a `->with()` test shared one entry and the last writer won. On replay that single status was handed to every row. `TestMethod::id()` already appends `#dataSetName`, and `TestCase` reaches the same value object through `valueObjectForEvents()`, so both sides can use it.
jeffreyvanhees
marked this pull request as ready for review
July 30, 2026 19:02
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi @nunomaduro!
I watched Laracon US and I've been putting TIA to work immediately on a fairly large Laravel suite, but I noticed the replay didn't really agree with the run that recorded it:
Two tests that had passed came back as skipped. A plain
--no-tiarun agrees with the recording, so the replay was the odd one out.The cause is that the result cache is keyed on
Class::methodand never sees the dataset. It happens in three spots:EnsureTiaResultsAreCollectedwrites$test->className().'::'.$test->methodName()EnsureTiaAssertionsAreRecordedOnFinishedbuilds the same stringTestablereads it back as$this::class.'::'.$this->name(), andTestCase::name()gives you the bare method nameSo every row of a
->with()test writes into the same entry and the last one to finish wins. On replay that one status gets handed to all the rows without running any of them. In my case three tests callmarkTestSkipped()for one harness only, so the skip got reported for the sibling row too, which is where the extra two came from.The counts are the harmless half of it.
Graph::shouldRerunStatus()re-runs a cached failure, so red in the cache is fine, but a cached pass gets replayed onto its siblings, and a row that would have failed never runs. I haven't hit that in practice, since a change that breaks a row usually marks the file affected anyway, but it seemed worth mentioning.The fix is small:
TestMethod::id()already appends#dataSetNamewhen the test comes from a data provider, andTestCase::valueObjectForEvents()hands you the same value object, so both sides can just use that. Tests without a dataset keep the key they have today.Afterwards the replay lines up:
A few notes:
5.x(the first one prints the two rows collapsing into a single key), the third pins that a test without a dataset is untouched.composer test:unithas one failure onTests\Unit\Support\Backtrace, but it fails on a clean5.xhere too, so it isn't from this change.Happy to adjust anything, and thanks for the TIA-feature, the speedup is awesome!