Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,7 @@

- Initial port of the 2026-08-04 lecture-intro.zh-cn study into a package: config-driven scan, provenance tiers by precedence, prose-only blame, edit pairs and recurring substitutions, overwrite analysis, derived review state, freshness from state files.
- Regression test reproducing the study's per-lecture numbers on the pinned checkout.
- `baseline.strategy` is validated: unknown values and the planned but unimplemented `state-file` are
rejected instead of silently scanning with `script-jump`.
- README and `docs/method.md` no longer claim character-level churn is reported; that work is tracked in
[#4](https://github.com/QuantEcon/textstrata/issues/4).
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ One YAML file per target repository — see [docs/configuration.md](docs/configu
## Limits worth knowing

- Squash merges hide human cleanup done inside a machine-drafted PR: `ai-initial` means *as landed*, so human effort is a **lower bound**.
- Blame credits the last toucher: a one-character fix claims the whole line, so human shares are an **upper bound** at line granularity. Churn is also reported in changed characters.
- Blame credits the last toucher: a one-character fix claims the whole line, so human shares are an **upper bound** at line granularity. Churn in changed characters, alongside the line counts, is planned ([#4](https://github.com/QuantEcon/textstrata/issues/4)).
- Pairing lines inside rewritten paragraphs is heuristic; category counts are indicative, not exact.
- Latin-script targets (e.g. French from English) have no script signal; the `source-diff` prose strategy for them is planned, not implemented.

Expand Down
2 changes: 1 addition & 1 deletion docs/method.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ All stock metrics count **lines containing the target script**. On raw lines mos
## Known limits

- **Squash merges** hide human work done inside a machine-drafted PR. `ai-initial` means *as landed*; human effort is a lower bound.
- **Last-toucher blame** credits a whole line to whoever changed one character of it. Human shares are an upper bound at line granularity; churn is also available in changed characters from the pairs.
- **Last-toucher blame** credits a whole line to whoever changed one character of it. Human shares are an upper bound at line granularity; reporting churn in changed characters is planned ([#4](https://github.com/QuantEcon/textstrata/issues/4)) — until then the before/after text in `pairs.jsonl` is the only character-level signal.
- **Line pairing** inside rewritten paragraphs is heuristic (similarity-matched within a hunk). Category counts are indicative.
- **Identity** is resolved by e-mail and GitHub noreply handle only; display names are ignored. Unresolved authors fall to `ai-assisted` and should be reviewed in `commits.jsonl`.
- **Pre-engine history** has no recorded engine version. Stratify flow metrics by version downstream and label the pre-engine stratum as such; do not read its rates as the shipping engine's.
Expand Down
7 changes: 6 additions & 1 deletion src/textstrata/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -145,13 +145,18 @@ def load_config(path: str | Path) -> Config:
raise ConfigError(f"prose.strategy must be script or source-diff, got {prose.strategy!r}")
if prose.strategy == "source-diff":
raise ConfigError("prose.strategy source-diff is planned but not implemented (Stage 2)")
baseline = _sub(BaselineConfig, raw.get("baseline"), "baseline")
if baseline.strategy not in ("script-jump", "state-file"):
raise ConfigError(f"baseline.strategy must be script-jump or state-file, got {baseline.strategy!r}")
if baseline.strategy == "state-file":
raise ConfigError("baseline.strategy state-file is planned but not implemented")
cfg = Config(
name=str(raw["name"]),
repo=Path(raw["repo"]),
files=str(raw.get("files", "lectures/*.md")),
source=_sub(SourceConfig, raw.get("source"), "source"),
prose=prose,
baseline=_sub(BaselineConfig, raw.get("baseline"), "baseline"),
baseline=baseline,
machine=_sub(MachineConfig, raw.get("machine"), "machine"),
disclosure=_sub(DisclosureConfig, raw.get("disclosure"), "disclosure"),
people=_sub(PeopleConfig, raw.get("people"), "people"),
Expand Down
2 changes: 1 addition & 1 deletion src/textstrata/git.py
Original file line number Diff line number Diff line change
Expand Up @@ -144,4 +144,4 @@ def diff_u0(repo: Path, sha: str, path: str) -> str:

def log_range(repo: Path, rng: str, path: str) -> list[tuple[str, str, str]]:
out = run(repo, "log", f"--format=%H{SEP}%aI{SEP}%s", rng, "--", path)
return [tuple(l.split(SEP, 2)) for l in out.splitlines() if l.strip()] # type: ignore[misc]
return [tuple(ln.split(SEP, 2)) for ln in out.splitlines() if ln.strip()] # type: ignore[misc]
4 changes: 2 additions & 2 deletions src/textstrata/scan.py
Original file line number Diff line number Diff line change
Expand Up @@ -177,8 +177,8 @@ def tier_for(f: str, sha: str) -> str:
hunks = parse_hunks(diff_u0(repo, c.sha, c.path))
except GitError:
continue
p_adds = sum(1 for h in hunks for l in h.new if prose.is_prose(l))
p_dels = sum(1 for h in hunks for l in h.old if prose.is_prose(l))
p_adds = sum(1 for h in hunks for ln in h.new if prose.is_prose(ln))
p_dels = sum(1 for h in hunks for ln in h.old if prose.is_prose(ln))
churn[tier] += p_adds + p_dels
row = rows_by_key[(f, c.sha)]
row["prose_adds"], row["prose_dels"] = p_adds, p_dels
Expand Down
23 changes: 19 additions & 4 deletions tests/test_units.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
from textstrata.config import Config, ProseConfig
import pytest

from textstrata.config import Config, ConfigError, ProseConfig, load_config
from textstrata.git import Commit, parse_trailers
from textstrata.pairs import categorise, line_pairs, parse_hunks
from textstrata.prose import Prose
Expand Down Expand Up @@ -52,8 +54,10 @@ def make_ctx(tmp_path, overrides=None):
ed = Person(id="ed", role="editor", emails=["ed@example.org"])
tr = Person(id="tr", role="translator", emails=["tr@example.org"])
for p in (ed, tr):
roster.people.append(p); roster.by_handle[p.id] = p
for e in p.emails: roster.by_email[e] = p
roster.people.append(p)
roster.by_handle[p.id] = p
for e in p.emails:
roster.by_email[e] = p
return TierContext(cfg, roster, overrides or {})


Expand Down Expand Up @@ -86,5 +90,16 @@ def test_tier_precedence(tmp_path):


def test_noreply_handle_resolution():
r = Roster(); p = Person(id="HumphreyYang", role="editor"); r.by_handle["humphreyyang"] = p
r = Roster()
p = Person(id="HumphreyYang", role="editor")
r.by_handle["humphreyyang"] = p
assert r.resolve_email("39026988+HumphreyYang@users.noreply.github.com") is p


def test_baseline_strategy_validated(tmp_path):
(tmp_path / ".git").mkdir()
for strategy, msg in (("state-file", "not implemented"), ("bogus", "must be script-jump")):
p = tmp_path / f"{strategy}.yml"
p.write_text(f"name: t\nrepo: {tmp_path}\nbaseline: {{strategy: {strategy}}}\n", encoding="utf-8")
with pytest.raises(ConfigError, match=msg):
load_config(p)
Loading