fix: skip llm parsing for oversized project affiliation files

[skwowet]

Summary

Adds a 1MB size limit for project-registry affiliation files. Files over the limit are marked unusable and skip Bedrock parsing, preventing multi-hour failures on huge data dumps (e.g. cncf/devstats-example / github_users.json). File hashing is unified to always use raw bytes on disk, with decode only used for LLM parsing.

Changes

• Add MAX_FILE_SIZE_BYTES (1MB) to AffiliationService
• Check file size with os.path.getsize before loading content into memory
• First oversize hit: stream-hash raw bytes, raise AffiliationAnalysisError with retain_file_hash=True → registry marked unusable
• Steady state (already unusable for same path): getsize only, reuse stored hash, skip LLM
• Files under 1MB: unchanged LLM parse path via resolve_snapshot
• Replace decode-then-UTF-8-encode hashing with compute_file_hash_from_path — single chunked reader that hashes raw bytes; retain_content=True only when content is needed for parsing
• Fixes hash mismatch for non-UTF-8 files that could incorrectly invalidate registry cache when crossing the size threshold

[github-actions]

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

• feat: add user authentication (CM-123)
• feat: add user authentication (IN-123)

Projects:

• CM: Community Data Platform
• IN: Insights

Please add a Jira issue key to your PR title.

[Copilot]

Pull request overview

This PR addresses multi-hour failures in the git_integration affiliation pipeline caused by very large project-registry affiliation files (e.g. cncf/devstats-example/github_users.json) being sent to Bedrock/LLM parsing. It introduces a 1 MB size guard: oversized files are marked unusable in the affiliation registry and skip LLM parsing, while file hashing is unified to always hash raw bytes on disk (decoding only when content is actually needed for parsing).

Changes:

• Add MAX_FILE_SIZE_BYTES = 1_000_000 and a pre-load os.path.getsize check that short-circuits LLM parsing for oversized files (first hit hashes + raises AffiliationAnalysisError(retain_file_hash=True) → registry UNUSABLE; steady state reuses the stored hash and skips parsing).
• Replace compute_file_hash(content: str) with async compute_file_hash_from_path(...), a chunked reader that hashes raw bytes and optionally retains content, fixing hash mismatches for non-UTF-8 files.
• Route under-limit files through the unchanged resolve_snapshot path after decoding retained bytes with safe_decode.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 2 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 2 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 2 changed files in this pull request and generated no new comments.