create follow-up for notion source handling when earlier classification skips occur in slack threads
- reported by <@U039RR91S> after a thread where junior initially answered that it could not verify Sentry's incident priority criteria from public sources, then ignored a later Notion link that contained the policy.
- thread context showed multiple messages marked
assistant skipped: classifier_error, including a direct Notion page URL: https://www.notion.so/sentry/Incident-Management-Policy-f25ff69ef3f24d80aad3ab74ab164e19?...
- current behavior gap: junior did not recover from the earlier skipped messages and did not use the newly provided Notion source before answering / concluding.
- likely expected behavior:
- when a follow-up user message includes a concrete internal doc source like a Notion URL, junior should inspect it if the task is still active or if later asked to explain the miss.
- classifier skips in intermediate thread messages should not prevent later evidence gathering from thread context.
- if a prior answer said something was unverified publicly, a subsequent internal source in-thread should trigger re-evaluation instead of staying anchored on the original conclusion.
- investigation starting points:
- how thread transcripts are surfaced to the model when some turns are tagged
classifier_error
- whether explicit URLs in skipped messages are omitted from tool-routing or evidence-gathering logic
- whether Notion skill selection is too dependent on the current turn text and ignores newly surfaced thread context
- whether follow-up repair behavior should scan the full thread for concrete sources before answering "why didn’t you use X"
- impact: missed internal source, lower trust, unnecessary manual correction in-channel.
references
- conversation id:
slack:C0AU9QHAVFD:1776889343.944739
- trace id:
dc5b58b9444b2e15679429ffa63f8b0b
create follow-up for notion source handling when earlier classification skips occur in slack threads
assistant skipped: classifier_error, including a direct Notion page URL:https://www.notion.so/sentry/Incident-Management-Policy-f25ff69ef3f24d80aad3ab74ab164e19?...classifier_errorreferences
slack:C0AU9QHAVFD:1776889343.944739dc5b58b9444b2e15679429ffa63f8b0b