feat: Add clinical-note-red-flag-scanner template - #296
Open
Vilsee wants to merge 6 commits into
Open
Conversation
Adds a clinical documentation compliance analysis template that accepts a clinical note as text and returns a structured, severity-ranked JSON list of documentation red flags (missing consent, drug interactions, incomplete vitals, ambiguous dosing, etc.) for clinician and compliance reviewer triage. - Single-flow template (API Request -> LLM Analyse Note -> Variables -> API Response) - 10 red-flag categories with 4 severity levels (CRITICAL/HIGH/MEDIUM/LOW) - Uses Gemini (gemini-3.1-flash-lite-preview) for analysis - Structured JSON output with per-flag reasoning, location, and remediation - Includes comprehensive agent.md, README with tested example I/O, and safety constitution
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughChangesAdds a Clinical Note Red-Flag Scanner template with Gemini analysis, structured JSON output rules, a four-node execution flow, safety guardrails, documentation, model configuration, and Lamatic packaging metadata. Clinical Note Red-Flag Scanner
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
:robot_face: AgentKit Structural ValidationNew Contributions Detected
Check Results
🎉 All checks passed! This contribution follows the AgentKit structure. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 13
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@kits/clinical-note-red-flag-scanner/agent.md`:
- Around line 15-19: Revise the standards-alignment section and guardrail claims
in kits/clinical-note-red-flag-scanner/agent.md at lines 15-19 to state only
that the template may support documentation-review workflows, and explicitly
disclaim regulatory compliance certification. Apply the same narrowed wording
and disclaimer to the user-facing compliance statement in
kits/clinical-note-red-flag-scanner/README.md at lines 12-14.
- Around line 29-33: Update the input-note guidance in the agent documentation
to remove the “any length” claim. Define and enforce a maximum supported note
size or implement chunking before scanning, document the actual limit and
behavior for oversized notes, and keep the required clinicalNote input contract
unchanged.
In `@kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts`:
- Around line 159-163: Update the finalText mapping in the Variables node to
reference only LLMNode_453.output.generatedResponse, removing the trailing
literal “{” so the complete model response remains valid JSON.
- Around line 110-115: Update the API Request trigger configuration for nodeName
"API Request" so advance_schema requires clinicalNote to be a non-empty,
non-whitespace string. If the schema cannot express trimming, add an explicit
guard immediately before the LLM node that rejects empty or whitespace-only
clinicalNote values before scanning.
In `@kits/clinical-note-red-flag-scanner/lamatic.config.ts`:
- Around line 9-12: Add the deployment environment key to the mandatory step
configuration identified by id "clinical-note-red-flag-scanner", using the step
ID expected by runtime flow resolution so it can read the deployed flow ID from
process.env. Preserve the existing id and type values.
In
`@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md`:
- Line 13: Update the REGULATORY guidance in the clinical-note red-flag scanner
prompt to require jurisdiction and care-setting context before emitting
regulatory findings. If those inputs are unavailable, restrict findings to
requirements explicitly identified by the caller or documented in the note; do
not make jurisdiction-specific claims from clinicalNote alone.
- Line 5: Update the DRUG_INTERACTION category definition in the system prompt
to avoid claiming definitive or exhaustive detection without an authoritative
interaction source; either specify a versioned, clinically verified interaction
database/tool, or restrict findings to interactions explicitly documented in the
note and label screening as non-exhaustive.
- Around line 23-27: Update the recommendation field instructions and the
prohibition near the medical-advice guidance to require documentation-only
remediation actions. Explicitly prohibit treatment choices, dosing regimens,
diagnoses, and triage or emergency-care advice, and revise the README example to
remove nitroglycerin and emergency-care instructions.
- Around line 15-30: Update the JSON response contract in the system prompt to
explicitly require that flagCount equals flags.length, flag IDs are unique and
sequential starting at FLAG-001, and category and severity values use only the
defined valid codes. Require responses to satisfy these invariants in addition
to the existing structure.
- Around line 37-42: Update the evidence guidance near the flag-generation rules
so `detail` and `location` use redacted or paraphrased evidence rather than
reproducing exact patient text. Explicitly prohibit names, MRNs, DOBs, contact
information, and other identifiers while preserving enough non-identifying
context to support each flag.
In
`@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_user_1.md`:
- Around line 1-2: Update the clinicalNote interpolation in the clinical-note
scanner prompt to prevent raw PHI from being sent to Gemini. Require a
pre-redacted or de-identified note, or enforce a documented compliant
zero-retention provider configuration before forwarding the content; do not rely
on prompt-level confidentiality instructions.
- Around line 1-2: Update the clinical-note prompt content around
triggerNode_1.output.clinicalNote to clearly delimit the note as untrusted data
and explicitly instruct the model to ignore any commands or instructions
contained within it, while preserving the required JSON, confidentiality, and
safety behavior.
In `@kits/clinical-note-red-flag-scanner/README.md`:
- Around line 1-8: Move the “# Clinical Note Red-Flag Scanner” H1 above the
deploy badge HTML so it is the README’s first content and satisfies MD041. Keep
the existing badge markup unchanged below the heading.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 83786c0b-8632-4fb4-8f70-44be325cb8c6
📒 Files selected for processing (9)
kits/clinical-note-red-flag-scanner/.gitignorekits/clinical-note-red-flag-scanner/README.mdkits/clinical-note-red-flag-scanner/agent.mdkits/clinical-note-red-flag-scanner/constitutions/default.mdkits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.tskits/clinical-note-red-flag-scanner/lamatic.config.tskits/clinical-note-red-flag-scanner/model-configs/clinical-note-red-flag-scanner_llmnode-453_generative-model-name.tskits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.mdkits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_user_1.md
Comment on lines
+23
to
+27
| "severity": "CRITICAL | HIGH | MEDIUM | LOW", | ||
| "title": "Brief flag title", | ||
| "detail": "Specific explanation of what is missing or problematic", | ||
| "location": "Where in the note this issue was found (or 'ABSENT' if the issue is something missing entirely)", | ||
| "recommendation": "Specific action to remediate this flag" |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Mission-critical: keep remediation guidance documentation-only.
The recommendation field asks for specific actions while Line 41 prohibits medical advice; the README example already supplies a nitroglycerin regimen and emergency-care instructions. Require documentation actions only, and explicitly prohibit treatment choices, dosing regimens, diagnoses, and triage advice.
Also applies to: 41-42
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md`
around lines 23 - 27, Update the recommendation field instructions and the
prohibition near the medical-advice guidance to require documentation-only
remediation actions. Explicitly prohibit treatment choices, dosing regimens,
diagnoses, and triage or emergency-care advice, and revise the README example to
remove nitroglycerin and emergency-care instructions.
- Fix trailing brace in variablesNode mapping - Add envKey to lamatic.config.ts - Add minLength validation to flow advance_schema - Narrow compliance claims and add data privacy warnings in README and agent.md - Add token limit to agent.md - Restrict system prompt DRUG_INTERACTION and REGULATORY claims - Enforce documentation-only recommendations and PII redaction - Add prompt injection defense to user prompt - Enforce strict JSON invariants - Fix markdownlint warnings
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
kits/clinical-note-red-flag-scanner/README.md (3)
67-69: 🔒 Security & Privacy | 🟡 Minor | ⚡ Quick winKeep the public example fully synthetic and non-identifying.
The sample uses patient-like clinical data and demonstrates reproducing MRN/DOB fields, contradicting the redaction requirements. Replace it with clearly synthetic, de-identified content and avoid showing identifier reproduction.
Also applies to: 87-89
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/README.md` around lines 67 - 69, Replace the clinicalNote example in the README with clearly synthetic, non-identifying content that does not resemble real patient data or include names, demographics, dates of birth, MRNs, or other identifiers. Update the related example at the referenced reproduction section as well, removing any demonstration of identifier reproduction while preserving the red-flag scanner usage context.
105-116: 🔒 Security & Privacy | 🟠 MajorRemove treatment and triage advice from the example output.
The sample recommends medication dosing and emergency-care instructions, while the system contract permits documentation-only remediation. Revise the example to use actions such as documenting missing fields, not treatment or return-to-ED guidance.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/README.md` around lines 105 - 116, Update the example output entries FLAG-003 and FLAG-004 in the README so their recommendation text only instructs users to document missing clinical fields or plan details. Remove medication dosing guidance and emergency-care or return-to-ED instructions while preserving the existing finding context.
73-80: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winAlign the documented response shape with the runtime contract.
This section says
resultcontains a JSON object and shows the report without the outerresultfield, while the flow and agent documentation defineresultas a JSON string. Show the actual wrapper and serialization, or change the implementation and all documentation consistently.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/README.md` around lines 73 - 80, Update the “Example Output” section to match the runtime contract: show the outer result field and represent its value as the serialized JSON string returned by the flow. Keep the documented summary and flagCount payload consistent with the implementation and related flow/agent documentation.
♻️ Duplicate comments (1)
kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts (1)
114-114: 🩺 Stability & Availability | 🟠 MajorEnforce the documented maximum note size.
The runtime accepts arbitrarily large notes while the documentation promises a 10,000-token limit. Add bounded validation or chunking before the single LLM request.
kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts#L114-L114: add a maximum-size guard or chunking path to the trigger contract.kits/clinical-note-red-flag-scanner/agent.md#L33-L33: document the enforced limit and oversized-input behavior.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts` at line 114, Enforce the documented 10,000-token maximum in the clinical note trigger contract around the advance_schema definition by adding bounded validation or chunking before the single LLM request. In kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts:114, reject or appropriately process oversized clinicalNote values; in kits/clinical-note-red-flag-scanner/agent.md:33, document the enforced limit and the behavior for oversized input.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md`:
- Line 13: Update the REGULATORY guidance in the system prompt to require
jurisdiction and care setting before making jurisdiction-specific findings. When
those inputs are unavailable, limit findings to requirements explicitly
identified by the caller or clinical note and label them non-jurisdictional.
In
`@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_user_1.md`:
- Line 4: Update the prompt input around clinicalNote so raw clinical notes
cannot be forwarded directly to Gemini without a data-control boundary. Require
and validate pre-redacted/de-identified content, or verify a compliant
zero-retention provider configuration before executing the node; otherwise block
execution rather than interpolating the complete note.
---
Outside diff comments:
In `@kits/clinical-note-red-flag-scanner/README.md`:
- Around line 67-69: Replace the clinicalNote example in the README with clearly
synthetic, non-identifying content that does not resemble real patient data or
include names, demographics, dates of birth, MRNs, or other identifiers. Update
the related example at the referenced reproduction section as well, removing any
demonstration of identifier reproduction while preserving the red-flag scanner
usage context.
- Around line 105-116: Update the example output entries FLAG-003 and FLAG-004
in the README so their recommendation text only instructs users to document
missing clinical fields or plan details. Remove medication dosing guidance and
emergency-care or return-to-ED instructions while preserving the existing
finding context.
- Around line 73-80: Update the “Example Output” section to match the runtime
contract: show the outer result field and represent its value as the serialized
JSON string returned by the flow. Keep the documented summary and flagCount
payload consistent with the implementation and related flow/agent documentation.
---
Duplicate comments:
In `@kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts`:
- Line 114: Enforce the documented 10,000-token maximum in the clinical note
trigger contract around the advance_schema definition by adding bounded
validation or chunking before the single LLM request. In
kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts:114,
reject or appropriately process oversized clinicalNote values; in
kits/clinical-note-red-flag-scanner/agent.md:33, document the enforced limit and
the behavior for oversized input.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: fe830c49-c5c0-4bba-bc7e-97d970a0f9b7
📒 Files selected for processing (6)
kits/clinical-note-red-flag-scanner/README.mdkits/clinical-note-red-flag-scanner/agent.mdkits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.tskits/clinical-note-red-flag-scanner/lamatic.config.tskits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.mdkits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_user_1.md
| 7. **FOLLOW_UP** — Missing follow-up plan, discharge instructions, or continuity-of-care documentation | ||
| 8. **ASSESSMENT** — Missing or incomplete clinical assessment/diagnosis relative to documented symptoms | ||
| 9. **IDENTITY** — Missing or incomplete patient identification markers (MRN, DOB, provider signature/attestation) | ||
| 10. **REGULATORY** — Documentation gaps that may violate specific regulatory requirements (CMS, Joint Commission, state-level mandates, subject to applicable jurisdiction and care setting) |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major
Mission-critical: require jurisdiction before regulatory findings.
The flow accepts only clinicalNote, so the model cannot reliably determine which CMS, Joint Commission, or state requirements apply. If jurisdiction and care setting are unavailable, restrict findings to requirements explicitly identified by the caller or note and label them non-jurisdictional.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md`
at line 13, Update the REGULATORY guidance in the system prompt to require
jurisdiction and care setting before making jurisdiction-specific findings. When
those inputs are unavailable, limit findings to requirements explicitly
identified by the caller or clinical note and label them non-jurisdictional.
- Replace realistic PHI with synthetic placeholders in README example - Strip remaining clinical advice from FLAG-003 and FLAG-004 examples in README - Wrap JSON example output in stringified result field in README - Enforce 45000 character limit in flow advance_schema - Document enforced token limit and rejection behavior in agent.md - Strengthen REGULATORY caveat in system prompt (advisory only, caller provides jurisdiction) - Add hard PHI input contract to user prompt
Contributor
|
/validate |
Contributor
|
📡 Running Studio validation — results will appear here shortly. |
Contributor
Studio Runtime Validation (Phase 2)✅ Studio validation passed. The kit loaded successfully in Lamatic Studio. This PR is ready for final review and merge. |
Contributor
|
@coderabbitai review |
Contributor
✅ Action performedReview finished.
|
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
kits/clinical-note-red-flag-scanner/README.md (2)
20-20:⚠️ Potential issue | 🟠 MajorMission-critical: align interaction detection with the non-exhaustive model contract.
The README claims “known unaddressed interactions,” while the system prompt correctly states that this is non-exhaustive screening without a verified interaction database. Reword this as interactions explicitly documented or suggested by the note, and state that it is not definitive interaction checking.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/README.md` at line 20, Update the DRUG_INTERACTION description in the README to describe only medication interactions explicitly documented or suggested by the clinical note, and clearly state that the scanner does not provide definitive or exhaustive interaction checking.
75-80: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winDescribe
resultas a JSON string, not a JSON object.The example correctly wraps the serialized report inside
"result", but the prose currently says it contains a JSON object. Document it as a string containing JSON to match the flow andagent.md.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/README.md` around lines 75 - 80, Update the README prose describing the flow’s result field to state that result is a JSON string containing the serialized report, not a JSON object. Keep the existing example and JSON structure unchanged.kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts (1)
161-181: 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy liftValidate the LLM report before returning it.
The model’s raw
generatedResponseis copied directly into the API result. If Gemini emits markdown, malformed JSON, or violatesflagCount/ID/category invariants, the documented response contract is broken. Add schema-backed structured output or parse and reject invalid reports beforeAPI Response.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts` around lines 161 - 181, Validate and normalize the LLM report before the responseNode named “API Response” returns it: parse the generatedResponse from LLMNode_453, enforce the documented schema and flagCount/ID/category invariants, and reject malformed or markdown-wrapped output. Update the variablesNode_197 mapping so outputMapping uses only the validated report, preserving the existing finalText/result contract for valid responses.
♻️ Duplicate comments (2)
kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts (2)
114-114:⚠️ Potential issue | 🟠 MajorProvide jurisdiction context before emitting regulatory findings.
The flow accepts only
clinicalNote, but the prompt requires caller-supplied jurisdiction and care-setting context. Add those inputs, or restrictREGULATORYfindings to requirements explicitly identified by the note/caller and label them non-jurisdictional.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts` at line 114, Add jurisdiction and care-setting inputs to the flow’s advance_schema and make them available to the regulatory-analysis prompt before emitting REGULATORY findings. Alternatively, constrain REGULATORY results to requirements explicitly identified by the clinical note or caller and label those findings non-jurisdictional; preserve the existing clinicalNote validation.
114-114:⚠️ Potential issue | 🟠 MajorMission-critical: enforce the PHI boundary before Gemini execution.
advance_schemavalidates only string length and whitespace, so raw PHI can still be submitted despite the user prompt’s de-identification contract. Require an enforced pre-redaction/attestation gate or a verified compliant provider configuration before forwarding the note.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts` at line 114, Update the clinical note execution flow around the advance_schema configuration to enforce the PHI boundary before any Gemini request: require a completed pre-redaction/attestation gate or verify a compliant provider configuration, and reject or stop processing when neither condition is satisfied. Do not rely on the schema’s length or whitespace validation as de-identification.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@kits/clinical-note-red-flag-scanner/README.md`:
- Line 79: Update the example JSON result so its summary matches the emitted
flags: either add a corresponding CONSENT entry to the flags array, or remove
the informed-consent claim from the summary while preserving the existing
five-flag output.
---
Outside diff comments:
In `@kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts`:
- Around line 161-181: Validate and normalize the LLM report before the
responseNode named “API Response” returns it: parse the generatedResponse from
LLMNode_453, enforce the documented schema and flagCount/ID/category invariants,
and reject malformed or markdown-wrapped output. Update the variablesNode_197
mapping so outputMapping uses only the validated report, preserving the existing
finalText/result contract for valid responses.
In `@kits/clinical-note-red-flag-scanner/README.md`:
- Line 20: Update the DRUG_INTERACTION description in the README to describe
only medication interactions explicitly documented or suggested by the clinical
note, and clearly state that the scanner does not provide definitive or
exhaustive interaction checking.
- Around line 75-80: Update the README prose describing the flow’s result field
to state that result is a JSON string containing the serialized report, not a
JSON object. Keep the existing example and JSON structure unchanged.
---
Duplicate comments:
In `@kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts`:
- Line 114: Add jurisdiction and care-setting inputs to the flow’s
advance_schema and make them available to the regulatory-analysis prompt before
emitting REGULATORY findings. Alternatively, constrain REGULATORY results to
requirements explicitly identified by the clinical note or caller and label
those findings non-jurisdictional; preserve the existing clinicalNote
validation.
- Line 114: Update the clinical note execution flow around the advance_schema
configuration to enforce the PHI boundary before any Gemini request: require a
completed pre-redaction/attestation gate or verify a compliant provider
configuration, and reject or stop processing when neither condition is
satisfied. Do not rely on the schema’s length or whitespace validation as
de-identification.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: e59d79d3-fd4b-4c08-9b2e-259d519d411f
📒 Files selected for processing (5)
kits/clinical-note-red-flag-scanner/README.mdkits/clinical-note-red-flag-scanner/agent.mdkits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.tskits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.mdkits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_user_1.md
- Narrow DRUG_INTERACTION description in README - Clarify README result payload description as JSON string - Fix example output summary to match flags array - Add summary invariant to system prompt - Document JSON Output Validation, Jurisdiction, and PHI Enforcement limitations in agent.md and README
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (4)
kits/clinical-note-red-flag-scanner/agent.md (1)
92-93: 🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy liftShared root cause: zero retention is presented as equivalent to PHI safety.
Zero retention alone does not establish provider authorization or the contractual and security controls required for PHI processing.
kits/clinical-note-red-flag-scanner/agent.md#L92-L93: require independently verified PHI-processing authorization and controls.kits/clinical-note-red-flag-scanner/README.md#L61: apply the same prerequisite wording to the setup instructions.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/agent.md` around lines 92 - 93, Update the Data Privacy guidance in kits/clinical-note-red-flag-scanner/agent.md at lines 92-93 to require independently verified provider authorization and contractual/security controls for PHI processing; do not present zero retention as sufficient. Apply the same prerequisite wording to the setup instructions in kits/clinical-note-red-flag-scanner/README.md at line 61.kits/clinical-note-red-flag-scanner/README.md (1)
31-33: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winMission brief: document the actual input schema.
The README shows
advance_schemaas{ "clinicalNote": "string" }, but the flow also enforcesminLength: 10,maxLength: 45000, and a non-whitespace pattern. Show the real constraints or clearly label the simplified shape as illustrative.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/README.md` around lines 31 - 33, Update the input schema documentation near “Flow Components” to reflect the enforced clinicalNote constraints: minimum 10 characters, maximum 45,000 characters, and at least one non-whitespace character. Alternatively, clearly label the existing shape as illustrative rather than complete.kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md (2)
37-37: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winMission brief: remove the exact-text evidence instruction.
“Cite the exact text” conflicts with the later redaction requirement and can cause verbatim PHI or sensitive clinical text to be returned. Require only redacted/paraphrased evidence, never exact patient text or identifiers.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md` at line 37, Update the evidence guidance in the clinical note red-flag scanner prompt to remove the requirement to cite exact text. Require only redacted or paraphrased evidence in the detail and location fields, explicitly prohibiting verbatim patient text and identifiers such as names, MRNs, and DOBs.
13-13: 🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy liftShared root cause: jurisdiction context is unavailable to the flow.
A disclaimer cannot make jurisdiction-specific regulatory screening reliable when the API accepts only
clinicalNote.
kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md#L13: add jurisdiction/care-setting inputs or constrain findings to note-explicit, non-jurisdictional observations.kits/clinical-note-red-flag-scanner/agent.md#L105: remove the claim that callers can supply context unless the flow accepts it.kits/clinical-note-red-flag-scanner/README.md#L86: document the same enforceable limitation.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md` at line 13, Constrain the REGULATORY guidance in kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md at line 13 to note-explicit, non-jurisdictional observations, or add the required jurisdiction and care-setting inputs to the flow. Update kits/clinical-note-red-flag-scanner/agent.md at line 105 to remove the unsupported claim that callers can supply context, and update kits/clinical-note-red-flag-scanner/README.md at line 86 to document the same enforceable limitation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@kits/clinical-note-red-flag-scanner/README.md`:
- Line 79: Update the example JSON in the README so the summary’s risk claims
match the listed flag categories: either remove “regulatory” from the summary or
add a corresponding REGULATORY flag, while preserving the existing category
invariant and JSON structure.
---
Outside diff comments:
In `@kits/clinical-note-red-flag-scanner/agent.md`:
- Around line 92-93: Update the Data Privacy guidance in
kits/clinical-note-red-flag-scanner/agent.md at lines 92-93 to require
independently verified provider authorization and contractual/security controls
for PHI processing; do not present zero retention as sufficient. Apply the same
prerequisite wording to the setup instructions in
kits/clinical-note-red-flag-scanner/README.md at line 61.
In
`@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md`:
- Line 37: Update the evidence guidance in the clinical note red-flag scanner
prompt to remove the requirement to cite exact text. Require only redacted or
paraphrased evidence in the detail and location fields, explicitly prohibiting
verbatim patient text and identifiers such as names, MRNs, and DOBs.
- Line 13: Constrain the REGULATORY guidance in
kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md
at line 13 to note-explicit, non-jurisdictional observations, or add the
required jurisdiction and care-setting inputs to the flow. Update
kits/clinical-note-red-flag-scanner/agent.md at line 105 to remove the
unsupported claim that callers can supply context, and update
kits/clinical-note-red-flag-scanner/README.md at line 86 to document the same
enforceable limitation.
In `@kits/clinical-note-red-flag-scanner/README.md`:
- Around line 31-33: Update the input schema documentation near “Flow
Components” to reflect the enforced clinicalNote constraints: minimum 10
characters, maximum 45,000 characters, and at least one non-whitespace
character. Alternatively, clearly label the existing shape as illustrative
rather than complete.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 5171a9b6-34bd-4598-8aa4-d5ee761ed7ed
📒 Files selected for processing (3)
kits/clinical-note-red-flag-scanner/README.mdkits/clinical-note-red-flag-scanner/agent.mdkits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md
|
|
||
| ```json | ||
| { | ||
| "result": "{\"summary\":\"The clinical note lacks critical documentation regarding patient identification, comprehensive allergy assessment, and explicit medication instructions, posing significant regulatory and safety risks.\",\"flagCount\":5,\"flags\":[{\"id\":\"FLAG-001\",\"category\":\"IDENTITY\",\"severity\":\"CRITICAL\",\"title\":\"Missing Patient Identifiers and Provider Attestation\",\"detail\":\"The note lacks an MRN, DOB, and provider signature/attestation, making the record legally incomplete and difficult to attribute to a specific patient.\",\"location\":\"Header/Footer\",\"recommendation\":\"Ensure all clinical notes include full patient identifiers and a clear, timestamped provider signature.\"},{\"id\":\"FLAG-002\",\"category\":\"ALLERGY\",\"severity\":\"CRITICAL\",\"title\":\"Missing Allergy Information\",\"detail\":\"There is no documentation regarding the patient's allergy status, which is a requirement prior to prescribing new medications like nitroglycerin.\",\"location\":\"ABSENT\",\"recommendation\":\"Document patient allergy status or explicitly state 'NKDA' (No Known Drug Allergies).\"},{\"id\":\"FLAG-003\",\"category\":\"DOSING\",\"severity\":\"HIGH\",\"title\":\"Incomplete Medication Dosing Instructions\",\"detail\":\"The nitroglycerin prescription is missing frequency, route, and duration/maximum dose instructions.\",\"location\":\"Plan\",\"recommendation\":\"Ensure all medication orders contain complete dosing instructions including route and frequency.\"},{\"id\":\"FLAG-004\",\"category\":\"FOLLOW_UP\",\"severity\":\"HIGH\",\"title\":\"Incomplete Follow-up and Emergency Plan\",\"detail\":\"The note lacks clear instructions on follow-up timeline and return precautions.\",\"location\":\"Plan\",\"recommendation\":\"Document specific follow-up instructions and return precautions.\"},{\"id\":\"FLAG-005\",\"category\":\"HISTORY\",\"severity\":\"MEDIUM\",\"title\":\"Incomplete Social and Family History\",\"detail\":\"The history is missing smoking status, alcohol use, and relevant family history of premature coronary artery disease, which are standard for a 67-year-old presenting with chest pain.\",\"location\":\"History\",\"recommendation\":\"Complete a focused social and family history relevant to the differential diagnosis of angina.\"}]}" |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Mission brief: make the example summary satisfy the invariant.
The summary mentions “regulatory risks,” but the flags contain no REGULATORY entry. Remove that claim or add the corresponding flag so the example follows the system prompt’s category invariant. This repeats the prior review finding.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@kits/clinical-note-red-flag-scanner/README.md` at line 79, Update the example
JSON in the README so the summary’s risk claims match the listed flag
categories: either remove “regulatory” from the summary or add a corresponding
REGULATORY flag, while preserving the existing category invariant and JSON
structure.
Contributor
|
/validate |
Contributor
|
📡 Running Studio validation — results will appear here shortly. |
Contributor
|
@Vilsee LGTM! There are two coderabbit comments left, please solve them too. |
- Remove 'regulatory' mismatch from README example summary - Require verified provider authorization for PHI handling instead of assuming zero-retention is sufficient - Document full advance_schema constraints (minLength, maxLength, pattern) in agent.md and README - Prohibit verbatim citations and require paraphrased evidence in system prompt - Constrain REGULATORY findings to note-explicit non-jurisdictional observations and document schema limits
Author
|
@akshatvirmani Thanks for the LGTM! Pushed a fix for the remaining CodeRabbit items just now (commit 28678d3) — ready for another look whenever you get a chance. |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@kits/clinical-note-red-flag-scanner/agent.md`:
- Line 92: The limitations guidance in
kits/clinical-note-red-flag-scanner/agent.md lines 92-92 must require
independently verified provider PHI authorization together with contractual and
security controls, without presenting zero retention as an alternative. Apply
the same authorization requirement to the user-facing setup guidance in
kits/clinical-note-red-flag-scanner/README.md lines 61-61.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 1adc3778-4291-4df9-8d2d-016dcf9a7877
📒 Files selected for processing (3)
kits/clinical-note-red-flag-scanner/README.mdkits/clinical-note-red-flag-scanner/agent.mdkits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md
- Rewrite Data Privacy guidance in agent.md and README.md to plainly require authorization and disclaim compliance verification - Remove all references to zero-retention as an alternative pathway
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a Clinical Note Red-Flag Scanner template — a single-flow API pipeline that accepts a clinical note as text and returns a structured, severity-ranked JSON list of documentation red flags for clinician and compliance reviewer triage.
Problem Statement
Clinical documentation frequently ships with silent gaps — missing informed consent language, undocumented drug-interaction risk, incomplete vitals/history, or ambiguous dosing instructions — that create real patient-safety and regulatory exposure (aligned with EU AI Act Article 12 documentation/audit requirements, HIPAA documentation-integrity expectations, and CMS/Joint Commission accreditation standards).
The existing AgentKit registry has a
medical-assistantsymptom-checker chatbot, but nothing that performs structured compliance analysis of clinical documentation. This template fills that gap — it's not a conversational agent, it's a single-pass analysis tool that outputs machine-readable results for dashboards, alerting systems, and CDI (Clinical Documentation Improvement) workflows.What This Template Does
CONSENT,DRUG_INTERACTION,ALLERGY,VITALS,HISTORY,DOSING,FOLLOW_UP,ASSESSMENT,IDENTITY,REGULATORYCRITICAL/HIGH/MEDIUM/LOW) with per-flag reasoning, location references, and remediation recommendationsgemini-3.1-flash-lite-preview) for analysisFlow Architecture
4 nodes, single flow, template type — no UI, no env vars, no external dependencies beyond a Gemini API key.
Tested Example
Input — a 67M chest pain presentation with intentional documentation gaps:
{ "clinicalNote": "Patient: John D., 67M. Chief Complaint: Chest pain x 2 hours. History: HTN, DM2. Current Meds: Metformin 500mg BID, Lisinopril 10mg daily. Exam: BP 158/94, HR 88, RR 18. Lungs clear. Heart: regular rhythm, no murmurs. Assessment: Probable angina. Plan: Start aspirin 81mg daily, order stress test. Prescribed nitroglycerin PRN." }Output — 5 flags across 5 categories, severity-ranked:
Each flag includes
detail,location, andrecommendationfields. Full JSON output is in the README.Files Added
How It Was Built
.tscontains authentic runtime IDs and edge metadatakits/following thearticle-summarisertemplate structure exactlylamatic.config.ts,README.md,agent.md, and flow meta with comprehensive documentationChecklist
kits/article-summariser/exactlylamatic.config.ts— typetemplate, author, tags, deploy/github links.tsexported from Lamatic Studio (authentic runtime metadata)agent.md— overview, purpose, flow details (4 nodes), I/O schemas, guardrailsREADME.md— deploy button, category table, real tested example with full JSON outputconstitutions/default.md— safety guardrails and PII handlingregistry.jsonentries (verified all 79)feat: Add ...conventionclinical-note-red-flag-scanneris unique in the registryAdded
kits/clinical-note-red-flag-scanner/kit:.gitignore: ignores.lamatic/,node_modules/,.env,.env.localREADME.md: template overview and deployment steps; documents 10 red-flag categories, severity-ranked JSON output schema (severity, detail, location/ABSENT, remediation), example request/response, and limitations around JSON-string payloads and non-enforcement of de-identificationagent.md: AgentKit documentation for a single-pass, API-invoked “clinical note red-flag scanner” (structured documentation gaps only; not medical advice); describes the GraphQL→Gemini analysis→variables mapping→GraphQL response flow and the JSON-stringresultconstitutions/default.md: default safety/guardrails (identity/safety/data handling/tone), including refusal of prompt-injection/jailbreak, no fabrication when uncertain, and privacy expectations (avoid storing/logging PII; treat input as adversarial)lamatic.config.ts: Lamatic template configuration (metadata + one required step wired toCLINICAL_NOTE_RED_FLAG_SCANNER_FLOW_ID)flows/clinical-note-red-flag-scanner.ts: flow definition exported as{ meta, inputs, references, nodes, edges }model-configs/clinical-note-red-flag-scanner_llmnode-453_generative-model-name.ts: Gemini model binding for the LLM node usinggemini/gemini-3.1-flash-lite-previewprompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md: system prompt enforcing “raw JSON only” with fixed schema (summary,flagCount,flags[]), per-flag required fields, enum constraints, sequential uniqueFLAG-001-style IDs, severity-based sorting, invariant checks, and documentation-only remediationprompts/clinical-note-red-flag-scanner_llmnode-453_user_1.md: user prompt contract requiring de-identified notes, injects{{triggerNode_1.output.clinicalNote}}into a<clinical_note>block, instructs the model to ignore instructions inside the embedded note, and reiterates no PHI detection/redactionFlow (
kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts)—node types + how it works:triggerNode_1(triggerNode/graphqlNode): GraphQL/API entrypoint validatingclinicalNote(string,minLength: 10,maxLength: 45000, non-whitespace)LLMNode_453(LLMNode): runs Gemini with referenced system+user prompt assets and the configured model; producesgeneratedResponsevariablesNode_197(variablesNode): mapsLLMNode_453.output.generatedResponse→finalTextresponseNode_triggerNode_1(responseNode/graphqlResponseNode): returns{ result: "{{variablesNode_197.output.finalText}}" }, where the JSON report is delivered as a JSON-serialized stringtriggerNode_1 → LLMNode_453 → variablesNode_197 → responseNode_triggerNode_1(single API request/response, 10-category red-flag JSON output)