Skip to content

Add self-modifying harness docs and edge-case benchmark#335

Open
MagMueller wants to merge 1 commit into
mainfrom
codex/self-modifying-edge-bench
Open

Add self-modifying harness docs and edge-case benchmark#335
MagMueller wants to merge 1 commit into
mainfrom
codex/self-modifying-edge-bench

Conversation

@MagMueller
Copy link
Copy Markdown
Contributor

@MagMueller MagMueller commented May 10, 2026

Summary

  • Add a self-modifying harness docs page that explains the inspect, patch, retry, verify loop
  • Cover four concrete edge cases: canvas signature, file upload, drag and drop, and coordinate-only targets
  • Add a standalone docs/edge-case-benchmark.html page that exercises those four mechanics and exposes window.edgeBenchmark.results
  • Link the new docs from the README

Validation

  • python3 HTML parser accepted docs/edge-case-benchmark.html
  • Ran the benchmark through Browser Harness remote browser: signature=true, upload=true, drag=true, coordinate=true
  • Saved verification screenshot at /tmp/browser-harness-edge-benchmark-result.png
  • env -u BU_CDP_URL -u BU_CDP_WS uv run --with pytest pytest -q tests/unit -> 77 passed

Pass/fix notes from benchmark run

  • Canvas signature passed with a CDP pointer stroke after reading the canvas geometry
  • File upload passed with upload_file() plus explicit input/change events
  • Drag/drop passed with a task-specific DataTransfer helper
  • Coordinate target passed with geometry-derived click_at_xy()
  • Initial data-URL navigation was flaky in the installed harness, so the run injected the benchmark HTML into about:blank via document.write()

Summary by cubic

Adds a self-modifying harness guide and a standalone browser benchmark to validate canvas signatures, uploads, drag-and-drop, and coordinate-only targets. The benchmark exposes window.edgeBenchmark.results for quick automated checks.

  • New Features
    • New docs/self-modifying-harness.md: explains the inspect → patch → retry → verify loop with four edge-case walkthroughs.
    • Added docs/edge-case-benchmark.html: exercises the four mechanics and exposes window.edgeBenchmark.results for verification.
    • Updated README.md to link the new docs and benchmark.

Written for commit a9a6977. Summary will update on new commits.

Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 3 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="docs/edge-case-benchmark.html">

<violation number="1" location="docs/edge-case-benchmark.html:323">
P2: The coordinate task can be passed via synthetic/programmatic click events, so it does not reliably enforce a real coordinate-driven pointer action.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.

}
});

document.getElementById("coordinate-target").addEventListener("click", () => {
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai Bot May 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The coordinate task can be passed via synthetic/programmatic click events, so it does not reliably enforce a real coordinate-driven pointer action.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At docs/edge-case-benchmark.html, line 323:

<comment>The coordinate task can be passed via synthetic/programmatic click events, so it does not reliably enforce a real coordinate-driven pointer action.</comment>

<file context>
@@ -0,0 +1,328 @@
+      }
+    });
+
+    document.getElementById("coordinate-target").addEventListener("click", () => {
+      setPass("coordinate", "red target clicked");
+    });
</file context>
Fix with Cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant