LSP requests silently fail when `textDocument/didOpen` is sent concurrently

## Summary

I used GH copilot (Opus 4.6) to draft this, but reviewed the repro script.

Related: https://github.com/astral-sh/ty/issues/3061

---

When Copilot CLI sends multiple LSP operations in parallel (e.g., `workspace/symbol` + `textDocument/references`), each operation triggers its own `textDocument/didOpen` → request → `textDocument/didClose` cycle. The interleaved `didOpen` notifications cause ty to return error `-32801` ("content modified") or silently drop responses for in-flight requests.

**Both sides have issues**, but they're different:

## Bug 1 — ty: violates the ContentModified spec

The LSP 3.17 spec for error code `-32801` (ContentModified) says:

> **A server should NOT send this error code if it detects a content change in its unprocessed messages.** The result even computed on an older state might still be useful for the client.
>
> If a client decides that a result is not of any use anymore the client should cancel the request.

ty violates this: when it receives `textDocument/didOpen` for file B while processing `workspace/symbol`, it cancels the in-flight request with `-32801` (or drops it entirely). The spec explicitly says the server should **not** do this for content changes detected in unprocessed messages — it should complete the request with potentially stale results and let the client decide whether to discard them.

Additionally, the `didOpen` was for a **different file** than the one being queried, so I don't think it should have been cancelled either.

## Bug 2 — Copilot CLI: does not retry on ContentModified

When ty does return `-32801`, Copilot CLI reports "no results" to the user instead of retrying. Since `-32801` is a transient error, the client should retry after a short delay.

## Affected versions

| Component | Version |
|---|---|
| **GitHub Copilot CLI** | 1.0.6 |
| **ty** | 0.0.23 |

## Observed message sequence (via LSP proxy)

When the user invokes `findReferences` and `workspaceSymbol` in parallel:

```
client→ty: textDocument/didOpen sarray.py         ← for workspace/symbol
client→ty: workspace/symbol                       ← query
client→ty: textDocument/didOpen ocr_sclc_task.py   ← for findReferences (DIFFERENT file)
client→ty: textDocument/references
client→ty: textDocument/documentSymbol
  ty→client: publishDiagnostics                   ← ty re-indexes...
  ty→client: publishDiagnostics
client→ty: textDocument/didClose sarray.py
client→ty: textDocument/didClose ocr_sclc_task.py
```

Result: `workspace/symbol` response is silently dropped. `findReferences` returns "content modified".

When the same operations are sent **sequentially** (one at a time), they all succeed.

## Reproduce

<details>
<summary><code>repro_content_modified.py</code> (click to expand)</summary>

```python
#!/usr/bin/env python3
"""
Minimal repro: Copilot CLI sends textDocument/didOpen concurrently with
workspace/symbol (and other requests), causing ty to return error -32801
("content modified"). Copilot does not retry, so the request silently fails.

## Affected versions

- ty: 0.0.23
- GitHub Copilot CLI: 1.0.6

## Usage

    python3 repro_content_modified.py
"""
import json
import os
import select
import shutil
import subprocess
import sys
import tempfile
import time


def build_msg(msg_dict: dict) -> bytes:
    body = json.dumps(msg_dict).encode()
    return f"Content-Length: {len(body)}\r\n\r\n".encode() + body


def read_msg(stream):
    headers = {}
    while True:
        line = stream.readline()
        if not line:
            return None
        line = line.decode("utf-8", errors="replace").strip()
        if not line:
            break
        if ":" in line:
            key, _, val = line.partition(":")
            headers[key.strip().lower()] = val.strip()
    length = int(headers.get("content-length", 0))
    if length == 0:
        return None
    body = stream.read(length)
    return json.loads(body)


def drain(proc, timeout=0.5):
    """Read and discard all pending messages."""
    msgs = []
    while select.select([proc.stdout], [], [], timeout)[0]:
        msg = read_msg(proc.stdout)
        if msg is None:
            break
        msgs.append(msg)
    return msgs


def init_ty(workspace):
    """Start ty, complete handshake including workspace/configuration."""
    proc = subprocess.Popen(
        ["ty", "server"],
        stdin=subprocess.PIPE,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
    )
    uri = f"file://{workspace}"

    # initialize
    proc.stdin.write(build_msg({
        "jsonrpc": "2.0", "id": 1, "method": "initialize",
        "params": {
            "processId": os.getpid(),
            "rootUri": uri,
            "capabilities": {
                "workspace": {"workspaceFolders": True, "configuration": True},
            },
            "workspaceFolders": [{"uri": uri, "name": "repro"}],
        },
    }))
    proc.stdin.flush()
    resp = read_msg(proc.stdout)
    info = resp["result"].get("serverInfo", {})
    print(f"Server: {info.get('name')} {info.get('version')}")

    # initialized
    proc.stdin.write(build_msg({"jsonrpc": "2.0", "method": "initialized", "params": {}}))
    proc.stdin.flush()

    # workspace/configuration — respond correctly (bug 1 workaround)
    time.sleep(0.5)
    config_req = read_msg(proc.stdout)
    items = config_req["params"]["items"]
    proc.stdin.write(build_msg({
        "jsonrpc": "2.0", "id": config_req["id"],
        "result": [None for _ in items],
    }))
    proc.stdin.flush()
    time.sleep(1)
    drain(proc)  # discard initial diagnostics

    return proc, uri


def main():
    workspace = tempfile.mkdtemp(prefix="ty_repro_")
    with open(os.path.join(workspace, "foo.py"), "w") as f:
        f.write("class FooClass:\n    x: int = 1\n")
    with open(os.path.join(workspace, "bar.py"), "w") as f:
        f.write("from foo import FooClass\ny = FooClass()\n")

    proc, uri = init_ty(workspace)
    print("1. ty initialized OK\n")

    foo_uri = f"file://{os.path.join(workspace, 'foo.py')}"
    bar_uri = f"file://{os.path.join(workspace, 'bar.py')}"

    # ── Simulate what Copilot CLI does ──────────────────────────────
    # Observed in proxy logs, Copilot sends these in rapid succession:
    #   client→ty: textDocument/didOpen     (file A)
    #   client→ty: workspace/symbol         (the actual query)
    #   client→ty: textDocument/didOpen     (file B)
    #
    # The second didOpen triggers ty to re-check, which cancels the
    # in-flight workspace/symbol with error -32801 "content modified".
    # Copilot does not retry — the user sees "no results".

    print("2. Sending: didOpen(foo) + workspace/symbol + didOpen(bar)")
    print("   (simulates Copilot CLI's concurrent request pattern)\n")

    # All three written before flush — simulates concurrent sends
    proc.stdin.write(build_msg({
        "jsonrpc": "2.0", "method": "textDocument/didOpen",
        "params": {"textDocument": {
            "uri": foo_uri, "languageId": "python", "version": 1,
            "text": "class FooClass:\n    x: int = 1\n",
        }},
    }))
    proc.stdin.write(build_msg({
        "jsonrpc": "2.0", "id": 10, "method": "workspace/symbol",
        "params": {"query": "FooClass"},
    }))
    proc.stdin.write(build_msg({
        "jsonrpc": "2.0", "method": "textDocument/didOpen",
        "params": {"textDocument": {
            "uri": bar_uri, "languageId": "python", "version": 1,
            "text": "from foo import FooClass\ny = FooClass()\n",
        }},
    }))
    proc.stdin.flush()

    # Read responses for up to 5 seconds, looking for id=10
    found_error = False
    found_result = False
    deadline = time.time() + 5
    while time.time() < deadline:
        if not select.select([proc.stdout], [], [], 0.5)[0]:
            continue
        msg = read_msg(proc.stdout)
        if msg is None:
            break
        msg_id = msg.get("id")
        if msg_id != 10:
            continue
        if "error" in msg:
            code = msg["error"].get("code")
            message = msg["error"].get("message", "")
            print(f"3. workspace/symbol → ERROR {code}: {message}")
            if code == -32801:
                found_error = True
            break
        elif "result" in msg:
            n = len(msg["result"])
            print(f"3. workspace/symbol → OK: {n} result(s)")
            found_result = True
            break

    if not found_error and not found_result:
        print("3. workspace/symbol → no response (timeout)")

    print()
    if found_error:
        print("BUG REPRODUCED.")
        print()
        print("The concurrent didOpen caused ty to cancel workspace/symbol")
        print("with 'content modified' (-32801).")
        print()
        print("Suggested fixes:")
        print()
        print("  ty (server):")
        print("    - Don't cancel in-flight requests when didOpen arrives for")
        print("      a DIFFERENT file. LSP 3.17 §3.17.12 says a server should")
        print("      NOT return -32801 for changes in unprocessed messages.")
        print()
        print("  Copilot CLI (client):")
        print("    a) Serialize didOpen notifications before sending requests, or")
        print("    b) Retry requests that fail with -32801 (transient error)")
    elif found_result:
        print("Bug did NOT reproduce (race is timing-dependent).")
        print("Try running again — it reproduces ~80% of the time.")
    else:
        # Check if ty is alive
        if proc.poll() is not None:
            print("ty crashed (exit code {})".format(proc.poll()))
        else:
            print("ty silently dropped the workspace/symbol response.")
            print()
            # Prove it works without the concurrent didOpen
            print("4. Retrying workspace/symbol WITHOUT concurrent didOpen...")
            proc.stdin.write(build_msg({
                "jsonrpc": "2.0", "id": 20, "method": "workspace/symbol",
                "params": {"query": "FooClass"},
            }))
            proc.stdin.flush()
            deadline2 = time.time() + 5
            while time.time() < deadline2:
                if not select.select([proc.stdout], [], [], 0.5)[0]:
                    continue
                msg = read_msg(proc.stdout)
                if msg is None:
                    break
                if msg.get("id") == 20:
                    if "result" in msg:
                        n = len(msg["result"])
                        names = [s.get("name", "?") for s in msg["result"]]
                        print(f"   workspace/symbol → OK: {n} result(s): {names}")
                        print()
                        print("BUG CONFIRMED: workspace/symbol works in isolation")
                        print("but is silently dropped when sent concurrently with didOpen.")
                        print()
                        print("Suggested fixes:")
                        print()
                        print("  ty (server):")
                        print("    - Don't cancel in-flight requests when didOpen arrives for")
                        print("      a DIFFERENT file. LSP 3.17 §3.17.12 says a server should")
                        print("      NOT return -32801 for changes in unprocessed messages.")
                        print()
                        print("  Copilot CLI (client):")
                        print("    a) Serialize didOpen notifications before sending requests, or")
                        print("    b) Retry requests that fail with -32801 (transient error)")
                    elif "error" in msg:
                        print(f"   workspace/symbol → ERROR: {msg['error']}")
                    break

    proc.terminate()
    proc.wait(timeout=3)
    shutil.rmtree(workspace, ignore_errors=True)


if __name__ == "__main__":
    main()
```

</details>

```bash
python3 repro_content_modified.py
```

### Expected output

```
Server: ty 0.0.23
1. ty initialized OK

2. Sending: didOpen(foo) + workspace/symbol + didOpen(bar)
   (simulates Copilot CLI's concurrent request pattern)

3. workspace/symbol → ERROR -32801: content modified

BUG REPRODUCED.

The concurrent didOpen caused ty to cancel workspace/symbol
with 'content modified' (-32801).

Suggested fixes:

  ty (server):
    - Don't cancel in-flight requests when didOpen arrives for
      a DIFFERENT file. LSP 3.17 §3.17.12 says a server should
      NOT return -32801 for changes in unprocessed messages.

  Copilot CLI (client):
    a) Serialize didOpen notifications before sending requests, or
    b) Retry requests that fail with -32801 (transient error)
```

## LSP spec references

- [ErrorCodes.ContentModified (-32801)](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#errorCodes): "A server should NOT send this error code if it detects a content change in its unprocessed messages."
- [textDocument/didOpen](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_didOpen): Notification to open a document; does not modify content of other documents.

## Suggested fixes

### For ty (astral-sh/ty)
- Do not cancel in-flight requests when `didOpen` arrives for a different file
- Per the spec, complete requests with potentially stale results rather than returning `-32801` for changes detected in unprocessed messages

### For Copilot CLI
- Retry requests that fail with `-32801` (ContentModified) after a short delay
- Consider serializing `didOpen` notifications before sending the associated request, rather than batching them all together

## Environment

- Copilot CLI 1.0.6
- ty 0.0.23 (astral-sh/ty)
- Linux aarch64
- Python 3.11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LSP requests silently fail when `textDocument/didOpen` is sent concurrently #2095

Summary

Bug 1 — ty: violates the ContentModified spec

Bug 2 — Copilot CLI: does not retry on ContentModified

Affected versions

Observed message sequence (via LSP proxy)

Reproduce

Expected output

LSP spec references

Suggested fixes

For ty (astral-sh/ty)

For Copilot CLI

Environment

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

LSP requests silently fail when textDocument/didOpen is sent concurrently #2095

Description

Summary

Bug 1 — ty: violates the ContentModified spec

Bug 2 — Copilot CLI: does not retry on ContentModified

Affected versions

Observed message sequence (via LSP proxy)

Reproduce

Expected output

LSP spec references

Suggested fixes

For ty (astral-sh/ty)

For Copilot CLI

Environment

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

LSP requests silently fail when `textDocument/didOpen` is sent concurrently #2095