feat(mtproto): self-heal wedged and silent transport connections

[pylakey]

Motivation

This work started from a specific, recurring failure on busy connections: a
request whose write hits a saturated socket — the peer's receive window has
closed and stays closed — wedges the entire connection. The socket stays
ESTABLISHED and inbound updates may still arrive, but no request completes, and
nothing recovers it short of destroying and recreating the whole client from
the outside.

Two things make that worse than it should be:

• The three official Telegram clients (Android, TDLib, Telegram Desktop) all
  place transport-failure detection outside the write path and recover
  connections internally, so a stalled or half-dead socket never surfaces
  to the application as a terminal state. gotd does the opposite in a few
  places (below).
• gotd already retries not-yet-processed requests on a new connection once the
  reconnect loop replaces the primary connection (the work that closed
  [Upload Error]: [upload part: send upload part 81 RPC: rpcDoRequest: retryUntilAck: engine forcibly closed: context canceled] #1021 / bug: client can't recover from connection loss #1030). That retry is gated on the reconnect actually firing — and in
  exactly these failure modes it never does, so the existing recovery machinery
  sits idle while requests block forever. (See "Relationship to the existing
  reconnect handling" below.)

Concretely, the current transport has these observable problems:

1. A blocked write wedges the entire connection. transport.connection.Send
   holds writeMux for the whole codec.Write, and clears its own deadline,
   only re-arming one if the caller's context happens to carry a deadline.
   Background loops (pingLoop, ackLoop) call with no deadline. When the
   peer's receive window closes and stays closed (a saturated connection), the
   write blocks forever holding writeMux. Every other sender — including the
   ping loop, the only stall detector — then blocks on that same mutex. The
   errgroup never returns, so the higher-level reconnect loop never runs and
   the connection is never rebuilt. The single tool that could interrupt the
   I/O, Close(), only runs after the group dies — which the wedged write
   prevents. The result is a live-looking connection that passes no requests
   until the process is restarted.

2. Key exchange can hang indefinitely. In exchange/client_flow, steps 5
   and 7 read directly from the connection without applying ExchangeTimeout.
   If the peer completes the TCP handshake but never answers the key exchange
   (a silently dropped inbound path), the read blocks with no bound: the
   connection sits mid-handshake forever, no timeout ever fires, and because
   this is on the connect path the client never reaches a usable state — the
   reconnect loop cannot even start its next attempt, because the current one
   never returns.

3. The only stall detector is the ping round-trip. With interval=60s +
   timeout=15s a silent-but-established socket is noticed in at best 75s —
   and, per (1), never, if the ping loop is itself wedged behind the write
   lock. Until it is noticed, every request on that connection quietly waits
   with no error and no progress. The official clients additionally run a
   detector keyed on time since last receive, independent of the write path
   (Android's checkTimeout, TDLib's last_read_at_/last_pong_at_,
   Desktop's _waitForReceivedTimer).

4. Transport write failures leak to the caller raw. A frame that fails
   mid-write (e.g. EPIPE on a connection the peer already dropped) is not
   classified as retryable, so the caller receives a broken-pipe error for a
   request the server never processed — one that is provably safe to resend on
   a fresh connection, yet is surfaced as a hard failure instead.

5. An acked-but-unanswered request is indistinguishable from user
   cancellation. When the engine is force-closed after the server
   acknowledged a request but before the result arrives, the caller sees only a
   context error — identical to its own cancellation. It therefore cannot tell
   "the server may already have run this, do not resend" from "I cancelled
   this," so it can neither safely retry the request nor safely treat it as
   never-sent.

What this changes

• transport — closes (1), enables (4). Send/Recv acquire the
  read/write I/O locks under the caller's context (a context-cancellable lock
  acquisition) and force a deadline from Close, so a blocked Write can no
  longer trap the ping loop and Close can no longer block behind an in-flight
  I/O. That breaks the deadlock and lets the errgroup return, which is what
  finally allows the reconnect loop to run. Write failures are annotated with
  the new transport.ErrWriteFailed sentinel.

• exchange — closes (2). The existing ExchangeTimeout is now enforced on
  the raw reads in key-exchange steps 5 and 7, so a handshake against a
  non-answering peer fails within the timeout and hands control back to the
  reconnect loop instead of hanging forever.

• mtproto — closes (3), backstops (1). The connection is closed when the
  context is cancelled mid key exchange; a new idle watchdog closes a
  connection whose reads have gone silent past IdleTimeout, keyed on time
  since last receive and therefore working even when the ping loop itself is
  wedged behind the write lock. IdleTimeout and PingDelayDisconnect become
  configurable connection options.

• rpc — closes (5). An engine-closed-after-ack is reported via the new
  rpc.ErrEngineClosedAfterAck sentinel (still errors.Is(context.Canceled),
  so existing checks keep working), letting a caller finally tell that case
  apart from its own cancellation. A request the server already acknowledged is
  never transparently resent — neither on the Do close path nor in the
  retryUntilAck retry path (new rpc.ErrRetrySendFailed).

• telegram — closes (4), opt-in. Transport write failures can be
  classified as retryable on a fresh connection, so a request that never reached
  the server is retried instead of failing. This sits behind
  Options.RetryOnWriteFailed and is off by default: a caller that acts on
  that error itself — rotating a proxy or an endpoint — needs to keep receiving
  it, so the default preserves today's behaviour. The new ping options are
  forwarded from telegram.Options.

Relationship to the existing reconnect handling

gotd already carries requests that the server has not processed across a
reconnect: Client.invokeConn retries a request classified as retryable-on-a-
new-connection once the reconnect loop replaces the primary connection (the
work that closed #1021 / #1030). That retry is gated on the reconnect
actually firing — it blocks on the connection being replaced before retrying.

The failure modes above are precisely the ones where the reconnect never fires.
A wedged write, a silent socket, or a hung key exchange keeps Conn.Run from
ever returning, so the reconnect loop never iterates, the connection is never
replaced, and the waiting retries block indefinitely. This PR makes Conn.Run
return in those cases (forced deadline on Close, idle watchdog, bounded key
exchange), so the existing retry path runs as intended — the connection is
rebuilt and the not-yet-processed requests are resent on it.

It also completes that retry classification without crossing its safety
boundary. A request whose frame failed mid-write (transport.ErrWriteFailed)
is now retryable, since the server discards partial frames. But a reconnect
starts a new session with a fresh msg_id, so an acknowledged request must
never be transparently resent — a fresh-msg_id resend would duplicate a
request the server may already have executed. That request stays non-retryable;
ErrEngineClosedAfterAck only makes the case identifiable instead of
surfacing as a bare cancellation. (The recent stalled-request tracing added in
mtproto targets this same class of hangs; this PR supplies the fixes.)

Public contract / breaking changes

No signatures change; all additions are backward compatible for callers that
do not opt in. The additions and behavioral changes to be aware of:

New exported errors

• transport.ErrWriteFailed — a write to the underlying connection failed.
  Included in the "retryable on a new connection" classification only when
  RetryOnWriteFailed is enabled; by default it still surfaces to the caller.
  Messages the codec rejects locally never carry it, since they would fail the
  same way on any connection.
• rpc.ErrEngineClosedAfterAck — the engine was force-closed after the server
  acknowledged the request but before a result arrived. Deliberately not
  retryable (a transparent resend could duplicate a side-effecting RPC). Still
  satisfies errors.Is(err, context.Canceled), so existing cancellation checks
  are unaffected; the sentinel only adds the ability to distinguish this case.
• rpc.ErrRetrySendFailed — a request went unacknowledged for RetryInterval
  and the retry send then failed. The original send already reached the wire,
  so this is not retryable either.

New connection options (mtproto.Options, forwarded from telegram.Options)

• PingDelayDisconnect time.Duration — the disconnect_delay value sent to the
  server in ping_delay_disconnect. Previously computed inline as
  PingInterval + PingTimeout; now configurable, defaulting to the same
  value, so the wire behavior is unchanged unless set. Validated to exceed
  PingInterval after truncation to whole seconds; an invalid value falls back
  to the default with a warning.
• IdleTimeout time.Duration — maximum time without any received data before
  the idle watchdog closes the connection. Defaults to
  max(PingDelayDisconnect, PingInterval + PingTimeout) and is validated
  against a PingInterval + PingTimeout floor (below the floor a late-but-fine
  pong could trip it). PingInterval/PingTimeout are now also settable at the
  telegram.Options level.

New behaviour option

• telegram.Options.RetryOnWriteFailed (mirrored by
  pool.DCOptions.RetryOnWriteFailed) — retry a request whose transport send
  failed on the next connection instead of returning the error to the caller.
  Off by default, so callers that rotate a proxy or an endpoint on that
  error keep receiving it.

Behavioral changes

• The idle watchdog runs by default (IdleTimeout derived as above). On a
  healthy connection pongs and traffic keep the receive timer fresh, so it only
  fires on a genuinely silent socket; it is a backstop for the case where the
  ping loop itself cannot run.
• Nothing else changes what a caller sees. Retrying a request that failed
  mid-write is opt-in (Options.RetryOnWriteFailed, and
  pool.DCOptions.RetryOnWriteFailed for pools); left off, the write error
  surfaces exactly as it does today. Requests that were sent but not
  acknowledged (ErrConnDead, ErrEngineClosed) are retried unconditionally,
  as they already were.

Testing

• Every touched package (transport, exchange, mtproto, rpc, pool,
  telegram) passes under -race; the full suite passes under
  -coverpkg=./... with no failures.
• Patch coverage 91.6% (263/287 new statement lines); project coverage is
  62.2%, so the patch raises overall coverage. The uncovered remainder is
  defensive branches (a Close() that itself errors, a watchdog tick that
  outruns the first receive) that are not deterministically reachable in a
  unit test.
• New tests exercise each invariant directly (blocking Write simulated at the
  transport layer, not via a live network):
  • a blocked write no longer starves the ping loop / other senders; Close
    forces the in-flight I/O to unblock (transport/connection_test.go);
  • key-exchange reads are bounded by ExchangeTimeout
    (exchange/client_flow_timeout_test.go);
  • the idle watchdog closes a silent connection and never fires on a healthy
    one, with the PingInterval + PingTimeout floor enforced
    (mtproto/watchdog_test.go, mtproto/options_ping_test.go);
  • context cancellation during connect closes the connection rather than
    hanging (mtproto/connect_cancel_test.go);
  • an acked-then-closed request is not resent and is reported as
    ErrEngineClosedAfterAck while still matching context.Canceled; a failed
    retry send is reported as ErrRetrySendFailed and not resent
    (rpc/engine_test.go);
  • ping options are forwarded from telegram.Options
    (telegram/ping_options_test.go).

Invariants the patch protects:

1. A blocked write cannot deadlock the connection or starve the stall detector.
2. Key exchange is always time-bounded.
3. A silent-but-established socket is detected independently of the write path.
4. An acknowledged, side-effecting request is never transparently resent.
5. Defaults preserve current wire behavior; new knobs are opt-in.

Out of scope (deliberately)

• Separating socket and session lifetime (reusable mtproto.Conn, at-least-once
  resend with a stable msg_id, msgs_state_req). The official clients do
  this; it is a larger, core-touching change and is intentionally left out so
  this PR stays a set of self-contained transport fixes.
• Obfuscation-by-default and out-of-band DC config (DoH/fallback) — unrelated.

[codecov]

Codecov Report

❌ Patch coverage is 93.30357% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.83%. Comparing base (a5506de) to head (40a4693).

Files with missing lines	Patch %	Lines
mtproto/conn.go	79.54%	5 Missing and 4 partials ⚠️
mtproto/connect.go	93.10%	1 Missing and 1 partial ⚠️
rpc/engine.go	77.77%	1 Missing and 1 partial ⚠️
transport/connection.go	96.22%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1830      +/-   ##
==========================================
+ Coverage   71.44%   71.83%   +0.38%     
==========================================
  Files         509      509              
  Lines       24137    24289     +152     
==========================================
+ Hits        17245    17448     +203     
+ Misses       5650     5594      -56     
- Partials     1242     1247       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[borzovplus]

Hello!

Won't there be endless reconnection in different cases with unstable proxy connections?

I use gotd to connect thousands of accounts with different proxies, and it is important to me that when the socket falls off due to a timeout or any other network problems, there are no unnecessary reconnection requests inside the library, because in these cases I rotate the proxy and reconnect at my service level.

Have you tested your changes on a large number of accounts with proxies that have a large number of active chats?

I have a highly loaded service based on this library and the cost of error is very high, so I'm worried about such a big diff)

Did you write using AI or by yourself? Did you do a review using AI? I took the liberty of running your diff through the AI and this is what it gave out:

I found a few regression risks in this commit.

1. transport.ErrWriteFailed is treated as retryable, but it does not trigger reconnect

telegram/invoke.go:101 now treats transport.ErrWriteFailed as retryable and then waits for connChanged before retrying on a fresh primary connection.

The problem is that the write failure path itself does not appear to make the current connection die. In mtproto/write.go:66, c.conn.Send(ctx, b) failure is only returned to the caller. It does not close the current mtproto.Conn or otherwise force Conn.Run to return. Production reconnect happens only after replaceConn, but the test in telegram/invoke_test.go:72 simulates that manually.

So a user RPC that gets ErrWriteFailed can block in invokeConn waiting for a reconnect that this error path does not directly initiate. It may only recover later when ping/idle detection fires, or when the caller context is cancelled.

Suggested fix: on a real transport write failure, force-close the current mtproto.Conn so Run exits and the normal reconnect path replaces the connection immediately.

2. Pool retries on ErrWriteFailed can create a new connection while the old one is still running

pool/pool_conn.go:25 now classifies transport.ErrWriteFailed as retryable. Then pool/pool.go:258 calls c.dead(conn, err) and retries by acquiring/creating another connection.

However, pool.Conn has no Close method, and dead() only updates pool bookkeeping. It does not stop the old connection’s Run goroutine. The new test with MaxOpenConnections: 1 effectively creates a second connection while the first fake connection is still alive until DC.Close.

In production, a sequence of write failures could leave old transport goroutines/sockets alive while the pool has already decremented total and created replacements, undermining MaxOpenConnections.

Suggested fix: the underlying mtproto connection should be actively closed on write failure, or pool needs a way to close the failed connection before accounting it as dead.

3. Local codec/pre-write validation errors are now wrapped as retryable transport write failures

transport/connection.go:177 wraps every codec.Write error with ErrWriteFailed.

But codec implementations can return errors before any I/O happens. For example, proto/codec/intermediate.go:55 validates outgoing message size/alignment before writing to the connection. These are local/programming/protocol errors, not transport failures, and retrying on a new connection will not help.

With the new wrapper, such errors can be misclassified by telegram/invoke.go / pool/pool_conn.go as safe to retry on a fresh connection.

Suggested fix: distinguish pre-I/O codec validation errors from actual net.Conn.Write failures. Only actual underlying write failures should match transport.ErrWriteFailed.

4. Sub-second ping settings can still produce disconnect_delay=0

mtproto/options.go:179 validates PingDelayDisconnect after truncation to whole seconds, but the fallback is PingInterval + PingTimeout and is not revalidated.

For example:

PingInterval = 500 * time.Millisecond
PingTimeout = 100 * time.Millisecond

[pylakey]

Fair questions, and two of the four findings were real. Answering the process ones first, since they frame the rest.

Was this written/reviewed with AI? Yes, AI-assisted, with the design decisions and the final call mine. Your skepticism is reasonable and it paid off here: your run found a genuine bug I had missed. For what it is worth, the process did catch itself once — an earlier draft of the rpc change was going to return context.Cause for an acknowledged-but-unanswered request, which would have made an already-executed RPC look retryable and duplicate it; that got rejected during implementation and replaced with the ErrEngineClosedAfterAck sentinel that is in the PR now.

Have you tested this on many accounts with proxies and many active chats? No, and I would rather say so plainly than imply otherwise. What was tested: targeted fault injection against a single live account (packets blackholed both ways, REJECT --reject-with tcp-reset, SYN drop, inbound-only drop, and a case where TCP connects but the MTProto key exchange never gets an answer), plus unit and mutation tests at the transport layer where a blocking Write is simulated directly. Not thousands of accounts, not proxy rotation, not high chat volume. If you can run this branch on a slice of your fleet, that is exactly the signal this PR is missing.

Will this cause endless reconnection, and does it break your proxy detection?

This is the part I took most seriously. Separating what the PR changes from what gotd already did:

Already true on main, untouched here. Client.invokeConn already parked on connChanged and retried indefinitely for pool.ErrConnDead and rpc.ErrEngineClosed, and reconnectUntilClosed already ran with MaxElapsedTime = 0. Note replaceConn runs from the backoff's notify callback on every failed attempt, so connChanged closes each cycle and a parked RPC keeps waking and re-parking. That loop has no retry cap on main and none here — only your ctx or client shutdown ends it. A deadline-free ctx against a permanently dead proxy hangs indefinitely, on main and on this branch alike.

What the PR changed for you, and what I did about it: exactly one error class was newly fed into that loop — a transport write failure on the first send, the EPIPE/RST your own write syscall catches. On main that surfaced to you synchronously (for RST main was already a coin flip: if the read loop noticed first you got ErrEngineClosed, which main already treated as retryable and swallowed). You are right that this is exactly the signal you rotate on, so I have pushed the retry behind Options.RetryOnWriteFailed, off by default, mirrored as pool.DCOptions.RetryOnWriteFailed. As of the latest commit this PR no longer changes what you see for a failed write: the error surfaces exactly as it does on main, and the retry is available to anyone who wants it. It does not restore fast-fail for ErrConnDead / ErrEngineClosed — those were already retried before this PR, and no change here can give back something that was never there.

But the RPC error path was never a dependable proxy signal, including before this PR. Dial failures, blackholes caught by the ping loop, and read-side RSTs all produced ErrConnDead / ErrEngineClosed on main — already retryable, already swallowed. Options.OnDead is the hook that actually reports these, and it is unaffected by this PR: it fires on every connection death because it is driven by the connection lifecycle, not by the RPC error path.

On that hook this PR gives you strictly more than main:

• telegram.Options had no ping knobs at all before this PR — NewClient did not forward any. Now PingInterval / PingTimeout / PingDelayDisconnect / IdleTimeout are settable, and e.g. PingInterval: 10s, PingTimeout: 5s lowers the idle floor to 15s, cutting dead-link detection from ~75s to ~15s. For thousands of proxied accounts that is probably the knob you actually want.
• mtproto.ErrIdleTimeout now reaches OnDead, so "the link went silent" is distinguishable from "dial failed" (context.DeadlineExceeded) and from a hard net.OpError.
• The wedged-write case — a saturated send window with the ping loop trapped behind the write mutex — used to hang with no signal at all and no reconnect. That is the one failure your current detection genuinely could not see.

Concretely, what I would suggest: rotate on OnDead (classify with errors.Is), put a deadline on every RPC ctx (mandatory — it is the only bound on that retry loop, before and after this PR), and tune the ping knobs down if 75s is too slow for you.

One thing I should flag rather than let you discover: the idle watchdog currently cannot be turned off — a zero or negative IdleTimeout is clamped to the derived default, which makes the "disabled" branch unreachable from any public entry point. I can make it genuinely disableable if you need that, though for your case turning it down looks like the better lever.

---

On the four findings — two fixed, two not the mechanism they were reported as. Both fixes are pushed.

3. Codec validation errors misclassified — correct, fixed. This is the real bug, and it is worse than the report suggests. codec.Write mixes local validation with I/O: checkOutgoingMessage and checkAlign reject before the socket is touched, and I was wrapping every codec.Write error in ErrWriteFailed. So an unencodable message (>16 MiB frame, empty, misaligned) was reported as "never reached the server, safe to retry elsewhere" while the connection was perfectly healthy — meaning ping and the idle watchdog never fire, and the caller parks in invokeConn forever, or pool.DC spins marking connections dead and redialling. Fixed by exporting codec.ErrInvalidMessageLength / codec.ErrPayloadNotAligned and skipping the ErrWriteFailed wrap for them; the test asserts the error is not ErrWriteFailed and that writeCount() == 0, i.e. nothing reached the socket.

4. Sub-second ping settings → disconnect_delay=0 — correct, fixed. The validation caught the truncation but its correction re-applied the same PingInterval + PingTimeout sum, so it stayed 0 on the wire. Worth noting this one predates the PR: on main, pingLoop sends int((pingInterval + pingTimeout).Seconds()), which is already 0 for sub-second settings — the PR just made the knobs reachable without fixing it. Now the fallback is derived from the first whole second above PingInterval. Defaults are unchanged: 60s + 15s still yields exactly 75s.

1 and 2 — the symptoms are real, the mechanism is not, and fixing 3 resolves both.

First, the part of your suggestion that is simply right: all three official clients do destroy the connection on a genuine socket write error. Android does it most directly — send() < 0 → closeSocket(1, -1) synchronously inside the epoll callback, discarding every queued-but-unsent byte (ConnectionSocket.cpp:1046-1049, :667-687). TDLib returns a Status that unwinds to the owner, which tears the socket down in the same actor turn (RawConnection.cpp:107-117 → Session.cpp:625-627). tdesktop destroys and restarts on the socket's error signal (session_private.cpp:2624-2634 → :335-342). So the instinct behind your fix matches the official doctrine.

Three things about how they do it changed my mind about porting it as-is:

• The write path is nowhere the primary liveness detector. send() only errors once the kernel has already seen an RST; on a silently blackholed link it keeps succeeding into the socket buffer and the write path never notices. Android therefore relies on checkTimeout, keyed on inbound silence and explicitly never refreshed by outgoing traffic (ConnectionSocket.cpp:1115-1126); TDLib maps EAGAIN to "not an error" (SocketFd.cpp:461-468) and leans on last_read_at_/last_pong_at_; tdesktop gets no signal at all for a silent failure and depends on _waitForReceivedTimer. A detector outside the write path is the mechanism that actually catches the interesting failures — which is what this PR adds, and it is the part I would defend hardest.
• They can afford the teardown because the session outlives the socket. Android splits it in the type system (class Connection : public ConnectionSession, public ConnectionSocket) and resends with the original msg_id and seq_no (ConnectionsManager.cpp:2643-2654); TDLib keeps AuthData as a value member of Session and only lends a pointer to the connection (Session.h:201, Session.cpp:1287); tdesktop keeps haveSentMap in shared SessionData and replays via resendAll() with the same msg_id. In gotd mtproto.Conn is one-shot, so closing it means a new session_id and fresh msg_ids: the same move buys their teardown with none of their recovery. Getting the full behaviour means separating socket and session lifetime, which is the larger change this PR deliberately leaves out of scope.
• A local rejection never tears down a live connection — unanimously. In Android it is structurally impossible: the outgoing direction can only reach closeSocket from inside the EPOLLOUT callback, and even oversized handshake inner data restarts the logical handshake while explicitly preserving the socket (Handshake.cpp:490-496). tdesktop's pre-send checks return false and the sole caller discards the result — no restart, no error (session_private.cpp:2656-2664, :986). TDLib has no such path at all; oversize is split and re-queued. That is exactly finding 3, and it is the one place where all three agree with you against my original code.

Concretely for gotd: a genuine socket write failure already tears the connection down here, just by a different route — the read half breaks at the same instant, readLoop returns, the group dies and replaceConn closes connChanged. The delta an explicit force-close would buy is milliseconds in the RST case, against the cost of a new teardown trigger whose blast radius is a fresh session. I would rather not add that in this PR without a demonstrated case where the write fails and the read half stays alive; if you or @ernado want it anyway, it is a small change now that finding 3 makes ErrWriteFailed mean only a real I/O failure.

Here is each write-failure shape I checked before deciding not to touch pool or mtproto/write.go:

• Genuinely broken socket (RST/EPIPE): the read half breaks at the same instant, so the parked Recv in readLoop fails, the group dies, Conn.Run returns and replaceConn closes connChanged — milliseconds, not ping/idle-bound.
• Write i/o timeout: prepareWrite sets the write deadline to exactly ctx.Deadline() and nothing else, and the caller's ctx is passed down unmodified. So a write timeout implies the caller's ctx has already expired, and invokeConn returns immediately via its own <-ctx.Done() — it cannot park.
• Forced deadline / closed-connection paths (added by this PR): reachable only after connection.Close(), whose only caller is mtproto's Conn.close() from handleClose, the idle watchdog, or connect(). In all of those the connection is already being torn down and the replacement is in flight.
• ctx-cancelled write-lock acquisition: returns a plain wrapped ctx.Err(), not ErrWriteFailed — verified with a probe.

So the only case where a caller could park on a reconnect that never comes was the codec rejection, where the connection is healthy and the retry is futile by construction. That is finding 3, now fixed at the source. With it fixed, the invariant pool.dead() relies on — retryable implies the connection is already dying — holds again, which is why I did not add a force-close to the write path: on a live socket it would tear down a connection that recovers on its own in milliseconds, and on the paths above the teardown is already happening.

If you disagree with that reasoning I would genuinely like to see the counter-case — a write-failure shape where the socket survives and the read half stays alive.

[pylakey]

Closing this in favour of a series of smaller, independently reviewable PRs.

On reflection this PR's blast radius is too large to ask anyone to review as a unit: +2840/-105 across 29 files, six independent changes in six packages, adding five exported option fields and six exported error values. Two of those changes rewrite transport/connection.go and mtproto/conn.go — the load-bearing internals every connection goes through — so accepting it means re-verifying the concurrency properties of the whole transport on an outside contributor's word. There is no partial-accept path either: the six changes have nothing to do with each other, yet they can only be taken or refused together. That is a packaging mistake on my part, not a reflection on the review.

The findings behind it are real, and I am resubmitting them one at a time, smallest and best-evidenced first.

#1838 — fix(exchange): apply ExchangeTimeout to key exchange steps 5 and 7. Two lines, no exported API change, using the package's own tryRead helper. Exchanger.WithTimeout documents itself as bounding "every exchange request" and the server flow honours that for all three of its reads, but the client flow reads steps 5 and 7 through a bare conn.Recv, and transport.connection.Recv takes its deadline solely from ctx.Deadline(). In PFS mode the exchange context has no deadline, so those two reads are unbounded. It comes with a production goroutine dump pinning the exact frames and a regression test that hangs on main and passes with the change. This is the only part of the original PR required to stop a permanent wedge.

Planned next, separately: closing the connection when the context is cancelled during connect(). handleClose is the existing cancellation mechanism but is started at mtproto/conn.go:227, after connect() returns at :221, so the entire key exchange runs with no closer — cancelling during that window does nothing at all.

Later, and only if the above are welcome: an idle watchdog for connections whose peer goes silent in steady state. That one does add exported API, so it deserves its own discussion rather than riding along.

The remaining pieces here I am keeping in our fork rather than proposing. The context-aware transport locks rewrite the least-churned file in the module, and for the failure they address I have the mechanism by inspection but no captured dump — weaker evidence than I would want for a change that size. The rpc acknowledgement semantics encode a product-specific error taxonomy; that is a design preference rather than a bug fix, and it belongs in a discussion rather than a patch.

Thanks for the patience, and apologies for the size of this one.