Hosted relay on communities.buzz.xyz unreachable — Cloudflare 521, all relay HTTP endpoints time out

[Foreist]

Summary

A hosted community relay under communities.buzz.xyz has been unreachable for ~25 minutes. Cloudflare terminates TLS fine, but the origin relay never answers — the edge returns 521 on the WebSocket upgrade, and every relay HTTP endpoint times out.

Filing here for visibility since I couldn't find a status page. If this is the wrong venue for hosted-relay operational issues, happy to move it.

Relay host is masked below as <community>.communities.buzz.xyz.

Impact

The desktop app stays up, but every agent sits permanently disconnected — no messages in or out. Nothing is actionable from the client side.

Timeline (UTC, 2026-07-28)

Time	Event
~00:06	Existing agent sessions lose the relay, shut down cleanly
00:08:14	Agents restart, initial relay connect begins failing
00:11:00	Cloudflare starts returning explicit 521 on the upgrade
00:16:16	One transient HTTP 200 on /, immediately back to timeouts
00:30:39	Still down — all endpoints time out

Evidence

1. Relay HTTP endpoints (from crates/buzz-relay/src/router.rs)

All time out with no response after a successful TLS handshake:

$ for p in health _liveness _readiness info; do
    curl -s -o /dev/null -w "/$p %{http_code} %{time_total}s\n" \
      "https://<community>.communities.buzz.xyz/$p" --max-time 10
  done
/health      000 10.003s
/_liveness   000 10.003s
/_readiness  000 10.004s
/info        000 10.002s

A WebSocket upgrade handshake (Connection: Upgrade / Upgrade: websocket / Sec-WebSocket-Version: 13) also returns 000.

2. Cloudflare edge is healthy; the origin is not

$ nc -zvw 8 104.18.16.37 443
Connection to 104.18.16.37 port 443 [tcp/https] succeeded!

curl -v shows TCP connect + full TLS handshake completing, then no HTTP response until timeout. DNS resolves normally to communities.buzz.xyz.cdn.cloudflare.net.

Baseline from the same machine at the same moment:

Host	Result
<community>.communities.buzz.xyz	000 (timeout)
communities.buzz.xyz	000 (timeout)
buzz.xyz	000 (timeout)
block.xyz	200
cloudflare.com	200
github.com	200

So this is not local network or DNS — and it does not look scoped to a single community, since the apex and the communities host are down too.

3. buzz-acp client logs

00:08:44  WARN buzz_acp::relay: initial relay connect attempt 0 failed: Connection closed
00:09:15  WARN buzz_acp::relay: initial relay connect attempt 1 failed: Connection closed
00:11:00  WARN buzz_acp::relay: autonomous reconnect attempt 3 failed:
          WebSocket error: HTTP error: 521 <unknown status code>
00:11:04  WARN buzz_acp::relay: autonomous reconnect attempt 4 failed:
          WebSocket error: HTTP error: 521 <unknown status code>

Cloudflare 521 = "Web Server Is Down", i.e. the edge cannot reach the origin relay.

Client behaviour is fine — no client bug here

Worth noting so this isn't misread as a client defect: buzz-acp's reconnect logic behaves correctly throughout. After the inner 5-attempt burst it falls through to an outer reconnect loop with bounded, jittered backoff and keeps retrying indefinitely:

00:18:09  WARN buzz_acp::relay: relay reconnect failed: Connection closed
00:18:09  WARN buzz_acp::relay: retrying reconnect in 27.2s
00:19:06  WARN buzz_acp::relay: relay reconnect failed: Connection closed
00:19:06  WARN buzz_acp::relay: retrying reconnect in 52.3s
00:20:29  WARN buzz_acp::relay: relay reconnect failed: Connection closed
00:20:29  WARN buzz_acp::relay: retrying reconnect in 48.4s

Agents should reattach on their own once the origin is back. This issue is purely about the relay being down.

Environment

• Desktop app 0.4.26, macOS 15 (arm64)
• Relay: hosted communities.buzz.xyz deployment (not self-hosted)

Ask

1. Is there a known outage / status page for hosted communities.buzz.xyz relays?
2. If a hosted relay is down, is there any in-app signal beyond agents silently showing as disconnected? Right now the only way to tell the difference between "relay down" and "my agents are broken" is reading buzz-acp logs.

[Foreist]

Recovered at 2026-07-28 00:36:00Z — total outage ~28 min

The origin came back on its own. /health returns 200 ok again.

Leaving this open for the two questions at the bottom of the issue, which the recovery doesn't answer.

Client recovered with no intervention

All agent processes reattached on the first attempt, same PIDs — no restart, no manual step:

00:36:00  WARN buzz_acp::relay: relay connection lost — reconnecting…
00:36:05  INFO buzz_acp::relay: autonomous reconnect succeeded (attempt 1)
00:36:05  INFO buzz_acp::relay: resubscribing to 1 channel(s) after reconnect

This confirms the "client behaviour is fine" section above — the reconnect loop did exactly what it should across a 28-minute origin outage.

Correction to my own report: /_liveness and /_readiness return 403 by design

Worth recording since it affects how anyone else probes a hosted relay. Those two endpoints are RBAC-gated:

$ curl https://<community>.communities.buzz.xyz/_liveness
RBAC: access denied

A 403 there means the relay is up — it's a response from the origin. Only /health is usable as an unauthenticated liveness probe from outside:

Endpoint	While down	After recovery
/health	000 (timeout)	200 ok
/_liveness	000 (timeout)	403 RBAC: access denied
/_readiness	000 (timeout)	403 RBAC: access denied
/info	000 (timeout)	403

I had briefly treated /_readiness != 200 as "still down", which was wrong. The distinction that actually matters is timeout vs. any HTTP response.

Minor: reconnect thundering herd

When several local agents reattach simultaneously, the relay rate-limits the burst:

WARN buzz_acp::relay: relay NOTICE: rate-limited: too many concurrent requests
WARN buzz_acp::relay: rate-limit gate armed via NOTICE until ~4.3s from now

It self-clears in ~4s and the client handles it correctly, so this is not a problem at my scale. Flagging only in case the gate is tuned for a smaller number of concurrent agents per owner than deployments actually run.

The open questions

Both still stand, and the second one is the reason I'd like to keep this open:

1. Is there a status page or notification channel for hosted communities.buzz.xyz relays?
2. During the outage there was no in-app signal distinguishing "the relay is down" from "my agents are broken" — agents just showed as disconnected. Diagnosing it required reading buzz-acp logs on disk. Given the client already knows the difference (it has the 521 and the reconnect-loop state), surfacing it in the UI seems cheap and would have saved the whole investigation.