Skip to content

docs: add RPC endpoint best practices#873

Open
beck-8 wants to merge 1 commit into
FilOzone:masterfrom
beck-8:docs/rpc-endpoints
Open

docs: add RPC endpoint best practices#873
beck-8 wants to merge 1 commit into
FilOzone:masterfrom
beck-8:docs/rpc-endpoints

Conversation

@beck-8

@beck-8 beck-8 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a developer guide for using external RPC endpoints with Synapse SDK and avoiding public endpoint rate limits.

The guide covers:

  • creating a viem client and passing it to Synapse
  • enabling JSON-RPC HTTP batching with http(url, { batch: true })
  • enabling viem multicall batching with batch: { multicall: true }
  • using GLIF authenticated RPC tokens through URL query parameters
  • using GLIF authenticated RPC tokens through Authorization: Bearer headers
  • linking to the Filecoin Docs list of commonly used RPC endpoints
  • practical tips for high-volume storage workflows

Also adds the page to the Developer Guides overview and adjusts sidebar ordering.

Closes #866.

Validation

  • Verified the documented viem client configuration type-checks locally.
  • Verified GLIF Calibration RPC token authentication at runtime:
    • URL query token returned eth_chainId = 0x4cb2f
    • Authorization: Bearer token returned eth_chainId = 0x4cb2f
    • invalid token returned 403
  • Ran markdown lint for the new/updated docs pages:
    • pnpm exec markdownlint-cli2 --config .github/.markdownlint-cli2.jsonc docs/src/content/docs/developer-guides/rpc-endpoints.md docs/src/content/docs/developer-guides/index.md
  • Confirmed local docs page returns 200 OK:
    • http://localhost:4321/developer-guides/rpc-endpoints/

@beck-8 beck-8 requested review from hugomrdias and rvagg as code owners July 8, 2026 09:29
@github-project-automation github-project-automation Bot moved this to 📌 Triage in FOC Jul 8, 2026
@beck-8

beck-8 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author
image

@rjan90 rjan90 moved this from 📌 Triage to 🔎 Awaiting review in FOC Jul 8, 2026
@rvagg

rvagg commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

why are we not seeing the full CI run on this ..

| Package | Best for | Builds on |
| --------- | ---------- | ----------- |
| **[Synapse SDK](/developer-guides/synapse/)** | Most applications | Core |
| **[RPC Endpoints](/developer-guides/rpc-endpoints/)** | Production apps, bulk uploads, dashboards | Viem |

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not a published "package" like the others so doesn't belong here

Suggested change
| **[RPC Endpoints](/developer-guides/rpc-endpoints/)** | Production apps, bulk uploads, dashboards | Viem |

| Uploading to an account with existing data sets | The SDK resolves reusable data sets before creating new ones. |
| Dashboards and migrations | Repeated listing, filtering, and inspection can multiply reads across many data sets. |

For these workflows, use an authenticated endpoint, avoid tight polling loops, and cache known data set IDs or piece information when your application can tolerate cached state.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we dare suggest that for heavy, large-scale workloads, that they considering installing Forest, or Lotus nodes to have their own trusted, local, un-capped endpoint?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered it yesterday, but it felt a bit heavy on them, especially for developers who are just getting to know them. However, adding this part here is not a problem; it just gives them another option, which I will add later.

title: RPC Endpoints
description: Best practices for using external RPC endpoints and avoiding rate limits.
sidebar:
order: 3

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should put this in position 5, after the packages themselves

@rvagg

rvagg commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Good content, but there's a fundamental limitation that I think we need to figure out a path through before we adopt docs like this. The "recommended" path is only suitable for server-side apps and other situations where you have a local private key you can feed in, it's not helpful for wallet connected clients, like MetaMask, which has its own RPC, which you can't just patch to point to your authenticated RPC endpoint (a dapp developer might be able to document all of the ways users of their dapp could configure their wallets to talk to an authenticated endpoint and how to sign up for an endpoint, but that's pretty tedious and not likely to help cause average users just don't grok this stuff).

The batching and multicall advice is still useful. batch: { multicall: true } can reduce compatible contract reads, including for wallet-backed clients if the client is constructed that way. But it does not solve the root problem that triggered this issue: browser wallet flows still use the wallet provider's RPC endpoint, so glif public rate-limit reductions can still hurt users.

Comes back to #504 and I don't think we really fully resolved that to solve for this problem in the sdk at least. In synapse-core at least you can choose what client you pass in, but the sdk takes one client option.

There seem to be two possibilities here we can pursue:

  1. Build or expose a hybrid viem client/transport that routes read-like RPC methods to an app-controlled authenticated RPC endpoint and wallet methods to the wallet provider. This might work, but it is gnarly and I would prefer not to roll our own if viem/wagmi already has a blessed pattern for this.
  2. Add first-class split client support to Synapse, probably as publicClient and walletClient. IIRC @hugomrdias suggested this at some point? Then reads, discovery, metadata, pricing, simulation, and receipt polling use publicClient, while signing and transaction submission use walletClient. This would let browser apps use an authenticated/proxied RPC endpoint for read-heavy FOC work while keeping wallet UX for signatures and transactions.

Until we have that, I think this page should be scoped more narrowly:

  • Present the current createClient + privateKeyToAccount + authenticated http() setup as the server/CLI/migration/service-account path (clearly stated)
  • Keep the batching and multicall advice, with caveats.
  • Add an explicit browser-wallet limitation section.
  • Avoid implying that this solves authenticated RPC for MetaMask-connected Synapse apps today.

I'll defer to @hugomrdias for more input, he's thought more about this than me. I think I'd be happy adding option 2 and I don't think it'd be too hard with what we have now.

@rvagg

rvagg commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

why are we not seeing the full CI run on this ..

PR incoming

@rvagg

rvagg commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Even a split client still doesn't fully solve the fundamental problem because I don't think we can practically advise dapp developers to push their glif RPC token out to their clients. Clients are either going to have to configure their own RPC (which I don't think scales well as advice), dapps may have to proxy RPC and expose that (kind of gross), or we have some kind of graceful rate limit handling that slows down calls when you hit limits (a bit gross and prone to breakage, but we could pull something basic off).

OR we could limit scope of this even further and be clear that caring about this is for the cases of high volume usage, particularly where data sets get very large or you have lots of data sets. The common case should be fine, but "if you encounter 429 rate limits then ..." might be the right framing for this doc instead. Then our job becomes focusing on how we can limit the number of RPC calls to get it under threshold as much as possible.

@beck-8

beck-8 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

Yes, my starting point is that I am worried that glif will be more likely to have this problem after reducing the RPM.

After waiting for more ideas to enter, I will to update this section.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: 🔎 Awaiting review

Development

Successfully merging this pull request may close these issues.

provide best practices for using external RPC endpoints

3 participants