-
Notifications
You must be signed in to change notification settings - Fork 28
docs: add RPC endpoint best practices #873
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,3 @@ | ||
| label: Payments | ||
| collapsed: true | ||
| order: 5 | ||
| order: 6 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,124 @@ | ||
| --- | ||
| title: RPC Endpoints | ||
| description: Best practices for using external RPC endpoints and avoiding rate limits. | ||
| sidebar: | ||
| order: 3 | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think you should put this in position 5, after the packages themselves |
||
| --- | ||
|
|
||
| Synapse SDK reads on-chain state while it prepares payments, finds or creates data sets, resolves providers, and inspects piece metadata. For accounts with many data sets or pieces, those reads can exceed the rate limits of public RPC endpoints. | ||
|
|
||
| For production apps, bulk uploads, migrations, and dashboards, use a dedicated RPC endpoint with higher limits and enable batching in the viem client that you pass to Synapse. | ||
|
|
||
| ## Recommended setup | ||
|
|
||
| Create a viem client yourself, configure its transport, and pass that client to `Synapse`. This gives you access to viem's HTTP batching, request headers, and multicall batching options. | ||
|
|
||
| ```ts twoslash | ||
| import { Synapse, calibration } from '@filoz/synapse-sdk' | ||
| import { createClient, http, type Hex } from 'viem' | ||
| import { privateKeyToAccount } from 'viem/accounts' | ||
|
|
||
| // GLIF accepts authenticated RPC tokens in the URL query string. | ||
| const rpcUrl = 'https://api.calibration.node.glif.io/rpc/v1?token=YOUR_TOKEN' | ||
| const privateKey = '0x0000000000000000000000000000000000000000000000000000000000000001' | ||
|
|
||
| const client = createClient({ | ||
| account: privateKeyToAccount(privateKey as Hex), | ||
| chain: calibration, | ||
| transport: http(rpcUrl, { | ||
| batch: true, | ||
| }), | ||
| batch: { | ||
| multicall: true, | ||
| }, | ||
| }) | ||
|
|
||
| const synapse = new Synapse({ | ||
| client, | ||
| source: 'my-app', | ||
| }) | ||
| ``` | ||
|
|
||
| This setup enables two different batching layers: | ||
|
|
||
| - `transport: http(url, { batch: true })` batches multiple JSON-RPC requests into one HTTP request when the RPC provider supports JSON-RPC batch requests. | ||
| - `batch: { multicall: true }` lets viem aggregate compatible `eth_call` contract reads through Multicall3. | ||
|
|
||
| These settings reduce request count, but they do not remove the need for an endpoint with enough capacity for high-volume accounts. | ||
|
|
||
| ## Authenticated endpoints | ||
|
|
||
| Most RPC providers offer authenticated endpoints with higher limits than public URLs. Prefer a token-authenticated endpoint for production and migration workloads. The Filecoin Docs maintain a list of commonly used [Filecoin RPC endpoints](https://docs.filecoin.io/networks-and-tools/networks/mainnet/rpcs). | ||
|
|
||
| GLIF accepts RPC tokens in the URL query string. Pass the authenticated URL directly to `http()`: | ||
|
|
||
| ```ts twoslash | ||
| import { Synapse, calibration } from '@filoz/synapse-sdk' | ||
| import { createClient, http, type Hex } from 'viem' | ||
| import { privateKeyToAccount } from 'viem/accounts' | ||
|
|
||
| const privateKey = '0x0000000000000000000000000000000000000000000000000000000000000001' | ||
|
|
||
| const client = createClient({ | ||
| account: privateKeyToAccount(privateKey as Hex), | ||
| chain: calibration, | ||
| transport: http('https://api.calibration.node.glif.io/rpc/v1?token=YOUR_TOKEN', { | ||
| batch: true, | ||
| }), | ||
| batch: { multicall: true }, | ||
| }) | ||
|
|
||
| const synapse = new Synapse({ client, source: 'my-app' }) | ||
| ``` | ||
|
|
||
| GLIF also accepts the same token in an `Authorization: Bearer` header. Configure `fetchOptions.headers` on the viem HTTP transport: | ||
|
|
||
| ```ts twoslash | ||
| import { Synapse, calibration } from '@filoz/synapse-sdk' | ||
| import { createClient, http, type Hex } from 'viem' | ||
| import { privateKeyToAccount } from 'viem/accounts' | ||
|
|
||
| const privateKey = '0x0000000000000000000000000000000000000000000000000000000000000001' | ||
|
|
||
| const client = createClient({ | ||
| account: privateKeyToAccount(privateKey as Hex), | ||
| chain: calibration, | ||
| transport: http('https://api.calibration.node.glif.io/rpc/v1', { | ||
| batch: true, | ||
| fetchOptions: { | ||
| headers: { | ||
| Authorization: 'Bearer YOUR_TOKEN', | ||
| }, | ||
| }, | ||
| }), | ||
| batch: { multicall: true }, | ||
| }) | ||
|
|
||
| const synapse = new Synapse({ client, source: 'my-app' }) | ||
| ``` | ||
|
|
||
| For other RPC providers, check the provider's documentation for the exact authentication format and whether JSON-RPC batch requests are supported. | ||
|
|
||
| ## When public endpoints are not enough | ||
|
|
||
| Public endpoints are useful for examples and small tests, but ordinary production usage can outgrow them quickly: | ||
|
|
||
| | Workflow | Why it can create many RPC reads | | ||
| | --------- | -------------------------------- | | ||
| | Listing data sets | The SDK reads data set IDs and enriches them with on-chain metadata. | | ||
| | Inspecting a large data set | Piece metadata reads scale with the number of pieces in the data set. | | ||
| | Uploading to an account with existing data sets | The SDK resolves reusable data sets before creating new ones. | | ||
| | Dashboards and migrations | Repeated listing, filtering, and inspection can multiply reads across many data sets. | | ||
|
|
||
| For these workflows, use an authenticated endpoint, avoid tight polling loops, and cache known data set IDs or piece information when your application can tolerate cached state. | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. do we dare suggest that for heavy, large-scale workloads, that they considering installing Forest, or Lotus nodes to have their own trusted, local, un-capped endpoint?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I considered it yesterday, but it felt a bit heavy on them, especially for developers who are just getting to know them. However, adding this part here is not a problem; it just gives them another option, which I will add later. |
||
|
|
||
| ## Storage workflow tips | ||
|
|
||
| Use the SDK's data set reuse behavior deliberately: | ||
|
|
||
| - Set a stable `source` value for your application so the SDK can isolate and reuse your app's data sets. | ||
| - Use consistent data set metadata for content that should share a data set. | ||
| - If you already know the target provider or data set, pass explicit options such as `providerId` or `dataSetId` to avoid broad discovery. | ||
| - For bulk uploads, keep one configured `Synapse` instance and reuse it instead of repeatedly creating new clients in a loop. | ||
|
|
||
| These choices do not replace a properly provisioned RPC endpoint, but they reduce unnecessary discovery reads. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,3 @@ | ||
| label: Storage | ||
| collapsed: true | ||
| order: 7 | ||
| order: 8 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's not a published "package" like the others so doesn't belong here