If you're picking a Solana RPC provider, you'll want to benchmark them. The marketing pages aren't enough; the SLA promises aren't enough; you need actual measurements against your actual workload. But most public RPC benchmarks you'll find online are bad — wrong methodology, wrong metrics, or just outdated.
This piece is a practical methodology for benchmarking Solana RPCs in a way that produces actionable conclusions. What to measure, how to measure it, and the common mistakes that make benchmarks useless.
The Wrong Way to Benchmark
A few common patterns that produce bad results:
Single-shot latency tests. "I called getSlot once on each RPC; the fastest was X." Useless. One measurement is noise.
Off-peak measurements. Benchmarking at 3am when nothing is happening tells you about RPCs in idle conditions. Production cares about peak.
Average-only metrics. "Average latency was 200ms." Averages hide the long tail. Production cares about P95 and P99.
Synthetic workloads. Benchmarking with manufactured traffic that doesn't match your real workload. Tells you nothing about your case.
Comparing different commitment levels. Apples-to-oranges. If you're testing one RPC with processed commitment and another with confirmed, you're not benchmarking the RPCs.
Not accounting for client-side variance. Network from your client to each RPC matters. Comparing RPCs from one client location tells you about that client; you need diverse client locations for robust conclusions.
What to Actually Measure
The metrics that matter for production decisions:
End-to-end transaction latency. From "I called sendTransaction" to "the transaction has landed in a block." Not just the HTTP response time of sendTransaction.
P95 and P99 latency. Average is for marketing. Production cares about worst-case behaviour.
Latency under congestion. Measure during peak network hours, not off-peak. The interesting comparison is what happens when load is high.
Effective fill quality (for trading workloads). Not just whether the transaction landed but at what price. Compares to AMM-math expected.
Failure rate. How often do transactions silently fail? What's the cost per failed transaction?
Cost per landed transaction. Total cost (fees + RPC pricing) divided by successfully landed transactions. Different from cost per request.
Tail behaviour. What's the worst case? P99.9? Beyond?
The Methodology That Works
A workable approach:
1. Define your workload. What kind of transactions? What position sizes? What network conditions? Be specific.
2. Set up parallel submission. Configure the same workload to submit identical transactions to N RPCs simultaneously. Most languages can handle this easily.
3. Run during representative conditions. Cover business hours and off-peak. Cover weekday and weekend. Cover congestion windows specifically.
4. Sample size matters. Single-day measurements are noisy. Run for at least a week to get robust results.
5. Track per-signature outcomes. Log every transaction with full context: which RPC, signature, attempt time, landing time, slot, outcome.
6. Analyse the distribution, not just averages. Histograms, percentile breakdowns, tail analysis. Production lives in the tail.
7. Account for cost. Total cost per landed transaction across the whole sample. Some RPCs are cheap per-request but expensive per-landed.
Specific Things to Test
For Solana specifically:
Send latency. Time from your sendTransaction call to the transaction landing in a block. Measure both the HTTP round-trip (RPC accepted) and the on-chain landing.
Confirmation latency. Time from acceptance to "confirmed" commitment.
Read latency. For workloads with significant reads, measure getAccountInfo and similar. Different from write latency but matters for hybrid workloads.
Behaviour during congestion. Solana has predictable congestion windows (token launches, NFT mints, market volatility). Test during these.
Stake-weighted QoS effectiveness. Hard to measure directly, but you can infer from inclusion behaviour during congestion. RPCs with stronger SWQoS infrastructure should land transactions more reliably under load.
Sandwich exposure. For trading workloads, compare actual fills against AMM-math expected. Systematic gaps = sandwich tax. RPCs with Anti-MEV routing should show smaller gaps.
Implementation Sketch
A basic benchmark harness in TypeScript:
import { Connection, Keypair, Transaction, SystemProgram } from "@solana/web3.js";
const RPCS = [
{ name: "rpc1", url: "https://...", connection: null },
{ name: "rpc2", url: "https://...", connection: null },
];
for (const r of RPCS) {
r.connection = new Connection(r.url, "processed");
}
async function submitToAll(tx, signers) {
const promises = RPCS.map(async (r) => {
const start = Date.now();
try {
const sig = await r.connection.sendTransaction(tx, signers, {
skipPreflight: true,
maxRetries: 0,
});
// Wait for landing
const status = await r.connection.confirmTransaction(sig, "confirmed");
const end = Date.now();
return {
rpc: r.name,
signature: sig,
latency_ms: end - start,
outcome: status.value.err ? "failed" : "success",
err: status.value.err,
};
} catch (e) {
return {
rpc: r.name,
outcome: "error",
err: e.message,
};
}
});
return Promise.all(promises);
}
// Submit periodically and log results
Production version would include:
- Persistent logging to a database or file
- Real workload generation, not just transfers
- Multi-region client testing
- Statistical analysis post-collection
- Cost tracking
Mistakes That Produce Bad Benchmarks
Comparing on different days. Network conditions vary. Compare RPCs at the same time.
Different transaction types. A transfer benchmarks differently from a complex DeFi call. Pick one consistent transaction type.
Different priority fees. Higher tips affect landing latency. Hold the priority fee constant across RPCs.
Same blockhash. If you reuse a blockhash across RPCs, you're testing race conditions, not RPC performance. Each submission should have its own fresh blockhash.
Including warmup. First few calls have connection setup overhead. Discard the first N calls before computing statistics.
Ignoring variance. Two RPCs both averaging 200ms might have very different distributions. Look at the spread.
Interpreting Results
Once you have data:
Which RPC has best P95? Often a different question from "which has best average."
Which RPC has lowest tail? P99 and worse. Production sensitivity lives here.
Which RPC has best cost per landed transaction? Total cost divided by successful sends. Beware of comparing list prices instead of effective costs.
Which RPC behaves most consistently? Variance matters. A predictable RPC is often more useful than a sometimes-faster one.
Which RPC degrades least under load? Compare same-RPC behaviour during peak vs off-peak. Less degradation = more production-grade.
What to Do This Week
If you're evaluating RPCs:
- Pick 2-3 candidates. More than that is too much benchmarking work.
- Define your workload precisely. What you measure must match what you'll do in production.
- Set up parallel submission. Run for a week minimum.
- Track per-signature with rich metadata. Lets you analyse retroactively.
- Analyse the distribution. Not just averages.
- Compute cost per landed transaction. The most honest metric.
- Make a decision. Don't run benchmarks forever; pick and commit, with monitoring to catch regressions.
Try BoltTx as a Benchmark Candidate
If you want to include BoltTx in your benchmarks:
import { Connection } from "@solana/web3.js";
const connection = new Connection(
"https://bolttx.io/?api-key=YOUR_API_KEY",
"processed"
);
Free tier signup. Free tier is enough to run a meaningful week-long benchmark. Per-signature delivery telemetry on the BoltTx side helps with analysis.
FAQ
How long should I run benchmarks? At least a week. Day-level variance is too high; week-level captures most patterns.
How many RPCs should I compare? Two or three. More gets unwieldy; fewer doesn't give you alternatives.
Should I publish my benchmark results? If you're confident in methodology, yes — community benefits from honest data. If methodology is shaky, you'll mislead more than help.
What's a good baseline benchmark to compare against? "Public mainnet-beta endpoint" is a reasonable baseline for "what you can get without trying." Anything substantially better than that is meaningful.
Can I trust marketing benchmarks from providers? Skeptically. Methodology is often unclear. Run your own.