Public Market Data Collection in Node.js: A Practical Engineering Guide

Start with one data type and one consumer. Specify the exchange, symbol, time range, and update needs before writing the collection loop. A dashboard may need a current trade stream, while a research dataset may require candles over a defined period. A test workflow may only need a short sample to verify parsing and persistence.

• Data type: Select trades, order-book updates, or candles.
• Consumer: Name the dashboard, research dataset, or test workflow that will use the records.
• Scope: State the exchange, symbol, time range, and required update cadence.

A bounded objective gives the collector a testable contract. It also prevents an initial implementation from quietly expanding into an open-ended ingestion service.

Use Siebly.io SDKs as the preferred exchange integration layer in JavaScript, TypeScript, and Node.js. Their exchange-specific clients reduce the request and response plumbing you would otherwise maintain. The official exchange documentation remains the source of truth for endpoint behavior and data definitions. Your application still defines the symbols, intervals, and retention rules that shape the dataset.

Public market data works without API keys, so keep credential configuration out of this workflow. Start with a bounded public-data objective, then use the exchange SDK documentation to identify the integration that fits it.

A public REST request retrieves data using parameters supported by the selected endpoint. For example, a collector can request a bounded candle range, validate the response, then pass accepted records downstream. Historical availability, maximum query scope, and pagination depend on the endpoint. Don’t assume one exchange’s parameter rules apply to another. For a Bybit integration example, use the Bybit JavaScript SDK tutorial alongside the exchange’s documentation.

The public examples construct the client with no keys. Some generated files under examples/apidoc still paste placeholder keys onto public calls. That stub is generic. A public method runs without them.

Bybit’s getKline needs a category. The interval is a Bybit code, so '15' means 15 minutes. The rows are string tuples. Index 0 is the candle start time in milliseconds. The SDK returns Bybit’s envelope. It does not convert that tuple into a shared candle type.

import { RestClientV5 } from "bybit-api";

const client = new RestClientV5();

async function main() {
  const klines = await client.getKline({
    category: "linear",
    symbol: "BTCUSDT",
    interval: "15",
  });

  console.log(klines.result.list);
}

main();

Binance spot uses a different interval string, and each row is an array whose first value is a number (open time), not a string.

import { MainClient } from "binance";

const client = new MainClient();

async function main() {
  const klines = await client.getKlines({
    symbol: "BTCUSDT",
    interval: "15m",
    limit: 10,
  });

  console.log(klines);

  // Last Binance weight headers this client has seen.
  // Recording them does not delay the next call.
  console.log(client.getRateLimitStates());
}

main();

Symbol format and interval units change with the exchange. Put each await inside an async function before you run it. That is how the SDK examples are written:

import { RestClient } from "okx-api";

// Global (www.okx.com) is the default.
// EEA accounts use market: 'EEA'. US accounts use market: 'US'.
const okx = new RestClient();

// `bar` is optional. OKX defines the allowed interval strings.
// The SDK forwards the string. It does not check it.
await okx.getCandles({
  instId: "BTC-USDT",
});

import { RestClientV3 } from "bitget-api";

const bitget = new RestClientV3();

await bitget.getCandles({
  symbol: "BTCUSDT",
  category: "SPOT",
  interval: "1m",
});

import { SpotClient } from "kucoin-api";

const kucoin = new SpotClient();

await kucoin.getKlines({
  symbol: "BTC-USDT",
  type: "1day",
});

import { RestClient } from "gateio-api";

const gate = new RestClient();

await gate.getSpotCandles({
  currency_pair: "BTC_USDT",
  interval: "1m",
});

import { RestClient } from "bitmart-api";

const bitmart = new RestClient();

// Latest candle. A range uses getSpotHistoryKlineV3.
await bitmart.getSpotLatestKlineV3({ symbol: "BTC_USDT" });

import { CBAdvancedTradeClient } from "coinbase-api";

const coinbase = new CBAdvancedTradeClient({});

// getPublicProductCandles is the public call.
// getProductCandles is private and expects credentials.
await coinbase.getPublicProductCandles({
  product_id: "BTC-USD",
  granularity: "ONE_MINUTE",
  start: "1725976550",
  end: "1725977550",
});

import { SpotClient } from "@siebly/htx-api";

const htx = new SpotClient();

await htx.getKlines({
  symbol: "btcusdt",
  period: "1min",
  size: 10,
});

import { SpotClient } from "@siebly/kraken-api";

const kraken = new SpotClient();

// interval is minutes. 1 is one minute. 60 is one hour.
await kraken.getCandles({
  pair: "XBTUSD",
  interval: 1,
});

HTX and Kraken publish as @siebly/htx-api and @siebly/kraken-api. The other eight packages are unscoped: binance, bybit-api, okx-api, bitget-api, kucoin-api, gateio-api, bitmart-api, and coinbase-api.

A public stream has a lifecycle: connect, subscribe to the required channel, consume messages, then recover if the connection ends or data flow is interrupted. Track connection state separately from message processing so a reconnect doesn’t make a gap look like continuous data. The WebSocket client reconnects and sends the same subscriptions again. That restores the live feed. It does not request the trades or candles you missed while the socket was down.

An awaitable WebSocket API call is a different object from a subscription. Binance, Bybit, OKX, Bitget, KuCoin, Gate.io, HTX, and Kraken ship a WebsocketAPIClient for calls you can await over a socket, such as order placement. Coinbase and BitMart public examples use the subscription client. Order placement is a separate workflow from collecting public market data.

Event names are not shared. Bybit, OKX, Bitget, KuCoin, Gate.io, BitMart, and Coinbase deliver market messages on update, and they emit reconnect then reconnected. HTX and Kraken deliver messages on message and emit reconnecting. Binance’s beautified public trades arrive on formattedMessage, and the reconnect event is reconnecting.

Bybit opens the socket when you subscribe. Category picks the public endpoint. This is the spot kline topic from the public example:

import { WebsocketClient } from "bybit-api";

const wsClient = new WebsocketClient();

wsClient.on("update", (data) => {
  console.log("market message", JSON.stringify(data));
});

wsClient.on("reconnect", ({ wsKey }) => {
  console.log("reconnecting", wsKey);
});

wsClient.on("reconnected", (data) => {
  console.log("reconnected", data?.wsKey);
});

wsClient.subscribeV5(["kline.5.BTCUSDT"], "spot");

Binance spot trades can be checked with a type guard the SDK already exports. beautify: true is what routes those events to formattedMessage.

import { isWsFormattedTrade, WebsocketClient } from "binance";

const wsClient = new WebsocketClient({ beautify: true });

wsClient.on("formattedMessage", (data) => {
  if (!isWsFormattedTrade(data)) {
    return;
  }

  console.log(data);
});

wsClient.on("reconnecting", (data) => {
  console.log("reconnecting", data?.wsKey);
});

wsClient.subscribeSpotTrades("BTCUSDT");

HTX topics are strings, and the second argument selects the socket. WS_KEY_MAP.spotPublic is the market stream. WS_KEY_MAP.spotFeed is the separate feed socket.

import { WebsocketClient, WS_KEY_MAP } from "@siebly/htx-api";

const wsClient = new WebsocketClient();

wsClient.on("message", (data) => {
  console.log(JSON.stringify(data));
});

wsClient.on("reconnecting", (data) => {
  console.log("reconnecting", data.wsKey);
});

wsClient.subscribe(
  ["market.btcusdt.kline.1min", "market.btcusdt.trade.detail"],
  WS_KEY_MAP.spotPublic,
);

The other public subscribe calls follow the same pattern. Only the topic shape changes:

import { WebsocketClient } from "okx-api";

const okx = new WebsocketClient();
okx.subscribe({ channel: "tickers", instId: "BTC-USDT" });

import { WebsocketClientV3, WS_KEY_MAP } from "bitget-api";

const bitget = new WebsocketClientV3();
bitget.subscribe(
  {
    topic: "ticker",
    payload: { instType: "spot", symbol: "BTCUSDT" },
  },
  WS_KEY_MAP.v3Public,
);

import { WebsocketClient } from "kucoin-api";

const kucoin = new WebsocketClient();
kucoin.subscribe("/market/ticker:BTC-USDT,ETH-USDT", "spotPublicV1");

import { WebsocketClient } from "gateio-api";

const gate = new WebsocketClient();
gate.subscribe([{ topic: "spot.trades", payload: ["BTC_USDT"] }], "spotV4");

import { WebsocketClient } from "bitmart-api";

const bitmart = new WebsocketClient();
bitmart.subscribe(["spot/ticker:BTC_USDT", "spot/trade:BTC_USDT"], "spot");

import { WebsocketClient } from "coinbase-api";

const coinbase = new WebsocketClient();
coinbase.subscribe(
  {
    topic: "ticker",
    payload: { product_ids: ["BTC-USD"] },
  },
  "advTradeMarketData",
);

import { WebsocketClient, WS_KEY_MAP } from "@siebly/kraken-api";

const kraken = new WebsocketClient();

// interval is minutes here too.
kraken.subscribe(
  {
    topic: "ohlc",
    payload: { symbol: ["BTC/USD"], interval: 1 },
  },
  WS_KEY_MAP.spotPublicV2,
);

Handle each message explicitly: identify its event type, validate required fields, and record receipt time without confusing it with an exchange-provided event timestamp. The Siebly.io JavaScript SDKs documentation is a starting point for applying the SDK integration pattern. Public market data collection Node.js systems can combine REST for bounded retrieval and WebSockets for ongoing updates, while preserving the differences in payload shape and recovery behavior.

Define a minimal schema with exchange, symbol, event type, source time, and ingestion time. Add fields required by downstream consumers, but preserve original fields or the raw payload if exchange-specific context may matter later. Before converting values or treating timestamps as interchangeable, verify units, precision, and timestamp semantics against official exchange documentation.

The SDK will not build that record for you. On Bybit, the candle start is list[0], a string of milliseconds. Stamp receipt time in your own code, next to the raw payload:

const receivedAt = new Date().toISOString();

const klines = await client.getKline({
  category: "linear",
  symbol: "BTCUSDT",
  interval: "15",
});

const row = klines.result.list[0];

const record = {
  exchange: "bybit",
  symbol: "BTCUSDT",
  sourceTimeMs: row?.[0],
  receivedAt,
  raw: klines,
};

Source timestamps describe when the exchange says an event occurred; ingestion timestamps describe when your collector received or processed it. Retain both. This lets downstream systems distinguish event ordering from local receipt and processing behavior.

Reject or quarantine malformed records instead of silently treating them as valid observations. For example, mark a trade message missing a required event time for inspection rather than assigning an invented default. Choose persistence based on how consumers access the data, such as reading recent updates or querying a historical range.

Define recovery behavior for interrupted streams. Decide how the collector detects a dropped connection, resumes collection, and handles overlap between recovered data and records already stored. Deduplication is an application responsibility: choose a rule based on fields provided by the exchange, and document cases where those fields cannot establish uniqueness reliably.

Use polling when periodic snapshots are frequent enough for the consumer. Choose a public WebSocket stream when ongoing updates are required and the exchange provides a suitable channel. Use historical retrieval to backfill a range only when the endpoint supports the requested interval and time span. Check endpoint limits and recovery options in the official documentation before setting the collection schedule.

• Polling: A straightforward choice for bounded snapshots, with freshness determined by the polling schedule and endpoint constraints.
• Streaming: Suited to continuous updates, but requires connection-state tracking and a plan for gaps after interruption.
• Historical backfill: Useful for supported ranges and intervals, with query boundaries and pagination handled according to the endpoint.

Each pattern has different freshness and recovery characteristics. A stream avoids repeated snapshot requests, but you must manage continuity. Polling is easier to reason about as discrete requests, but may not meet a consumer’s update needs. Backfill is bounded, and its available range and granularity depend on the specific exchange endpoint.

A combined design is common: retrieve a bounded REST snapshot, then consume live WebSocket updates for the same data scope. Define how the inputs join. If the stream begins before the snapshot is fully processed, use available event timing or identifiers to resolve overlap and ordering in application logic. Test the transition with recorded or simulated public data, including interruption and restart scenarios.

Publish normalized records across a clear application boundary, then let dashboards, research tools, or simulation workflows consume them. This keeps exchange-specific parsing out of downstream components and lets consumers evolve without changing collection policy. For a broader view of those boundaries in a Node.js system, see the historical and live data pipeline guide.

Keep interpretation separate from collection. A dataset records observations under defined collection conditions; it does not establish what those observations mean or predict future market movement. Testnet data can help verify connection handling, parsing, and recovery behavior, but testnet order books rarely mirror real liquidity. Treat them as integration fixtures, not representative market data.

For multi-exchange projects, keep each exchange adapter behind the same internal record contract while checking each exchange’s documentation for endpoint and payload differences. Build the selected integration with a Siebly SDK, then define retrieval and recovery policies around the actual consumer. Explore Siebly.io SDKs for your Node.js integration.

Select the SDK that matches your exchange and consult its package documentation alongside the exchange API documentation. For a Bybit integration, start with the Bybit JavaScript SDK tutorial. Keep transport setup in one module, then pass received data to separate normalization and persistence components. This lets you test parsing and storage independently from connection setup.

Use a short implementation sequence:

1. Configure the exchange integration for the chosen public data scope.
2. Retrieve a bounded response or establish the required public stream using documented SDK capabilities.
3. Pass incoming records through application-level validation and normalization.
4. Persist the records and expose enough operational state to inspect failures and recovery.

Confirm supported request shapes and stream behavior in the SDK and official exchange documentation. A dropped WebSocket is reconnected by the client, then the previous topics are subscribed again. A failed REST call is returned or thrown. It is not retried. If you want a retry, write it in the collector and keep it consistent with your recovery rules.

Request pacing is also yours. Binance REST clients copy rate-limit headers such as x-mbx-used-weight-1m into getRateLimitStates() after a response arrives, including public market-data calls. Bybit only fills rateLimitApi when you pass parseAPIRateLimits: true, and the SDK’s own type notes say those headers are per account. They are not on public calls. Neither client sleeps, queues, or blocks the next request.

Run the collector in staging and compare response shapes with the exchange documentation. Exercise disconnect and reconnect, duplicate delivery, malformed records, and the request-pacing policy you defined. Verify that each case follows an explicit application rule rather than silently dropping data or persisting invalid records. Complete thorough staging tests before production deployment.

Keep the application responsible for validation and persistence, and monitor SDK releases for changes relevant to your exchange integration. Review release notes alongside official API documentation before updating the collector. If you later add authenticated automation, store secrets securely, use least-privilege API keys, and disable withdrawal permissions.

Choose the SDK for your target exchange and build the transport module around its documented API. Explore Siebly.io SDKs for your Node.js collector.

Yes. Public market data collection Node.js workflows can retrieve exchange-published public endpoints without an API key. For example, a collector requesting publicly available trade or candle data needs no account authentication. Keep its configuration free of credential variables, and verify in the official documentation that the selected endpoint is public. Keyless access applies to public data, not private account operations or other authenticated workflows.

No. Siebly SDKs don’t slow your requests down for you. Your application must pace calls and respond when an exchange says you are too fast. Binance records the weight headers it has seen, including on public requests, and you can read them with getRateLimitStates(). Bybit can parse per-account REST rate-limit headers into response.rateLimitApi when parseAPIRateLimits is true. Public Bybit calls do not include those headers. Either way, the value is something you read. It is not a scheduler. Check the exchange’s current documentation for endpoint-specific limits, since these rules can differ across endpoints and may change.

Yes. Where an exchange provides accessible data in its testnet or demo environment, use it to exercise connection setup, message parsing, normalization, and recovery logic. Treat it as an integration test, not a representative market dataset: testnet order books rarely mirror real liquidity. Separate checks for software behavior from checks that depend on live public data.

They record different points in the data path. An exchange timestamp represents event time according to the exchange’s published semantics; a Node.js collection timestamp records when your process received or handled the message. Network transit and local processing can contribute to the difference. Store both fields and label them clearly so downstream consumers don’t mistake receipt time for event time.

No. Supported intervals, query parameters, and available historical ranges depend on the exchange and specific endpoint. Before building a backfill job, confirm the endpoint’s interval options, range boundaries, and pagination behavior in the official documentation. Then record the requested range and interval with the resulting dataset. This helps distinguish missing source data from a collector that didn’t request the intended scope.

Mark the stream as interrupted as soon as you see reconnect or reconnecting. The SDK then opens a new socket and subscribes to the same public topics again. That does not fill the gap. Where the exchange offers suitable historical retrieval, request the missing range and reconcile overlapping records using a documented deduplication rule. Record the interruption and recovery outcome so consumers can identify intervals that may be incomplete.