| Routing | Period | Queries | Avg elapsed (ms) | Avg execution (ms) | Avg backlog | Queries/min |
|---|---|---|---|---|---|---|
| Data-aware routing | Outside EOD | 190 | 566 | 528 | 0.1 | 60.9 |
| Data-aware routing | During EOD | 66 | 453 | 452 | 0.0 | 62.2 |
| Blind routing | Outside EOD | 128 | 8755 | 7519 | 2.2 | 15.1 |
| Blind routing | During EOD | 128 | 9696 | 7911 | 2.0 | 14.8 |
LBT: Customizable Load Balancing
KDB-X tools designed for 24-hour operation. Safer deploys. Trouble caught before anyone sees it.
If your firm is like many trading operations, a huge share of what happens each day — every trade, every quote, every price you need to look back on — runs through a specialized, extremely fast database such as KDB-X.
LBT is a load balancer for KDB-X which has many uses. A key capability of LBT is to provide ways to support 24-hour operation of your KDB-X systems.
Many KDB-X systems were built assuming markets close. Somewhere overnight, there was a quiet window to safely load in the day’s new data and make any changes the systems needed. For a growing number of operations — cryptocurrency that never closes, trading hours that keep extending, a global book with clients in many time zones — that quiet window is gone.
Simplicity getting to 24-hour operation need not mean your own application code gets more tangled — and it shouldn’t mean only one or two engineers understand how it works.
Speed queries stay fast because the right host already has the right data warm, without your organization paying for more compute.
Cost the same warmth that makes things fast also means you get more out of hardware you’re already paying for.
Resilience when something breaks, there’s already a path around it, not a war room.
An example architecture in one picture
Client queries come into LBT, which routes each one — not to whichever server happens to be free, but to the specific host already holding the data that query needs. Each remote host runs two HDB servants over one shared historical data set, so one servant can absorb a new day’s data while the other keeps answering queries — bringing in new data never means taking anything offline.
Notice what the picture already shows before any benchmark does: every remote host runs two HDB servants over a single historical data set, so one can absorb the day’s new data while the other keeps answering queries. That single design choice is what removes the fixed overnight maintenance window — not a promise to minimize downtime, a structural reason it doesn’t need to happen. It’s also just one of several ways to arrange this; more on the alternatives below.
Proof: what keeping a host “warm” is actually worth
Here’s a specific test, designed to isolate one effect and measure it precisely: what happens to query speed when a host’s cache stays warm with a coherent, related slice of data — say, all of last month’s history — versus when a host has to serve requests for whatever data happens to be asked of it, warm or not. We ran the same 256 historical queries twice, against the same two-host KDB-X deployment on AWS, changing exactly one thing: how LBT routed each query. In the first run, LBT sent recent-history queries to one host and older-history queries to the other, so each host’s cache stayed warm with a narrow, predictable slice of data — that’s what we mean by data-aware routing. In the second run, LBT ignored what data a query actually needed and just sent it to whichever host looked least busy — the conventional approach most load balancers use. Same queries, same two AWS instances, same hardware. Only the routing logic changed.
Outside end-of-day processing, data-aware routing ran the same queries more than 14 times faster on average, and more than 15 times faster once you count time spent waiting in queue. During simulated end-of-day processing — historically the riskiest, most avoided time to touch a KDB-X system — that gap didn’t shrink. It widened: more than 17 times faster on raw execution time, more than 21 times faster end-to-end. The busiest, most dangerous hour of the day is exactly when this pays off the most.
An occasional very slow query can be a real problem. Keeping track of average values will often miss these problems, which is why the response time experienced by the slowest 5% of queries — often abbreviated p95 — is a number we investigate and report. Put plainly, it’s the time 19 out of 20 queries typically finish within.
Under 750ms The worst 1-in-20 query, data-aware routing — even during end-of-day
About 15 seconds The worst 1-in-20 query, conventional routing — no better on a quiet day
Zoom into how those queries actually finished, not just the average: outside end-of-day, about 98% of data-aware queries finished in half a second or less; during end-of-day, that barely moves. Blind routing never gets close: about 81% of its queries take longer than half a second even on an ordinary run, climbing higher during end-of-day. Data-aware routing doesn’t degrade under end-of-day pressure. Blind routing does.
These numbers come from a realistic benchmark, run on real AWS infrastructure with a working end-to-end demo: a ticker plant, a database holding a rolling window of current data, and paired historical-data hosts that hot-swap a full day’s worth of data while the system keeps serving queries the entire time.
Some LBT-empowered consulting ideas
There are lots of things we can do to support your operation using LBT. The simplest is to help you re-organize one of your systems to operate 24 hours a day and then train your team to do this with all your KDB-X systems.
Once LBT is in operation, there are many benefits that can be achieved. Analyzing logs can allow a team to determine the amount of time different queries typically take. That information has many uses, including saving money by autoscaling services up and down, detecting performance problems before customers know they’re happening, and routing queries intelligently to the best server to support them.
Once configured for one of your KDB-X systems, an LBT subsystem powered by machine learning could help systems administrators know that these API-related symptoms —
- slowness
- memory usage increases
- network utilization increases
— occurred after certain configuration changes were made and remained in effect as long as they did, and know which servers experienced the problem versus which servers recived similar API calls but did not experience the issue. It could write all that up in a ticket, or invoke an LLM to examine code and configurations and recommend a range of solutions to consider, generating a ticket with more insight.
Several ways to get to zero downtime
There’s more than one way to eliminate the need for an overnight maintenance window with LBT — which one fits best depends on what you’re willing to trade for it. Three examples:
Hot-Swap Twins. This is the configuration behind the benchmark above: each host runs two KDB-X servants over a historical database, so one can keep serving queries while the other does what used to be “end-of-day processing”; once that’s done, both servants restart, one at a time, and LBT splits traffic between the two of them. This works whether or not each host also specializes in its own footprint — the benchmark above pairs it with the same data-aware routing described earlier, but the zero-downtime mechanism itself doesn’t require that. You achieve a speed improvement at the cost of running more than one AWS EC2 instance. In the benchmark, the first HDB servant to restart also builds the new partition upon startup.
Footprint & Safety Net. Here, give each host a specific slice of data to specialize in — recent history vs less recent, a particular set of symbols, whatever shape your queries actually take — and add one shared host that catches anything that doesn’t fit neatly into one of the so-called “footprints.” When a specialist host goes offline to handle what was formerly end-of-day processing, its traffic reroutes to the safety net until it’s caught back up. You get the speed of a well-matched specialist host most of the time, without needing every query to fit neatly into one of the defined “footprints.”
Follow-the-Sun. During a region’s own end-of-day window, LBT routes local requests to a host in a different region that is active, instead of running redundant infrastructure. It costs a small amount of added latency during that window, in exchange for meaningfully less infrastructure and low-latency responses most of the time.
What is LBT actually worth to you?
The computer resources you don’t allocate. A double-digit multiple in query speed, on the same two machines, isn’t only a speed story — it’s a capacity story. The work conventional routing needed meaningfully more hardware to approximate, data-aware routing did on what was already running. Whatever multiple your own workload lands at, it’s a number you can put directly against your next capacity-planning conversation.
The re-architecture you don’t run. LBT sits between your clients and the KDB-X processes you already have — nothing to migrate, nothing to re-platform, nothing your compliance team has to re-certify from scratch. The alternative, re-engineering your KDB-X system to run 24 hours a day, means talented and sophisticated KDB-X developers are distracted from providing real value to your organization for weeks or months.
The problems your best engineer actually gets to work on. Every hour spent babysitting a routing configuration, firefighting an end-of-day window, or manually diagnosing a “queries feel slow” ticket is an hour not spent on the thing your organization actually needs that engineer to be good at. That cost never shows up on a line item — and it’s usually the biggest one.
The full feature list
Every benefit above, and several more, with the specific mechanism that achieves each one:
| Benefits | Features | Notes |
|---|---|---|
| Skip a multi-quarter re-architecture project | LBT sits between clients and the KDB-X processes you already run — additive, not a replacement | Nothing to migrate or re-certify with compliance |
| Queries stay fast because data stays cache-warm, not because you allocated more computer resources | Routing sends each query to the host whose data is already warm — by date, by symbol, by table | Mechanism behind the speedup and sub-second worst-case times shown above |
| No scheduled outage window to roll the day’s data over | Several configurable patterns (see above) keep one copy of the data serving queries while another absorbs new data | Demonstrated end-to-end: hot-swaps a full day’s data with zero downtime |
| Closes the “something’s slow” ticket before an engineer starts digging | Per-query latency baselines and a rule-based classifier isolate what’s actually wrong and write a plain-English report | No AI guessing; proven on realistic workloads today, live-telemetry wiring is next |
| Role-based access without building your own system | Username/password authentication plus per-role permissions, available as a drop-in addition | Not a core rewrite to add |
| The system’s own behavior is auditable, not a black box | Every routing decision and configuration change is plain, readable configuration, with an automatic backup on every edit | Useful for walking a regulator or a postmortem through exactly why a query went where it did |
| React to a live incident in real time, not on the next deploy cycle | Configuration edits apply immediately and atomically | No specialized database expertise required to operate |
| Global or 24-hour operations get correct routing without extra application code | Routing can take into account any combination of query details — date range, region, instrument type | E.g. a region handling its own intraday trading while another holds history, handled transparently |
| Adding capacity is an afternoon, not a project | New host machines join the pool automatically, with traffic distributed across them | Same pattern scales from the two-host benchmark above to many |
| Every query is measurable, not just guessed at | Every query’s timing is tracked automatically — when it arrived, when it was sent out, when it came back | Same data the benchmark above and the diagnostic layer’s baselines are both built from |
| One slow query doesn’t stall the others | All communication is non-blocking — nothing waits in line behind a single slow request | Slowdowns are tracked per query, so they show up as a number, not a guess |
| Swap in a different routing approach without touching core code | Routing logic is a single, replaceable piece — several approaches ship built-in, and custom ones are straightforward to add | Doesn’t require forking or modifying the core system |
| Mixed fleets of different machine types and data are a configuration change, not a special case | Each host can carry its own routing rules and fallback behavior | Fallback rules catch anything a more specific rule misses, so nothing silently drops |
| A machine can be pulled for maintenance without anyone noticing | Disabling a host is a single configuration change, applied atomically | No query is ever mid-flight to a machine that’s already been pulled |
| Ship an emergency patch you wrote an hour ago without betting the whole trading day on it | The patched servant takes a small, defined slice of live traffic first while everything else keeps running the previous version; if LBT detects a problem, it ends the canary automatically and falls back | The automatic report names what was detected and how often it occurred — not a guess at 3am |
Alternatives to LBT
KX provides mserve_np.q — Nathan Perrem’s modified version, shipped via First Derivatives based on mserve.q that Arthur Whitney wrote for kdb+. These do load balancing on a single server, but they are not designed to support 24-hour operation, nor is Socket sharding/SO_REUSEPORT, which is the other main recommended load-balancing approach.