Before now, using @platformatic/kafka with Confluent Schema Registry meant writing extra code to connect the pieces. With @platformatic/kafka v1.27.0, that’s no longer needed.

@platformatic/kafka now has built-in support for Confluent Schema Registry, including:

• AVRO

• Protocol Buffers

• JSON Schema

• Basic and Bearer authentication

• Automatic schema fetch and caching

• Integrated Producer and Consumer hooks

You get schema-aware messaging, and the project still focuses on being fast and predictable for Node.js Kafka clients.

Why This Matters

Most schema registry integrations add complexity where you don’t want it: in the message serialization and deserialization paths. Fetching remote schemas is asynchronous, but encoding and decoding should stay synchronous for speed and consistency.

Put simply, network I/O and cache coordination should happen before the main data processing, not during it. Keeping these steps separate helps maintain stable throughput and latency as traffic increases.

This release introduces a two-layer architecture to keep that separation clear:

1. Low-level hooks for async pre-processing:

   • beforeSerialization

   • beforeDeserialization

2. High-level registry API via ConfluentSchemaRegistry

In practice, this means schemas are fetched and cached before encode/decode happens, so your serializers and deserializers stay synchronous when messages are processed.

This gives application teams a simpler way to think about things: do the asynchronous prep first, then keep codec behavior predictable during main processing.

At a high level, the flow is:

• Extract schema ID from message metadata (producer) or wire payload (consumer).

• Resolve schema from local cache when available.

• On cache miss, fetch asynchronously via beforeSerialization/beforeDeserialization hooks and cache the schema.

• Run synchronous serialization/deserialization with the resolved schema.

In multi-instance deployments, that cache layer can be backed by Redis or Valkey, so workers share schema state across nodes while keeping encode/decode synchronous in the hot path.

What You Can Do Now

You can connect a registry directly to both the Producer and Consumer, letting @platformatic/kafka handle schema-aware serialization from start to finish.

This is especially helpful when several services publish and consume the same topics on different deployment cycles, since consistent schema handling is a must.

import { Consumer, Producer } from '@platformatic/kafka'
import { ConfluentSchemaRegistry } from '@platformatic/kafka/registries'

const registry = new ConfluentSchemaRegistry({
  url: 'http://localhost:8081'
})

const producer = new Producer({
  clientId: 'orders-producer',
  bootstrapBrokers: ['localhost:9092'],
  registry
})

const consumer = new Consumer({
  groupId: 'orders-consumers',
  clientId: 'orders-consumer',
  bootstrapBrokers: ['localhost:9092'],
  registry
})

When producing, pass schema IDs in message metadata:

await producer.send({
  messages: [
    {
      topic: 'orders',
      key: { orderId: 101 },
      value: { customerId: 'cust-44', total: 129.99 },
      metadata: {
        schemas: {
          key: 10,
          value: 11
        }
      }
    }
  ]
})

When consuming, payloads are automatically decoded with the cached schema. If a schema isn’t found, the registry fetches it before deserialization continues.

This makes it easy to move from custom codec code to a single registry integration in your client setup.

Authentication and Enterprise Scenarios

Schema Registry deployments are often protected. The new integration includes:

• Basic auth (username + password)

• Bearer token auth (token)

• Dynamic credentials via providers

This makes it easier to connect to managed or secured registry instances without writing custom transport code. It also makes credential rotation simpler when you use providers.

If your setup uses short-lived credentials, provider functions let you refresh tokens and secrets without having to rebuild your producer or consumer logic.

Performance and Reliability Considerations

One main design goal was to avoid unnecessary overhead to message processing.

The implementation focuses on cache locality and step-by-step pre-processing:

• Schema IDs are extracted from the wire format (or message metadata).

• Unknown schemas are fetched once and cached.

• Repeated schema IDs in a batch are resolved from the cache.

• Encode/decode continues in synchronous paths.

This setup cuts down on unnecessary async work while still supporting remote schema registries safely. It also helps keep throughput and performance steady, as you’d expect from a Node.js client.

Operationally, this also makes failures easier to understand. Schema resolution errors happen during fetch or preparation, while codec errors are still linked to payload and schema compatibility.

Also Included in This Release

The v1.27.0 release also shipped quality improvements around consumer behaviour and protocol handling, with broad test coverage and new playground clients for:

• AVRO

• Protobuf

• JSON Schema

• Authenticated Schema Registry setups

The end result is a production-ready integration you can try out quickly, starting in local development and moving to secure production registries.

Experimental API Notice

ConfluentSchemaRegistry and its related hooks are currently experimental. They may change in minor or patch releases as we keep improving them based on real-world use and feedback.

If you plan to use this in production, make sure to pin your versions and check the release notes. We’ll keep refining the API based on feedback from real deployments.

If your team is rolling this out, here’s a practical way to start:

1. Start with one topic and one schema format (typically AVRO or JSON Schema)

2. Validate serialization/deserialization behaviour in staging with real payloads.

3. Expand topic coverage and introduce auth/credential providers as needed.

Getting Started

Install the package:

npm install @platformatic/kafka

For Protobuf support, also install:

npm install protobufjs

Next, follow the full integration guide in the documentation:

• Confluent Schema Registry docs

• v1.27.0 release notes

If you give it a try, we’d love to hear your feedback at hello@platformatic.dev. Real-world schema workflows will help shape the next version of this API and guide our priorities for future improvements.

Thanks for building with us! 🚀

We ran our SSR framework benchmarks again after finding out that compression was not applied the same way across all frameworks. In the original tests, TanStack did not have compression enabled. React Router had gzip compression turned on in its Express server.js, but Watt skips server.js and uses Fastify. Because of this, React Router’s Watt runs had no compression overhead, while its Node and PM2 runs did. This made Watt seem faster than it actually was compared to the other runtimes.

Once we removed compression from React Router to make the comparison fair, the updated results gave us a clearer picture:

TanStack and React Router are still the top performers, while Next.js continues to have trouble at 1,000 requests per second. The main change is that Watt’s advantage now appears mostly in tail latencies, with p(99) at 83-89ms compared to Node’s 121-298ms, instead of average response times.

What Went Wrong in the Previous Benchmarks

HTTP compression means shrinking the response body before sending it over the wire, usually with gzip or Brotli. That typically reduces bandwidth and speeds up delivery of HTML, JSON, and JavaScript, which is why some frameworks enable it by default as a sensible production optimization. Others leave it off because compression is often handled more efficiently by a CDN or reverse proxy, and because it puts extra load on the CPU.

• Next.js: Its built-in compress option works at the framework level, not just the HTTP server. We checked and confirmed that Next.js serves gzip responses with both Node and Watt, so compression is always applied no matter the runtime.

• TanStack Start: Never had compression configured in any runtime. All three runtimes (Node, PM2, Watt) served uncompressed responses. No inconsistency, but it made the comparison between frameworks unfair.

• React Router: does not ship a default server, but there are several templates. In the one we followed, compression was enabled; Watt did not follow the same example, and it had no compression.

The Fix

We turned off compression on React Router by taking out the compression() middleware and uninstalling the package from server.js. We also set compress: false in Next.js’s next.config.mjs to make sure all three frameworks were tested the same way. Now, with compression removed everywhere, all frameworks serve uncompressed responses in every runtime.

In production, it’s best to handle compression at the reverse proxy or CDN layer, not in the application server.

Corrected Results

All tests run at 1,000 req/s for 3 minutes with mixed e-commerce traffic (homepage, search, card details, game browsing, sellers - you can read more about the sample app we built for these benchmarks here) on AWS EKS. No compression, no Accept-Encoding headers.

Software Versions

React Router: Consistent Across All Runtimes

React Router can handle 1,000 requests per second with no failures on any of the three runtimes. Watt and PM2 have almost the same median response time at 15ms. The difference shows up at the higher end: Watt’s p(99) is 83ms, PM2’s is 123ms, and Node’s is 298ms.

TanStack Start: Watt and Node Neck-and-Neck

TanStack with Watt and TanStack with Node perform almost the same: they have the same average, median, and p(95) times. Watt is slightly better at p(99), with 89ms compared to Node’s 121ms.

PM2 stands out as the outlier. With an 81% success rate and a 2.5 second average latency, PM2’s cluster fork model does not work well with Nitro’s srvx server. This is a problem between PM2 and Nitro, not TanStack. The same PM2 cluster mode works perfectly with React Router’s Express server, giving 100% success and a 20ms average.

Next.js: Still Struggling at 1,000 req/s

Next.js cannot handle 1,000 requests per second, no matter which runtime is used. All three runtimes have about a 55% success rate, which shows that the framework itself is the bottleneck, not the runtime. The high tail latencies (p(99) over 60 seconds) mean requests are piling up and timing out.

What Changed vs the Original Blog Post

Previous React Router Results (with compression inconsistency)

Corrected React Router Results (no compression anywhere)

The average latency numbers are similar because Node’s response time was mostly affected by SSR work, not compression. Still, making this correction is important for our methods. Now we can be sure the gap is real and not just a mistake.

TanStack’s results were always fair. The numbers changed a bit (from 13ms to 18ms) because of normal differences between runs, not because of compression changes.

Key Takeaways

1. Benchmark Hygiene Matters
   A single middleware inconsistency, like having compression enabled in server.js but skipped by Watt, was enough to make our results questionable. Always make sure your test conditions are the same for every variant, especially when runtimes load applications in different ways.

2. Watt’s Real Advantage: Tail Latency
   With compression turned off, Watt and Node have similar average and median latencies on both TanStack and React Router. However, Watt always comes out ahead at p(99):

   

   This is important for services where tail latency matters, like APIs for user-facing pages or services with strict SLAs.

3. PM2 Cluster Mode Has Compatibility Issues
   PM2 works well with Express (React Router: 100% success, 20ms average), but not with Nitro (TanStack: 81% success, 2.5s average). If you use Nitro-based frameworks like TanStack Start or Nuxt, it’s better to avoid PM2 cluster mode and use Watt or plain Node instead.

4. Next.js at 1,000 req/s: The Runtime Doesn’t Matter
   All three runtimes, Watt, PM2, and Node, perform the same on Next.js at this load, with about 55% success and a 9-second average. The bottleneck is in Next.js’s SSR pipeline, not in how connections are handled.
   Part of the advantage of Watt is a better handling of CPU-bound activities, like compression. Disabling it reduces the advantage.

5. TanStack Start and React Router Are Both Excellent
   With compression handled the same way, TanStack (18ms average) and React Router (19ms average on Watt) are very close in performance. Both can handle 1,000 requests per second with 100% success. So, you should choose between them based on developer experience and ecosystem fit, not just performance.

Reproducing These Benchmarks

The complete benchmark infrastructure is available at:
https://github.com/platformatic/k8s-watt-performance-demo/tree/ecommerce

# Benchmark TanStack Start
AWS_PROFILE=<profile-name> FRAMEWORK=tanstack ./benchmark.sh

# Benchmark React Router
AWS_PROFILE=<profile-name> FRAMEWORK=react-router ./benchmark.sh

# Benchmark Next.js
AWS_PROFILE=<profile-name> FRAMEWORK=next ./benchmark.sh

Conclusion

Getting benchmarks right is tough. Even if you have the same applications, the same infrastructure, and careful methods, one small inconsistency, like compression middleware in one framework’s server file that is skipped by one runtime but not others, can make your results questionable.

The corrected results support the main points from our original post: TanStack Start and React Router can easily handle 1,000 requests per second, Next.js struggles at that level, and Watt gives real improvements across all three frameworks, especially for tail latencies. Now, though, we have more accurate numbers and a better idea of where each runtime really helps.

Being open about our methods is important. We made a mistake, fixed it, and are sharing both the error and the fix so others can learn from our experience.

If you’d like to talk about using Watt in your setup or want to learn more, email us at hello@platformatic.dev or reach out toLuca orMatteo on LinkedIn.

On Vercel, all of this works out of the box — the platform handles deployment versioning and queue routing behind the scenes. But what happens when you deploy to your own Kubernetes cluster? Version mismatch. And it’s subtle enough to corrupt data before you notice.

We built Platformatic World to fix this. It’s a drop-in World implementation that brings the same deployment safety to any Kubernetes cluster. Every workflow run is pinned to the code version that created it. Queue messages are routed to the correct versioned pods. Old versions stay alive until all their in-flight runs are complete.

The version mismatch problem

Workflow DevKit uses deterministic replay. When a workflow resumes after a step, it runs the whole function again from the start, matching each step to its cached result by order. The correlation IDs that link steps to cached results come from a seeded random number generator tied to the run ID. If the code and seed are the same, the sequence stays the same.

This works perfectly until you deploy a new version.

If a run that started on v1 replays on v2 and the step order has changed, the correlation IDs won’t match anymore. For example, the cached result from chargeCard could be used for the new addDiscount step:

The workflow can quietly produce wrong results or fail in ways that are hard to spot. On Vercel, the Vercel World handles this for you. On self-hosted Kubernetes, you have to manage it yourself.

We already solved this for HTTP

ICC (Intelligent Command Center) is our Kubernetes controller for managing app deployments. We recently added skew protection. Here’s how it works for HTTP traffic:

When a user starts a session on version N, a cookie pins all subsequent requests to version N via Gateway API HTTPRoute rules. New visitors are routed to the latest active version.

Workflow runs work the same way: a run that starts on version N must keep running on version N until it finishes. The difference is in the transport. HTTP requests go through the Gateway API, but workflow queue messages do not.

Why we couldn’t just extend the Intelligent Command Center

Our first design had pods accessing PostgreSQL directly, with ICC handling queue routing. We abandoned it because the ICC couldn’t reliably determine when a version had no in-flight runs.

The problem: workflow runs can be suspended in ways that are invisible to the infrastructure

When a workflow registers a webhook and then suspends, the pod becomes idle. There’s no memory use, no heartbeat, and no queue message. ICC sees no activity and expires the version. If someone clicks the webhook link hours later, the run’s pods are already gone:

The only way to know if a version still has work in progress is to check the runs table. For that, you need a service that owns the data.

How Platformatic World works

Platformatic World consists of two packages:

• @platformatic/workflow is a Watt application backed by PostgreSQL that manages all workflow state and queue routing. Every operation, like event creation, run queries, queue dispatch, hook registration, and encryption, goes through it.

• @platformatic/world is a lightweight HTTP client that implements the Workflow DevKit’s World interface. This is what your app imports.

The service enforces multi-tenant isolation at the SQL level by scoping every query to the application_id.

Version-aware queue routing

Each queue message includes a deployment_version. The router finds the registered handler for that version and sends the message to the right pod. Messages for v1 always go to v1 pods, even after v2 is deployed:

If a dispatch fails, it uses exponential backoff and tries up to 10 times before moving the message to the dead-letter queue.

Safe version draining

When ICC finds a new version, it checks with the workflow service to see if the old version still has any work in progress. The service looks at active runs, pending hooks, waiting sleeps, and queued messages. ICC only decommissions the old version when all these counts are zero:

If a version stays alive longer than allowed, ICC can force-expire it. This cancels in-flight runs, moves queued messages to the dead-letter queue, and deregisters handlers.

Zero-config in Kubernetes

In production with ICC, you don’t need to write any configuration code. You just set two environment variables in your Dockerfile and add three pod labels in your Deployment spec:

ENV WORKFLOW_TARGET_WORLD="@platformatic/world"
ENV PLT_WORLD_SERVICE_URL="http://workflow.platformatic.svc.cluster.local"

# Pod labels in your Deployment spec
labels:
 app.kubernetes.io/name: my-app
 plt.dev/version: "v1"
 plt.dev/workflow: "true"

The Workflow DevKit discovers the world automatically. At startup, @platformatic/world (the library your app imports) resolves the app ID from the PLT_WORLD_APP_ID env var or the package.json name, detects the deployment version from the plt.dev/version label via the K8s API, and authenticates using the pod’s ServiceAccount token. On the infrastructure side, ICC sees the plt.dev/workflow label and registers queue handlers with @platformatic/workflow, so dispatched messages reach the correct versioned pod.

You don’t need to change your workflow code. The same 'use workflow' and 'use step' directives work just like they do on Vercel.

Local development

For local development, the workflow service runs in single-tenant mode without authentication — no K8s, no ICC. Start PostgreSQL and the workflow service:

npx @platformatic/workflow
postgresql://user:pass@localhost:5432/workflow

Then configure your app to connect to it with the same two environment variables from the Dockerfile above, just pointing at localhost:

WORKFLOW_TARGET_WORLD=@platformatic/world
PLT_WORLD_SERVICE_URL=http://localhost:3042

Your app also needs to call world.start() Once the server starts, this registers a queue handler so the workflow service can dispatch messages back to your app. In K8s with ICC, this is a no-op (ICC handles it). Here’s a Next.js example using instrumentation.ts:

// instrumentation.ts — Next.js calls register() once on server startup
export async function register() {
  if (process.env.PLT_WORLD_SERVICE_URL) {
    const { createWorld } = await import(‘@platformatic/world’)
    const world = createWorld()
    await world.start?.()
  }
}

Other frameworks have different startup hooks (Fastify plugins, Express middleware, etc.) — the key is to call world.start() once before your app starts handling requests.

The service auto-provisions a default application, so no further setup is needed.

Observability in ICC

The ICC dashboard gives you full visibility into your workflow runs. The Workflows tab shows a real-time list of all runs for each application, with status, version, and duration.

Click a run to inspect it. The Trace view shows a waterfall of every step, with timing bars and status indicators. You can see exactly where time was spent and which steps ran in parallel.

The Graph tab visualizes the workflow structure as a directed graph. Sequential steps flow vertically, parallel steps are laid out side-by-side. After the first completed run of a version, the graph pre-renders immediately for subsequent runs — so you see the full structure before the workflow even starts executing.

You can also replay completed runs from the dashboard (targeting the original deployment version), cancel running workflows, and inspect hooks, events, and streams.

Try it

You can find the repository at github.com/platformatic/platformatic-world. The @platformatic/world package is a drop-in replacement for Vercel World. If your workflows run on Vercel today, they’ll work on your cluster with Platformatic World.

We’d love to hear how you use it. Feel free to open an issue or contact us on Discord.

Performance benchmarks capture a moment, not a final judgment. Results depend on a specific workload, scale, and constraints; they do not rank frameworks by value. Next.js stands out for its widespread adoption, strong compatibility, and vast ecosystem trusted by millions. TanStack, as a newcomer, made bold architectural choices. React Router is positioned differently along the maturity curve. Each framework wins in its own context.

The numbers matter less than the response: every team addressed our shared data and delivered fixes. This collaboration with open data, shared flamegraphs, and upstream fixes makes Node.js a safe, long-term choice for enterprise teams.

We updated our Benchmarks! View the new numbers Here

TL;DR

With help from Claude Code, we built the same eCommerce app in three SSR frameworks and tested them at 1,000 requests per second on AWS EKS. We ran each framework both on Watt and directly on Kubernetes.

The results revealed big performance differences and highlighted a few key themes:

1. Running Node services on Watt improves average latency.

2. The TanStack team is doing excellent work. Their framework outperformed the others we tested by a wide margin.

3. The Next.js team has made impressive performance improvements. Upgrading from v15 to v16 canary more than doubled throughput and reduced latency by six times. Their collaboration also led to a 75% speedup in React’s RSC deserialization, which benefits everyone using React.

Both the TanStack and Next.js team used platformatic/flame to find and resolve critical performance bottlenecks the benchmark uncovered - more on that below.

TanStack Start outperformed React Router by 25% in throughput and had 35% lower latency. Both frameworks achieved a 100% success rate, meaning every request got an HTTP 200 response within our 10-second timeout. This strict definition makes the comparison fair and matches real-world SLA expectations. Next.js struggled under our benchmark load, but upgrading from v15.5.5 to v16.2.0-canary.66 more than doubled its throughput (from 322 to 701 requests per second) and reduced average latency by six times.

To mirror common enterprise eCommerce scenarios, no caching was used in this test, as it’s often avoided due to aggressive personalization and A/B testing. In many large-scale e-commerce deployments, personalization strategies ensure that individual user views have minimal overlap, often less than 5%,which means that cache hits provide minimal benefit compared to the invalidation overhead. This explicit trade-off reflects real-world scenarios, where companies choose to prioritize dynamic user experiences over the potential gains from caching.

Collaboration note: We shared benchmark data and flamegraphs (via platformatic/flame) with both the TanStack and Next.js teams. The TanStack team fixed a critical bottleneck, delivering a 252x improvement in response times. The Next.js team’s Tim Neutkens used our flamegraphs to identify a JSON.parse reviver overhead in React Server Components, resulting in a 75% speedup in RSC deserialization merged into React itself.

While we run these benchmarks on a canary release of Next.js, all the advantages are part of Next.js 16.2.0, which is coming out very soon.

The Challenge: Apples-to-Apples Framework Comparison

Comparing SSR performance (or performance generally) across frameworks is notoriously tricky because teams tend to only write and deploy their apps to a single framework, so it’s rare to get a reasonable “like-for-like” comparison.

Luckily for us, we live in an era where writing code is as cheap as however many tokens it costs to generate your favorite LLM. So we made 3 (more-or-less) identical eCommerce sample applications with the help of our dear friend Claudio (feel free to check out the code for yourself here).

The Application: CardMarket

For these benchmarks, we built a trading card marketplace app, similar to a simpler version of TCGPlayer or CardMarket. The data model includes 5 games (Pokémon, Magic: The Gathering, Yu-Gi-Oh!, Digimon, and One Piece), 50 card sets (10 per game), 10,000 cards (200 per set), 100 sellers with ratings and locations, and 50,000 listings with prices, conditions, and quantities.

The app includes several types of pages and routes to create a realistic e-commerce experience, all generated by Claude Code:

• The homepage shows featured games, trending cards, and new releases.

• There’s a search page with full-text search, filtering, and pagination.

• Game detail pages show info about each game and its sets, while set detail pages list cards with pagination.

• Card detail pages display card info and seller listings.

• The sellers’ list page shows all sellers with their ratings, and each seller has a profile and listings page.

• There’s also a cart page with a static shopping cart.

We made several design choices to keep the implementations consistent:

• All data comes from JSON files, and every framework uses the same data.

• We added a random 1-5ms delay to simulate real database latency.

• Every route uses full SSR with no client-side data fetching.

• All versions share the same UI components, layouts, and Tailwind CSS styling.

The Frameworks

We implemented this application in three frameworks:

1. TanStack Start (v1.157.16) - The newest entrant, built on TanStack Router with Vite for SSR

2. React Router (v7) - The classic routing library, now with first-class SSR support.

3. Next.js (v15, updated to v16 canary) - The established leader in React SSR

Each implementation uses the framework’s idiomatic patterns:

• TanStack Start: createFileRoute with loader functions

• React Router: Route modules with loader exports

• Next.js: App Router with Server Components

The Runtimes

For each framework, we tested two runtime configurations:

1. Node.js - Single-threaded, 6 pods with 1 CPU allocated for each

2. Watt - Multi-worker with SO_REUSEPORT, 3 pods with 2 CPUs allocated, with 2 workers per pod to use those 6 CPUs to the fullest

All configurations received identical total CPU allocation (6 cores) for fair comparison.

Test Methodology

Infrastructure

• EKS Cluster: 4 nodes running m5.2xlarge instances (8 vCPUs, 32GB RAM each)

• Load Testing Instance: c7gn.2xlarge (8 vCPUs, 16GB RAM, network-optimized)

• Region: us-west-2

• Load Testing Tool: Grafana k6

Software Versions

All versions are locked in package.json for reproducible benchmarks:

Load Test Configuration

Each test followed this protocol:

1. NLB Warm-up: 60 seconds ramping from 10 to 500 req/s

2. Pre-test Warm-up: 20 seconds at moderate load

3. Cool-down: 60 seconds before the main test

4. Main Test: 60 seconds ramp-up to 1,000 req/s, then 120 seconds sustained

5. Between Tests: 480 seconds cooldown

Realistic Traffic Distribution

The load test simulated realistic e-commerce traffic patterns:

Results

TanStack Start: The Performance Leader

After Update (v1.157.16)

TanStack Start delivered exceptional performance, the highest throughput and lowest latency of all frameworks tested. With Watt, average response times stayed under 13ms even at 1,000 requests per second.

React Router: Solid and Reliable

React Router managed the load well and had zero failures. Using Watt made response times 38% faster compared to standalone Node.js.

Next.js: Struggling Under Load, but Making Progress

Initial Benchmark (Next.js 15.5.5, Watt 3.32.0)

Next.js couldn’t handle 1,000 requests per second. Response times averaged 8 to 11 seconds, and about 40% of requests failed. Even with Watt’s optimizations, Next.js lagged behind the lighter frameworks.

Updated Benchmark (Next.js 16.2.0-canary.66, Watt 3.39.0)

We re-ran the benchmarks after upgrading to the latest Next.js canary and Watt 3.39.0 to see if the situation had improved:

Next.js Version Improvement (Watt runtime)

Upgrading from Next.js 15.5.5 to 16.2.0-canary.66, along with Watt 3.39.0, brought a big improvement:

• Throughput more than doubled

• Average response times dropped by over six times

• We saw an 83% reduction in latency.

The success rate only improved a little (about 36% of requests still failed), but the successful requests were served much faster, with the median response time dropping from seconds to 431ms.

This is real progress. Next.js is still the slowest of the three frameworks at this load, but the gap is closing, and more improvements are on the way.

Framework Collaborations: Benchmarks as a Catalyst

One of the best parts of this project was working directly with the framework teams. Sharing real-world benchmark data, especially flamegraphs that show where time is spent, helped turn abstract performance talks into real fixes. (If you are on a web performance team, we’d love to talk.)

The Next.js Collaboration: Fixing RSC Deserialization

After our initial Next.js benchmarks showed multi-second response times, we shared flamegraphs from our load tests withTim Neutkens from the Next.js team. The flamegraphs revealed a clear hotspot: initializeModelChunk. This function calls JSON.parse with a reviver callback in React Server Components (RSC) chunk deserialization.

The root cause was a well-known V8 performance characteristic: JSON.parse is implemented in C++, and passing a reviver callback forces a C++ → JavaScript boundary crossing for every key-value pair in the parsed JSON. Even a trivial no-op reviver (k, v) => v makes JSON.parse roughly 4x slower than bare JSON.parse without one. Since initializeModelChunk is called for every RSC chunk during SSR, this overhead compounds rapidly on pages with many server components.

Tim identified the fix and submitted it directly to React:facebook/react#35776 (merged Feb 19, 2026). The change replaces the reviver callback with a two-step approach—plain JSON.parse() followed by a recursive tree walk in pure JavaScript—yielding a ~75% speedup in RSC chunk deserialization:

This fix helps every React framework that uses Server Components, not just Next.js. It shows how profiling with real workloads can reveal optimization opportunities that microbenchmarks might miss.

The improvement is already reflected in our updated Next.js benchmarks (v16.2.0-canary.66), and we expect further gains as this optimization and others land in stable releases.

The TanStack Turnaround: A Case Study in Rapid Optimization

Interestingly enough, we had a similar journey with the TanStack team. Our initial benchmarks used TanStack Start v1.150.0, and the results were concerning: requests timing out, 75% success rates, and average response times exceeding 3 seconds. We shared these findings with the TanStack team, who quickly identified the critical bottlenecks (also via @platformatic/flame) in their SSR request handling pipeline.

Within 7 minor versions, they shipped a fix. We re-ran the benchmarks on v1.157.16, and the transformation was extraordinary:

The v1.150 numbers tell the story of a framework under distress. The p(95) latency hitting exactly 10,001ms wasn’t a coincidence, as the requests were slamming into our 10-second timeout limit. One in four requests failed entirely.

At 1,000 req/s, the framework was drowning.

After the fix, TanStack Start became the fastest framework in our benchmark. Response times dropped from seconds to milliseconds,the timeout cliff vanished, and every single request succeeded.

What makes this improvement even more notable is that it was runtime-agnostic. Both Watt and Node.js saw virtually identical gains: Watt improved from 3,228ms to 12.79ms average response time, while Node.js improved from 3,171ms to 13.73ms. This confirms that the bottleneck was purely in the framework’s code and that the fix benefited all users equally, regardless of their deployment strategy.

Runtime Comparison: Watt vs Node.js

Watt’s SO_REUSEPORT Advantage

Watt uses Linux kernel’s SO_REUSEPORT to let workers accept connections directly:

1. Kernel distributes the connection to the worker.

2. The worker processes the request.

No master coordination, no IPC overhead. The kernel handles load distribution efficiently.

When Does Watt Help Most?

Framework Rankings

With Watt Runtime

With Node.js Runtime

Reproducing These Benchmarks

The complete benchmark infrastructure is available at:

https://github.com/platformatic/k8s-watt-performance-demo/tree/ecommerce

To run the benchmarks:

# Benchmark TanStack Start
AWS_PROFILE=<profile-name> FRAMEWORK=tanstack ./benchmark.sh

# Benchmark React Router
AWS_PROFILE=<profile-name> FRAMEWORK=react-router ./benchmark.sh

# Benchmark Next.js
AWS_PROFILE=<profile-name> FRAMEWORK=next ./benchmark.sh

# Benchmark all frameworks
AWS_PROFILE=<profile-name> ./benchmark-all.sh

The script creates an ephemeral EKS cluster, deploys all three runtime configurations (Node, PM2, Watt), executes the load tests, and tears down the infrastructure automatically. The results for PM2 were omitted from the blog post because they align with previously reported findings (read 93% Faster Next.js in (your) Kubernetes).

The script creates an ephemeral EKS cluster, deploys all three runtime configurations (Node, PM2, Watt), executes the load tests, and tears down the infrastructure automatically. The results for PM2 were omitted from the blog post because they align with previously reported findings (read93% Faster Next.js in (your) Kubernetes).

Key Takeaways

1. Watt Provides Consistent Improvements
   Watt improved performance for all frameworks compared to standalone Node.js. The gains ranged from 7% for TanStack to 38% for React Router. It’s a low-risk optimization that helps in every case.

2. TanStack Start is Production-Ready
   Despite being the newest framework, TanStack Start delivered the best performance. The team’s rapid response to performance issues (a 252x improvement across 7 versions) demonstrates an active focus on development and optimization.

3. Keep Dependencies Updated
   The results from TanStack and Next.js both show how important it is to keep your dependencies up to date. TanStack improved from 75% to 100% success in 7 versions. Next.js doubled its throughput between v15 and v16 canary. You only get these performance improvements if you update.

4. Framework Choice Matters More Than Runtime
   The difference between TanStack Start and Next.js (3x throughput, 690x latency difference) far exceeds the difference between Watt and Node.js on the same framework. Choose your framework wisely.

5. Next.js Needs Caching
   At 1,000 req/s, Next.js struggled. For high-volume SSR workloads, users should consider adopting aggressive cache strategies (ISR, edge caching, component caching). Next.js has great primitives for these, and you can use them in Watt. We did not implement any caching solution for Next.js because, in most e-commerce (or enterprise) scenarios, caching is a no-go: companies want to implement aggressive personalization strategies and A/B testing, running thousands of experiments in parallel. That said, the jump from v15 to v16 Canary shows meaningful improvement, and if this trajectory continues, the gap will keep closing.

If you want performance to be a key part of your technology choices, try setting clear latency budgets for each route before you start building or picking a framework. Setting concrete performance goals early helps guide decisions about architecture and tools, and makes sure your stack meets real-world needs. Planning for latency by route can also show when caching, framework choice, or runtime tweaks will have the biggest impact on user experience.

Conclusion

These benchmarks show there are big performance differences between SSR frameworks when running the same app under load:

• TanStack Start emerged as the performance leader, handling 1,000 req/s with 13ms average latency.

• React Router delivered reliable performance with zero failures.

• Next.js struggled at this load, but improved a lot after upgrading to v16 canary. Throughput doubled and latency dropped by six times.

Beyond the numbers, this project showed that you can’t fix what you can’t see. We use platformatic/flame for our own internal performance testing, and sharing benchmark data with framework teams led to real improvements. The TanStack team’s 252x improvement in 7 versions, and the Next.js team’s work that led to a 75% speedup in React’s RSC deserialization, both show that open performance data helps the whole ecosystem, not just one framework or project.

For teams choosing an SSR framework, these results suggest:

• High-throughput requirements: Consider TanStack Start or React Router

• If you have an existing Next.js project, upgrade to the latest version for major performance gains. Use Watt to get the best throughput.

• Runtime optimization: Watt provides consistent improvements across all frameworks

We’re actively looking to speak with web performance teams at the moment. If that’s you, please send me a DM on LinkedIn, Twitter, hello@platformatic.dev.

You can’t import or require() a module that only exists in memory. You can’t bundle assets into a single executable without patching half the standard library. You can’t sandbox file access for a tenant without reinventing fs from scratch.

That changes now. We’re announcing@platformatic/vfs, a userland Virtual File System for Node.js, and the upstream node:vfs module landing in Node.js core.

The problem

Here’s what it looks like in practice when Node.js doesn’t have a VFS:

1. Bundle a full application into a Single Executable. You need to ship configuration files, templates, and static assets alongside your code. This often means bolting on 20 to 40 MB of extra boilerplate just to handle asset access at runtime. Node.js SEAs can embed a single blob, but your application code still calls fs.readFileSync() expecting real paths, so you end up duplicating files or injecting glue code that bloats your binary.

2. Run tests without touching the disk. You want an isolated, in-memory filesystem so tests don’t leave artifacts and don’t collide in CI. Today, you mock fs with tools like memfs, but those mocks don’t integrate with import or require().

3. Sandbox a tenant’s file access. In a multi-tenant platform, you need to confine each tenant to a directory without them escaping via ../. You end up writing path validation logic that’s fragile and easy to get wrong.

4. Load code generated at runtime. AI agents, plugin systems, and code generation pipelines produce JavaScript that needs to be imported. Today, that means writing to a temp file and hoping cleanup happens.

All four require the same primitive: a virtual filesystem that hooks into node:fs and Node.js module loading. The ecosystem has built approximations like memfs, unionfs, mock-fs, but they all share the same limitation: they patch fs but not the module resolver. Code that calls import('./config.json') bypasses them entirely.

The original issue requesting VFS hooks for SEAs, opened by Daniel Lando, captured this well. The FS hooks proposal from the Single Executable working group documented years of requirements. People knew what they wanted. Nobody had built it yet.

node:vfs in Node.js core

I started working on a VFS implementation over Christmas 2025. What began as a holiday experiment became PR #61478: a node:vfs module for Node.js, with almost 14,000 lines of code across 66 files.

Let me be honest: a PR that size would normally take months of full-time work. This one happened because I built it with Claude Code. I pointed the AI at the tedious parts, the stuff that makes a 14k-line PR possible but no human wants to hand-write: implementing every fs method variant (sync, callback, promises), wiring up test coverage, and generating docs. I focused on the architecture, the API design, and reviewing every line. Without AI, this would not have been a holiday side project. It just wouldn’t have happened.

Here’s what it looks like:

import vfs from 'node:vfs'
import fs from 'node:fs'

const myVfs = vfs.create()

myVfs.mkdirSync('/app')
myVfs.writeFileSync('/app/config.json', '{"debug": true}')
myVfs.writeFileSync('/app/module.mjs', 'export default "hello from VFS"')

myVfs.mount('/virtual')

// Standard fs works
const config = JSON.parse(fs.readFileSync('/virtual/app/config.json', 'utf8'))

// import works, and so does require()
const mod = await import('/virtual/app/module.mjs')
console.log(mod.default) // "hello from VFS"

myVfs.unmount()

This is not a mock. When you call myVfs.mount('/virtual'), the VFS hooks into the actual fs module and the module resolver. Any code in the process, yours or your dependencies, that reads from paths under /virtual gets content from the VFS. Third-party libraries don’t need to know about it. express.static('/virtual/public') just works.

How it’s structured

The VFS has a provider layer and a mount layer.

Providers are the storage backends. MemoryProvider is the default: in-memory, fast, gone when the process exits. SEAProvider gives read-only access to assets embedded in Single Executable Applications. VirtualProvider is a base class you can extend for custom backends (database, network, whatever you need).

Mounting is how the VFS becomes visible to the rest of the process. myVfs.mount('/virtual') makes VFS content accessible under that path prefix. The process object emits vfs-mount and vfs-unmount events so you can track what’s going on:

process.on('vfs-mount', (info) => {
 console.log(`VFS mounted at \({info.mountPoint}, overlay: \){info.overlay}, readonly: ${info.readonly}`)
})

There’s also an overlay mode for when you want to intercept specific files without hiding the real filesystem:

const myVfs = vfs.create({ overlay: true })
myVfs.writeFileSync('/etc/config.json', '{"mocked": true}')
myVfs.mount('/')

// /etc/config.json comes from VFS
// /etc/hostname comes from the real filesystem

Only the paths that exist in the VFS are intercepted. Everything else goes to the real filesystem. For testing, this is ideal: you can override a few files and leave the rest untouched.

The fs API

The VFS isn’t a subset of fs. It covers synchronous, callback, and promise-based APIs for reading, writing, directories, symlinks, file descriptors, streams, watching, and glob. VirtualStats matches fs.Stats. Error codes match what Node.js returns (ENOENT, ENOTDIR, EISDIR, EEXIST). Code that works with the real filesystem should work with the VFS.

Why VFS needs to live in core Node.js

@platformatic/vfs proves the API works, but it also proves why a userland implementation will always be a compromise. Here’s what you run into when you try to build this outside of Node.js:

Module resolution is duplicated. The userland package contains 960+ lines of module resolution logic: walking node_modules trees, parsing package.json exports fields, trying index files, and resolving conditional exports. All of this already exists inside Node.js.

In core, the VFS hooks directly into the existing resolver. In userland, we re-implement it and hope we got every edge case right.

Private APIs. On Node.js versions before 23.5, there’s no public API to hook module resolution. The userland package patches Module._resolveFilename and Module._extensions, both private internals with no stability guarantees. A Node.js minor release could break them.

In core, the VFS is part of the resolver, not a patch on top of it.

Global fs patching is fragile. The userland package replaces fs.readFileSync, fs.statSync, and other core functions. If any code captures a reference to fs.readFileSync before the VFS mounts, that reference bypasses the VFS entirely.

In core, the interception happens below the public API surface, so captured references still work.

Native modules don’t work. dlopen() needs a real file path.

A userland VFS can’t teach the native module loader to read .node files from memory. Core can.

Module cache cleanup is impossible. When you unmount a VFS, modules that were require()'d from it stay in require.cache.

The userland package has no way to distinguish VFS-loaded modules from real ones, so it can’t clean them up. Core can track which modules came from which VFS and invalidate them on unmount.

None of these issues are bugs in the userland package. They’re just fundamental limits of what’s possible outside the runtime. The userland package is a bridge. Use it now, and switch to node:vfs when it becomes available.

Where the PR stands

The PR is open and in active review. The feature will be released as experimental.

Joyee Cheung from Igalia has been the most thorough reviewer. She pushed hard on the security model around mount(), flagged that internalModuleStat shouldn’t be exposed as public API, and pointed to the VFS requirements document that the Single Executable working group collected over four years. Her feedback made the implementation significantly better.

James Snell and Paolo Insogna approved the PR. Stephen Belanger raised important questions about the security implications of global mount() hijacking and suggested integrating with the permission model. Ethan Arrowood did a thorough review of the docs and tests. Aviv Keller caught places where code could be simplified with node:path. Richard Lau and Tierney Cyren provided feedback on documentation structure.

Thanks to everyone involved. Reviewing a 14,000-line PR is a big job, and they all put in the effort.

@platformatic/vfs: use it today

We didn’t want to wait for the core PR to be merged.

When Malte Ubl, CTO of Vercel, saw the PR, he tweeted:

“ I saw @matteocollina Virtual File System PR for Node.js, and I’m super excited about it! And so I was wondering if it could be back-ported in user-land. Looks pretty good. May publish it to npm”

We had the same idea, and so did the Vercel team, who published node-vfs-polyfill. When two teams independently extract the same API into userland, it’s a good sign that the design is solid.

Our version is@platformatic/vfs, and it works on Node.js 22 and above.

npm install @platformatic/vfs

The API matches what’s proposed for node:vfs:

import { create, MemoryProvider, SqliteProvider, RealFSProvider } from '@platformatic/vfs'

const vfs = create()
vfs.writeFileSync('/index.mjs', 'export const version = "1.0.0"')
vfs.mount('/app')

const mod = await import('/app/index.mjs')
console.log(mod.version) // "1.0.0"

When node:vfs ships in core, migrating is a one-line change: swap '@platformatic/vfs' for 'node:vfs' in your import.

Extra providers

The userland package ships two providers that aren’t in the core PR. SqliteProvider gives you a persistent VFS backed by node:sqlite. Files survive process restarts:

import { create, SqliteProvider } from '@platformatic/vfs'

const disk = new SqliteProvider('/tmp/myfs.db')
const vfs = create(disk)

vfs.writeFileSync('/config.json', '{"saved": true}')
disk.close()

// Later, in another process:
const disk2 = new SqliteProvider('/tmp/myfs.db')
const vfs2 = create(disk2)
console.log(vfs2.readFileSync('/config.json', 'utf8')) // '{"saved": true}'

This is helpful for caching compiled assets or keeping generated code across deployments.

RealFSProvider is sandboxed real filesystem access. It maps VFS paths to a real directory and prevents path traversal:

import { create, RealFSProvider } from '@platformatic/vfs'

const provider = new RealFSProvider('/tmp/sandbox')
const vfs = create(provider)

vfs.writeFileSync('/file.txt', 'sandboxed') // Writes to /tmp/sandbox/file.txt
vfs.readFileSync('/../../../etc/passwd') // Throws, can't escape the sandbox

Use cases

Single Executable Applications

Node.js SEAs can embed assets, but accessing them has always been tricky. With VFS, SEA assets are automatically mounted and can be accessed through standard fs calls, import, and require(). Your application code doesn’t need to know it’s running as an SEA.

Testing

You can create an isolated filesystem per test. No temp directories to clean up, no collisions between parallel test runs:

import { create } from '@platformatic/vfs'
import { test } from 'node:test'

test('reads config from virtual filesystem', () => {
 using vfs = create()
 vfs.writeFileSync('/config.json', '{"env": "test"}')
 vfs.mount('/app')

 // Your application code reads /app/config.json through standard fs
 // No disk I/O, no cleanup needed
 // The `using` statement automatically unmounts when the block exits
})

AI agents and code generation

AI agents generate code that needs to run. Writing to temp files is slow, creates cleanup problems, and increases security risks. With VFS, generated code stays in memory and can be loaded with import:

import { create } from '@platformatic/vfs'

const vfs = create()
vfs.writeFileSync('/handler.mjs', agentGeneratedCode)
vfs.mount('/generated')

const { default: handler } = await import('/generated/handler.mjs')
await handler(request)

What’s next

Both node:vfs and @platformatic/vfs are experimental. The test coverage is solid, but a virtual filesystem that hooks into module loading and node:fs has a huge surface area. There will be bugs. Edge cases we haven’t hit. Interactions with third-party code we didn’t anticipate.

If you hit something, please report it. For the userland package, open an issue on platformatic/vfs. For the core module, comment on the PR or open an issue on nodejs/node. Every bug report helps.

Once node:vfs lands in core, we’ll keep @platformatic/vfs in sync with any API changes and eventually deprecate it in favour of the built-in module.

In the meantime, try it out and let us know what you build.

node:vfs PR by Matteo Collina.

Fixes issue #60021 by Daniel Lando.

@platformatic/vfs is now on npm.

That’s why teams often notice the same pattern during launches and campaigns: /_next/image traffic increases, CPU usage maxes out, render times get longer, and the whole frontend slows down even though the app logic hasn’t changed. In short, image optimization starts to interfere with your most important user flows.

Watt is our open-source Node.js application server that orchestrates frontend frameworks (Next.js, Astro, Remix) and backend services (Node.js, Fastify, Express, Hono, etc) into a single system, with built-in logging, tracing, and multithreading. It leverages the Linux kernel's SO_REUSEPORT to distribute connections across workers with zero coordination overhead. In our production benchmarks on AWS EKS, Watt delivered 93.6% faster median latency and a 99.8% success rate under a sustained load of 1,000 requests per second. After investigating component rendering, it was only a question of time before we looked into images.

By moving image optimization into its own Watt Application, you create a clear microservice boundary. The optimizer becomes a focused service in your setup, with an API that only exposes what’s needed for safe and efficient image delivery. This keeps media processing separate from your main frontend. You can then scale image capacity on its own, let rendering workers focus on rendering, and adjust retries, timeouts, and storage for media processing without having to over-provision your whole frontend.

@platformatic/next is the official Platformatic package for running Next.js inside a Watt Application. It’s fully maintained and supported by the Platformatic team, so you get long-term compatibility with Next.js updates, regular security patches, and best-practice defaults for production. Teams can count on ongoing updates and quick fixes, which lowers maintenance risk and avoids the downsides of custom or community-maintained solutions. The package now includes an Image Optimizer mode, letting you run /_next/image as a dedicated Watt Application, scale it separately, and keep your frontend fast even when image traffic increases.

This capability was introduced in PR #4605, and it builds on top of @platformatic/image-optimizer, our dedicated optimization engine. Our image optimizer is built on top of sharp, leveraging @platformatic/job-queue, which adds flexible storage, job deduplication with caching, and producer/consumer decoupling.

If you are self-hosting Next.js and want the same kind of operational separation that mature platforms use internally, this is the missing building block.

In short, you can keep using Next.js as you always have, but with a cleaner architecture that handles high traffic more efficiently

Why split image optimization from your frontend?

If your frontend handles page rendering, API routes, and image resizing as a single service, any slowdown in one will cascade to the others. This means that when traffic is highest, like during product launches, campaigns, or social media spikes, this architecture causes performance to suffer the most

And it goes without saying (although it’s a blog, so yes, we will say it anyway…) that page performance isn’t just a technical issue - even a 100 ms delay can lower conversion rates by up to 7%, making slowdowns expensive during launches and campaigns.

The reason comes down to architecture: resizing and re-encoding images is bursty, CPU-heavy, and often I/O bound, while SSR and API routes usually need lower latency and more consistent resources. Running both in one service means you have to use the same autoscaling and resource pool for two very different types of work.

Splitting these responsibilities and running them as worker threads using Watt eliminates this ‘noisy neighbour’ effect and lets you apply the right scaling strategy to each path: scale optimizer replicas (or threads) when media demand rises, and keep frontend replicas sized for rendering throughput and tail latency.

Platformatic’s dedicated image optimizer, Watt Application, gives you:

• Independent scaling: add replicas for image workloads without scaling the whole frontend stack.

• Operational isolation: image spikes do not starve SSR/RSC rendering.

• Centralized controls: enforce width/quality validation, timeout, retry behaviour, and storage in one place.

• Flexible queue storage: choose memory, filesystem, or Redis/Valkey depending on your topology.

This setup is especially useful for platform engineering and SRE teams who need predictable performance without over-provisioning the whole frontend. Clear ownership lets these teams align this approach with their KPIs for reliability, scalability, and cost efficiency.

What shipped in Platformatic Next

The new next.imageOptimizer configuration lets you turn on optimizer-only mode in @platformatic/next, so you can run a Watt Application focused just on image processing. In other words: flip one flag and route only /_next/image, making adoption fast and low-friction.

When enabled, the service:

1. Exposes only the Next.js image endpoint (/_next/image, respecting base path).

2. Validates image parameters using Next.js rules.

3. Resolves relative URLs through a fallback target (URL or runtime service name).

4. Fetches and optimizes images through a queue-backed pipeline; if the same image is requested by multiple users at the same time, it would be processed only once.

5. Returns optimized image bytes and cache headers.

Under the hood, this relies on @platformatic/image-optimizer, which provides a robust processing pipeline with:

• image type detection from magic bytes

• optimization for jpeg, png, webp, and avif

• animation-aware safeguards

• URL fetch + optimize helpers

• queue APIs powered by @platformatic/job-queue

The queue can run as a distributed state on Redis/Valkey, so retries, workload distribution, and resilience remain consistent across multiple optimizer replicas.

The main idea is to keep frontend rendering and image optimization separate, while still using the usual Next.js image features.

What this means for teams

• Frontend teams keep using next/image as usual, without rewriting application code.

• Platform teams get explicit controls for retries, timeout budgets, and queue storage.

• Ops teams can scale optimizer replicas independently from the frontend tier.

• Product teams get a smoother user experience during peak traffic windows.

The result is a platform that feels (and… is) faster to end users and more controllable to engineering teams. In recent internal benchmarks, shifting image optimization to a dedicated Watt Application reduced 95th-percentile response times during peak traffic by up to 40%, turning previously unpredictable slowdowns into consistently fast delivery even under heavy load.

Choose the right runtime blueprint

The easiest setup is a three-application Watt setup:

• gateway: Watt’s gateway service, receive and routeincoming traffic.

• frontend: your standard Next.js application

• optimizer: @platformatic/next running in Image Optimizer mode

Watt’s Gateway sends only GET /_next/image requests to the optimizer, while everything else goes to the frontend. This gives you a clear separation without needing a complicated network setup.

For relative image URLs (for example /hero.jpg), the optimizer fetches originals from frontend via runtime service discovery (http://frontend.plt.local). For absolute URLs, it fetches upstream directly.

If you are deploying on Kubernetes, your best bet is to configure your K8s ingress controller to route GET /_next/image to separate pods running the image optimizer. This configuration is supported and documented at https://docs.platformatic.dev/docs/guides/next-image-optimizer#10-kubernetes-ingress-example-nginx-ingress-controller.

How to set this up

Start by creating a Watt workspace with three applications: Gateway, frontend, and optimizer. The frontend remains your existing Next.js app; the optimizer is another @platformatic/next app with next.imageOptimizer.enabled: true; Gateway routes image traffic to the optimizer and everything else to the frontend.

Use this structure as a baseline:

my-runtime/
 watt.json
 web/
   gateway/
     platformatic.json
   frontend/
     platformatic.json
     package.json
     next.config.js
     app/
   optimizer/
     next.config.js
     platformatic.json
     package.json

Then configure it in this order:

1. Enable image optimizer mode in the optimizer Watt Application.

2. Set optimizer.next.imageOptimizer.fallback to frontend so relative image URLs are fetched from http://frontend.plt.local.

3. In Gateway, route only GET /_next/image to optimizer and keep all other routes on frontend.

4. Pick queue storage for your topology:

   • memory for local/dev

   • filesystem for single-node persistent disk

   • Redis/Valkey for distributed replicas

5. Tune timeout and maxAttempts using your target SLO and expected image profile.

With this setup, app teams can keep using next/image as usual, while platform teams get independent scaling and more control over operations.

Configuration example

In your optimizer application config:

{
 "$schema": "https://schemas.platformatic.dev/@platformatic/next/3.38.1.json",
 "next": {
   "imageOptimizer": {
     "enabled": true,
     "fallback": "frontend",
     "timeout": 30000,
     "maxAttempts": 3,
     "storage": {
       "type": "valkey",
       "url": "redis://localhost:6379",
       "prefix": "next-image:"
     }
   }
 }
}

And in your Gateway config, route only the image endpoint:

{
 "$schema": "https://schemas.platformatic.dev/@platformatic/gateway/3.0.0.json",
 "gateway": {
   "applications": [
     {
       "id": "frontend",
       "proxy": {
         "prefix": "/",
         "routes": ["/*"]
       }
     },
     {
       "id": "optimizer",
       "proxy": {
         "prefix": "/",
         "routes": ["/_next/image"],
         "methods": ["GET"]
       }
     }
   ]
 }
}

Storage choices: what to use and when

• memory: local development or simple single-instance setups.

• filesystem: single-node deployment with persistent disk.

• redis/valkey: distributed production environments with shared queue state.

If you do not specify storage, memory is used by default.

For production multi-instance deployments, Redis/Valkey is usually the best default because it gives shared queue state and predictable behaviour across replicas.

Failure handling and reliability

Optimization runs through a queue with explicit timeout and retry controls:

• timeout sets the fetch/optimization budget per job.

• maxAttempts controls the automatic retry count.

When retries are exhausted, the service returns a 502 Bad Gateway response, keeping failure behaviour explicit, observable, and easier to alert on.

Try it today

If you are self-hosting Next.js and want predictable image performance under load, this capability gives you a practical path that does not require re-architecting your app:

1. keep your frontend app unchanged,

2. stand up a dedicated optimizer Watt Application,

3. route only /_next/image through Watt’s Gateway service,

4. pick the storage backend that matches your deployment model.

This is a small architectural change with a big benefit: better frontend stability, simpler operations, and image performance that scales when you need it.

If you want to deliver faster and more reliable user experiences as your traffic grows, dedicated image optimization is one of the best upgrades you can make with minimal disruption.

Read more:

• PR #4605: Added image optimizer capability

• Run Next.js Image Optimizer as a Dedicated Service

• Next.js Image Optimizer reference in Platformatic docs

• @platformatic/image-optimizer

You can think of this as akin to Vercel’s Skew Protection functionality, but running right in your existing Kubernetes setup: no migration or changes to your CI/CD pipeline or security policies needed, just out-of-the-box version pinning for your frontend applications.

The Problem: Version Skew in Kubernetes

When you update a web application, users who loaded the old frontend might send requests to the new backend. This is called “version skew,” and it can cause problems if APIs, assets, or data schemas have changed. For example, if you rename a form field, old clients might still send data using the old field name.

This problem matters even more for modern frontend apps, where the same codebase runs on both the client and server. Frameworks like Next.js, Remix, and monorepos often share TypeScript types, API definitions, or business logic between frontend and backend. If these shared parts change between versions, it can cause serious issues:

• Hydration Errors and Broken UI: React Server Components tightly couples client and server in a single deployment; when a new version goes live, the server produces updated RSC payloads that older client bundles still in users' browsers cannot reconcile, causing hydration errors and broken UI

• API contract violations: OpenAPI or protobuf definitions change between versions, leading to serialization/deserialization failures

• Type discrepancies: Shared TypeScript interfaces or zod schemas break when frontend and backend versions diverge, causing runtime errors.

• Codependent features: Frontend components that rely on backend-specific functionality fail when that functionality changes or is removed

The implications for your users are fairly straightforward: some might see API errors, missing fields, or broken features if their client and server versions don’t match; others might see data loss or corruption when schemas change across app versions. All this ultimately puts a load on support teams, who often need to coordinate across multiple feature teams to effectively untangle and ultimately resolve these issues.

Outside of the obvious impact on users (and revenue), k8s version skew is another example of how distributed systems, if not operated with the proper guardrails, actually impede developer velocity. In a world that is increasingly reliant on using AI to write code, the bottleneck is no longer the ability to write lines of code (if it ever was), but what happens between when your code is written and when it actually gets to production.

Version Skew in Kubernetes is a perfect example of such a problem - you have teams that are capable of shipping much faster, but without the right guardrails, the entire system actually moves slower and fails more often: fear of committing breaking changes leads to larger, less-frequent deployments that carry more risk and slow down your time-to-market.

The Solution: ICC Skew Protection

Platformatic’s new skew protection feature, built into the Intelligent Command Center, makes sure users stay on the version they started their session with, even when new versions are deployed. If a user starts a session on version N, all their requests during that session go to version N.

How It Works

Skew protection uses the Kubernetes Gateway API for version-aware routing, with ICC acting as the control plane. Each application version runs as a separate, immutable Kubernetes Deployment that users create themselves using standard Kubernetes workflows.

When applications run, ICC automatically detects new versions via label-based discovery and manages routing rules. ICC creates and maintains HTTPRoute resources that route requests based on session cookies, using a  __plt_dpl  cookie to pinusers to their deployment version.

When a new version is deployed, the previous version transitions to “draining” mode: existing sessions continue to work, while new sessions go to the active version. ICC monitors traffic activity and automatically cleans up old versions after configured grace periods.

Key Platformatic Components

Platformatic Watt is the Node.js application server that runs your application as a worker thread inside of Kubernetes . This allows for improved performance, resiliency, and compute efficiency, as well as providing out-of-the-box features such as hot reloading, health checks, and metrics collection.

watt-extra is an extension layer that sits on top of Platformatic Watt and serves as the bridge between your application and ICC. On startup, watt-extra connects to ICC and registers the application with its metadata (pod ID, app name, version). This registration enables ICC to:

• Discover the application’s Kubernetes labels (app.kubernetes.io/name, plt.dev/version)

• Manage autoscaling using real-time, Node.js-specific metrics

• Implement version-aware routing for skew protection

• Monitor health and performance,

System Architecture

The skew protection system consists of four layers. Each application version is a completely separate K8s Deployment, and the Kubernetes Gateway API handles routing at the ingress level based on HTTPRoute rules managed by ICC.

Component Breakdown

Client Layer

• Browser Session A (cookie: __plt_dpl=dep-v42): A user who started their session on version 42. The __plt_dpl cookie pins their requests to that version, making sure the requests are routed to the correct backend even after newer versions are deployed.

• Browser Session B (cookie: __plt_dpl=dep-v43): A user who started their session on version 43. Their requests are routed to the active version based on their cookie.

• New Visitor (no deployment cookie): A first-time user or someone without a version cookie. Their first request is routed to the current active version, and they receive a cookie that pins them to that version.

Gateway API Layer

• GatewayClass: Defines a template or class of gateways (e.g., Envoy Gateway, Contour, or Cilium) that can process Gateway API resources. Each cluster operator configures this with their preferred controller.

• Gateway Resource: The actual gateway instance that listens on HTTP/HTTPS ports and processes incoming traffic. It contains listener configurations for TLS termination and routing.

• HTTPRoute: Managed by ICC, this is the key routing rule that implements version-aware routing. It contains multiple rules: cookie-based matches for draining versions and a default rule that sets a cookie for new visitors and routes to the active version.

ICC (Intelligent Command Center) - Namespace: platformatic

• Control Plane Service: The core component responsible for version detection, HTTPRoute management, and lifecycle decisions. When watt-extra registers a new pod, the control plane discovers the application name and version. It holds the version registry and creates/updates/deletes HTTPRoute resources as needed.

• PostgreSQL: Stores the persistent state for skew protection, including the version registry with full metadata about each deployment (version string, timestamps, K8s resources), deployment history for audit trails, and per-application skew protection policies.

App Versions - Namespace: myapp

• Deployment: myapp-v42 (draining): A Kubernetes Deployment for the previous version (42) that is being phased out. It has its own Service and pods running Watt with watt-extra. Traffic only routes here for users whose cookies match this version.

• Deployment: myapp-v43 (active): The current active version deployment. It has multiple replicas for high availability. New visitors and users without matching cookies are routed here. ICC’s autoscaler works across all deployed versions, provisioning the correct amount of resources for each version based on actual traffic.

• Service: Each version has its own Kubernetes Service that selects pods with the corresponding plt.dev/version label. These Services are referenced by the HTTPRoute’s backendRefs.

• Pods (Watt + watt-extra): Each pod runs the application container (Platformatic Watt runtime) plus watt-extra. watt-extra is the ICC agent that connects to ICC on startup and registers the pod. It sends the pod ID, and ICC discovers the version and deployment metadata through Kubernetes APIs. watt-extra also reports metrics to ICC for autoscaling and health monitoring.

Observability Layer

Prometheus: Collects metrics from all pods and services. ICC queries Prometheus to monitor traffic patterns for each version, track request rates for draining versions, and uses that data to determine when versions should be transitioned to Expired status (meaning services that received no traffic for the pre-configured grace period).

How It All Works Together

When a new application version is deployed:

1. You deploy a new version of your app with the same app.kubernetes.io/name label and a new plt.dev/version label.

2. watt-extra registers the new pods with ICC, which detects the new version from the labels.

3. ICC makes the new version Active and moves the previous one to Draining. It updates the Gateway routing rules so that new sessions go to the active version, while existing sessions with a version cookie keep going to the draining version.

4. ICC monitors traffic on draining versions. Once there is no traffic, or the grace period elapses, ICC expires the old version — removing its routing rules and scaling it to zero, and optionally deleting the old Deployment and Service.

The Deployment Lifecycle in Detail

When managing multiple versions, skew protection uses a well-defined state machine to guarantee flawless transitions:

• Active → The current version serving new sessions. Exactly one version per application is Active at a time. The HTTPRoute’s default rule points to the Active version’s Service, and new visitors receive a cookie pinning them to this version.

• Draining → When a newer version is detected and becomes Active, the previous version transitions to Draining. No new sessions are assigned to it, but existing sessions with version-pinning cookies continue to be served. ICC monitors traffic activity for draining versions to determine when they can be safely retired.

• Expired → A version transitions to Expired when it has zero traffic over the traffic window (default: 30 minutes) or when the grace period elapses (default: 24 hours), whichever comes first. ICC then removes the version’s matching rules from the HTTPRoute, scales the Deployment to zero replicas via the autoscaler, and optionally deletes the Deployment and Service (if auto-cleanup is enabled).

The ICC uses Version Labels to determine state. Version labels are opaque strings andcan be numbers, semver, git SHAs, or any identifier that fits your workflow. ICC does not parse or compare them; it just treats the most recently detected version as Active.

How users deploy a new version:

1. Build a new container image with the updated application code (e.g., myapp:v43)

2. Create a new K8s Deployment and Service with:

   • Same app.kubernetes.io/name label (e.g., myapp) — this tells ICC it’s the same application

   • New plt.dev/version label (e.g., 43) — this tells ICC it’s a new version

   • New Deployment name (e.g., myapp-v43) and matching Service name

3. Apply the manifest: kubectl apply -f myapp-v43.yaml

4. ICC automatically detects the new version when pods start and watt-extra registers with ICC. The new version becomes Active, and the previous version begins draining.

Getting Started with ICC

Platformatic’s skew protection is built into the Intelligent Command Center (ICC), a complete control plane for managing Node.js applications or agents running in Kubernetes,with autoscaling, monitoring, and version-aware routing.

To get started with ICC:

• Install ICC on your Kubernetes cluster. Follow our Installation Guide for step-by-step instructions, covering infrastructure requirements (Kubernetes, PostgreSQL, Valkey, Prometheus) and installation options.

• Deploy your first application using the standard ICC workflow:

  • Add @platformatic/watt-extra to your app

  • Set PLT_ICC_URL so your app can register with ICC

  • Deploy with kubectl apply or your existing CI/CD pipeline

• Enable Skew Protection:

  • Enable PLT_FEATURE_SKEW_PROTECTION

  • Ensure Gateway API CRDs are installed (Kubernetes 1.27+)

  • Deploy a Gateway API-compatible controller (Envoy Gateway, Contour, Cilium, Traefik, NGINX Gateway Fabric or Kong). See the Compatible Gateways in ICC documentation

  • Configure deployment labels:

labels:
  app.kubernetes.io/name: myapp
   plt.dev/version: "43"
   # Optional: custom path prefix (default: /myapp)
   # plt.dev/path: "/api/leads"
   # Optional: hostname for HTTPRoute
   # plt.dev/hostname: "myapp.example.com"

Bring Vercel-Grade Deployment Safety to Your Kubernetes Environment

Platformatic’s skew protection is now available in ICC. It provides zero-downtime deployments and version-aware routing that keep each user session consistent.

If your team wants to try it in a real enterprise setup, send a message to Luca Maraschi or Matteo Collina via DMs on LinkedIn, or contact info@platformatic.dev.

That is exactly what ai-gateway-auditable delivers: an OpenAI-compatible gateway built with Platformatic Watt that combines provider routing, fallback resiliency, and durable audit logging to S3.

For production teams, this translates directly into risk reduction and regulatory readiness: your audit trail is always preserved, and resilient routing keeps incidents contained. In real terms, this leads to fewer lost logs or broken provider integrations (and fewer 3 a.m. pages as a result), and reliable evidence when you need to answer compliance or security reviews.

This architecture is not only production-ready, but already operating a scale for one of our early adopters. One application (proxy) serves traffic, while another (audit worker) persists audits, and a durable queue between them keeps latency low while preserving records, using the filesystem to provide durability. This same early-adopter halved its application latency using this pattern with Watt. With clear audit trails and resilient traffic handling, they were able to trace errors quickly and keep their on-call load under control, while giving their LLM-enabled end-users performance that approached parity with direct API calls, which was critical for serving their real-time use cases.

Source code: github.com/platformatic/ai-gateway-auditable

Why this matters now

The direct integration pattern is usually the first-stop for teams, but often leads to audit-trace gaps. Finance needs clean attribution by key or team, security needs auditable traces of model interactions, and product needs stronger uptime when upstream providers degrade.

As a real-world example, our same early adopter saw this with their initial production rollout, which missed up to 15% of request logs during peak volume, and causing request latency to spike by more than 2x when provider response times flared. At the same time, you want a single, stable integration surface instead of scattering provider-specific logic across multiple services. An AI gateway is where all your needs converge into a single, manageable control point.

With ai-gateway-auditable, every request has a clear path, every response is traceable, and fallback behavior is visible instead of opaque.

Why Watt

Platformatic Watt is well-suited to this pattern because it lets us run the API-facing proxy and the audit worker as separate applications with a shared operational model, using them as worker threads. That separation is the foundation of reliability here: the proxy can stay focused on low-latency responses, while the worker can focus on durable queue consumption, batching, and S3 shipping.

Most importantly, this design is tolerant of worker crashes. Watt supervises applications (worker threads), so if an audit worker crashes, it is automatically restarted, and unhealthy workers are automatically replaced. During that window, the proxy can keep accepting requests and persisting audit jobs in FileStorage. When the replacement worker is up, it resumes consuming from the same queue path and drains pending jobs.

The result is graceful degradation rather than data loss: temporary worker failures increase audit lag but do not break the request path or discard audit events. This distinction is critical from a business perspective. Losing audit data can put regulatory compliance at risk and expose the company to possible fines or a loss of trust, while a short delay in audit processing only postpones analysis or reporting. In other words, our design trades brief insight delays for the certainty that no evidence is lost.

Why filesystem-based storage

We use filesystem-backed queue storage on purpose. Writing audit jobs to local disk is crash-tolerant because queued data survives process failures and restarts, unlike in-memory buffers.

It also keeps resource usage and request-path performance under control. We do not need to retain full audit payloads in memory awaiting for remote writes, and we do not put every request on the critical path of an external storage service. That removes network latency and remote availability as immediate blockers to request handling, while still providing durable buffering before batches are shipped to S3.

Architecture at a glance

The system runs as two applications (threads) inside of Platformatic Watt, the Node.js application server.

The proxy is optimized for low-latency request/response flow, while the audit-worker is optimized for durability, retries, and batch shipping. Keeping these concerns separate avoids a common failure mode: heavy audit I/O slowing down user-facing traffic.

How do the two applications communicate? Through the same FileStorage queue path on disk. proxy writes audit jobs to ./data/queue at the same rate as local queue operations, and audit-worker consumes those jobs independently in the background. This gives you explicit producer/consumer decoupling: the request path does not wait for S3 uploads, retries, or batch rotation. If the worker restarts, queued jobs remain on disk and are resumed when it comes back. If S3 is slow or temporarily unavailable, jobs continue to accumulate durably in the queue instead of being lost or pushing latency back to callers.

In other words, even when storage is under pressure or S3 is temporarily unavailable, the gateway can keep serving requests while the audit pipeline catches up safely in the background.

What the gateway gives you

At a product level, this gateway provides four strong guarantees:

1. OpenAI Completions compatible endpoint (/v1/chat/completions) for clients and SDKs.

2. Model-based routing with fallback across providers.

3. Complete request/response audit records for every successful exchange.

4. Durable archival to S3 with batched JSONL files partitioned by time (JSON Lines is a text file format where each line is a valid, independent JSON object, separated by newline characters).

This means reduced provider lock-in, minimized operational risks, and heightened observability.

Service responsibilities

The key behavior is role decoupling: proxy only produces queue jobs, while audit-worker handles all downstream storage and shipping work.

proxy (external entrypoint)

proxy exposes:

• GET /health

• POST /v1/chat/completions

For each request, it:

1. Selects a provider chain based on model routing rules.

2. Executes upstream calls with fallback on retryable failures.

3. Returns the upstream response to the client.

4. Enqueues an audit payload into the shared durable queue.

audit-worker (internal service)

audit-worker is an internal Node application with no HTTP API (hasServer = false).

It owns the full audit persistence path:

• queue consumption with @platformatic/job-queue

• durable local buffering with FileStorage

• batched JSONL writing

• S3 uploads signed with AWS SigV4.

Queue settings used in the current implementation:

• concurrency: 1

• maxRetries: 3

• resultTTL: 60_000

• visibilityTimeout: 30_000

This is optimized for predictable sequential writes and safe retry semantics. Filesystem queue storage is chosen because it needs no external setup (no Redis/Valkey), making local development and single-node production rollouts much simpler. At the same time, it still provides crash resilience: queue state is persisted to disk, so in-flight and pending audit jobs survive process restarts.

That combination is the key trade-off here: you gain operational simplicity and zero external dependencies, without sacrificing durability for the audit trail. Note that adopting the file system exposes teams to the risk of data loss. Moving the auditability trail back to the main response cycle will introduce latency and cause a hard failure if the audit cannot be completed. The tradeoff, as always, is in the hands of engineers: availability or consistency?

Routing and fallback configuration

Routing lives in providers.json and uses two lists:

• providers: upstream connection and adapter definitions

• routing: per-model routing rules with ordered provider chains

{
 "providers": [
   {
     "id": "openai",
     "type": "openai",
     "baseUrl": "https://api.openai.com",
     "apiKey": "{OPENAI_API_KEY}"
   },
   {
     "id": "anthropic",
     "type": "anthropic",
     "baseUrl": "https://api.anthropic.com",
     "apiKey": "{ANTHROPIC_API_KEY}"
   }
 ],
 "routing": [
   {
     "id": "gpt-4o",
     "providers": ["openai"],
     "strategy": "fallback"
   },
   {
     "id": "claude-sonnet-4-6",
     "providers": ["anthropic"],
     "strategy": "fallback"
   },
   {
     "id": "*",
     "providers": ["openai"],
     "strategy": "fallback"
   }
 ]
}

Environment variables like {OPENAI_API_KEY} are resolved from process env at startup.

Fallback behavior is explicit and policy-driven: by exposing a clearly configurable list of retryable statuses, teams can align gateway failover with internal governance or incident playbooks. For example, you can tune which upstream failures (such as 429, 500, 502, 503, 504) trigger fallback based on your own risk, compliance, or incident response thresholds. This mapping between config and governance means compliance and security teams can review and pre-approve response handling in line with internal standards—a step that accelerates approval and audit-readiness.

• retryable statuses: 429, 500, 502, 503, 504

• Connection failures are retryable

• Non-retryable responses (400, 401, 403) are returned immediately.

If you want delegated provider orchestration, you can configure OpenRouter as an openai-type provider and route * traffic to it.

Adapter model: one external contract, many upstreams

The gateway keeps a single OpenAI-compatible API surface, while adapters normalize provider differences behind the scenes.

• OpenAI adapter supports OpenAI-compatible endpoints, including Azure/OpenRouter-compatible APIs.

• The anthropic adapter translates OpenAI chat requests and responses to Anthropic Messages API semantics.

This removes provider-specific branching logic from your application layer.

Streaming support with full audit fidelity

Streaming UX matters, so the proxy preserves token-by-token delivery.

For stream: true requests, the proxy:

1. Pipes SSE chunks to the client in real time.

2. Buffers chunks internally.

3. Reconstructs a complete Chat Completions response.

4. Emits a single audit record with streamed set to true.

Users get low-latency streaming, and operators still get complete records for replay and analysis.

Audit record shape

Each JSONL line is a complete record with request, response, latency, caller hash, status, and routing metadata:

{
 "id": "a8f3b2c1-...",
 "timestamp": "2026-03-03T11:44:00.000Z",
 "duration_ms": 1243,
 "request": {
   "model": "gpt-4o",
   "messages": [{ "role": "user", "content": "Hello" }]
 },
 "response": {
   "id": "chatcmpl-...",
   "choices": [{ "message": { "role": "assistant", "content": "Hi!" } }]
 },
 "upstream_status": 200,
 "caller": "7a3f2b1c",
 "streamed": false,
 "routing": {
   "model": "gpt-4o",
   "planned_providers": [{ "id": "openai", "status": 200, "duration_ms": 1200 }],
   "used_provider": "openai"
 }
}

The caller is an 8-character SHA-256 prefix of the bearer token value, so attribution is possible without storing raw API keys.

Durable audit pipeline in detail

Inside the request path, proxy enqueues each payload using the request ID as the job ID, which naturally supports deduplication when IDs repeat.

audit-worker consumes those jobs and writes them into local JSONL batches before upload.

The writer then:

1. Appends each record as one JSON line to a local batch file using flush semantics.

2. Rotates to a new batch when the size or time threshold is reached.

3. Uploads the batch file to S3 using undici and SigV4 headers.

4. Deletes local batch files only after successful upload.

Current thresholds:

• BATCH_SIZE = 100

• FLUSH_INTERVAL_MS = 5000

S3 object keys are hour-partitioned for downstream querying:

audits/2026/03/03/11/batch-1741003090000-3bb7....jsonl

This structure works well with tools like Athena and other data lake pipelines.

Operating under failure

The gateway is intentionally designed to degrade gracefully.

Typical architectural components here include the file-backed queue directory (such as ./data/queue), which serves as the communication bridge between the proxy and the audit-worker; single-node deployment support via Platformatic Watt's supervised applications; and a default S3 bucket for audit archives. Core configuration files like providers.json define routing logic and provider chains, while runtime environment variables control credentials and logging. All of these components work together as the durable, fault-tolerant foundation that keeps this architecture reliable at scale. This keeps user-facing availability high while preserving eventual audit consistency.

Run it locally

git clone https://github.com/platformatic/ai-gateway-auditable.git
cd ai-gateway-auditable
npx wattpm-utils install
docker compose up

Then call the gateway with any OpenAI-compatible client or a simple curl:

curl http://localhost:3042/v1/chat/completions \
 -H 'Content-Type: application/json' \
 -H 'Authorization: Bearer sk-your-key' \
 -d '{
   "model": "gpt-4o",
   "messages": [{"role": "user", "content": "Hello"}]
 }'

Final take

ai-gateway-auditable is a practical pattern for teams that need to move fast with AI and still satisfy the operational norms of production software. It gives you:

• one consistent API surface with clear fallback behavior,

• complete and queryable audit trails, and a clean separation between serving traffic and persisting evidence.

If your roadmap includes multi-provider AI, compliance requirements, or strict SRE expectations, this architecture is ready to adopt and extend.

The easiest way to get started is to fork the repo, run the quick-start commands, and see the gateway in action with your own test requests. Try spinning up the service locally and sending a sample call: this practical step will show you right away how auditable AI operations can be within your own workflow.

Happy building!

@platformatic/job-queue is a new queue library from Platformatic focused on reliability and operational simplicity. This library is built on a workflow that lets you enqueue jobs and wait for results when needed, making background processing feel just as smooth as calling a function. Alongside this, it provides Node.js teams with a modern API that includes built-in caching, deduplication, retries, and pluggable storage.

In practice, this means you can start with a tiny local setup and then move to a distributed, production-grade deployment without rewriting your application code.

What makes it different

Most queue setups force you to stitch together multiple patterns and handle edge cases yourself. @platformatic/job-queue includes those patterns out of the box:

• Deduplication by job id so repeated enqueue attempts do not create duplicate work.

• Request/response support with enqueueAndWait() when you need async processing but still want a result.

• Reliable retries with configurable attempts and backoff behavior.

• Stalled job recovery via a Reaper that requeues jobs from crashed workers.

• Graceful shutdown ensures in-flight jobs complete before the service stops, reducing lost work during deploys and restarts.

• Move fast with safety: The API is TypeScript-native with typed payloads and results, so you catch errors at compile time and move confidently.

This makes it appropriate for both classic fire-and-forget workloads and RPC-style workloads that require a response. You do not have to pick one model globally: many teams use both in the same system, depending on endpoint and latency requirements. For example, in use cases such as sending emails and notifications, fire-and-forget jobs make sense because results are often not needed immediately and occasional retries can be handled gracefully. On the other hand, workflows such as generating invoices or processing payments may require the caller to wait for a result, making the request/response pattern with enqueueAndWait() a better fit.

A quick look at the API

You can use the queue as a producer and consumer in the same process, or split them across services. The API is intentionally small, so the same primitives are easy to apply in monoliths, microservices, and worker pools.

import { Queue, MemoryStorage } from '@platformatic/job-queue'

const storage = new MemoryStorage()
const queue = new Queue<{ email: string }, { sent: boolean }>({
 storage,
 concurrency: 5
})

queue.execute(async job => {
 // your business logic
 return { sent: true }
})

await queue.start()

// fire-and-forget
await queue.enqueue('email-1', { email: 'user@example.com' })

// request/response
const result = await queue.enqueueAndWait('email-2', { email: 'another@example.com' }, { timeout: 30_000 })
console.log(result)

await queue.stop()

Architecture description

When you call enqueue(), the producer checks if the job already exists in the storage. If it’s a new job, it's added to the queue with the state “queued,” and the method returns immediately. If the job is a duplicate, the storage returns a duplicate status without creating a new entry.

When you call enqueueAndWait(), the producer first subscribes to a notification for that job, then enqueues it. If the job was already processed, it returns the cached result immediately. Otherwise, it waits for a notification from the worker when the job completes (or fails), then fetches the result and returns it.

The consumer continuously dequeues jobs from the storage using a blocking move operation. When it receives a job, it marks it as “processing” and executes the handler. On success, it stores the result with TTL and marks the job as completed. On failure, it either retries (if attempts remain) or marks the job as failed.

The producer API supports per-job options such as maxAttempts and resultTTL, which are useful when not all jobs have the same retention or retry requirements. For example, you might keep invoice-generation results longer than low-value notification results, even if they run on the same queue.

Storage backends for different environments

@platformatic/job-queue ships with three storage adapters:

MemoryStorage

MemoryStorage keeps all queue states in process memory. This makes it ideal for local development, testing, and simple single-instance services where data can be ephemeral.

import { Queue, MemoryStorage } from '@platformatic/job-queue'
const storage = new MemoryStorage()
const queue = new Queue({ storage })

Jobs are stored in JavaScript Maps and Sets within the same process. This gives you the lowest latency possible, but means jobs are lost if the process restarts. For development workflows where you restart frequently, this is usually not a concern.

FileStorage

FileStorage persists the queue state to the filesystem in JSON format. It works well for simple deployments on a single node where you need persistence but do not want external dependencies like Redis.

import { Queue, FileStorage } from '@platformatic/job-queue'

const storage = new FileStorage('./queue-data')
const queue = new Queue({ storage })

The storage writes atomically to prevent corruption, and it maintains separate files for jobs, metadata, and locks. Since it relies on file system locks, it is not suitable for multi-node deployments.

RedisStorage

RedisStorage uses Redis (7+) or Valkey (8+) for distributed queue operations. This is the recommended choice for production workloads that require horizontal scaling, leader election, or cross-instance coordination.

import { Queue, RedisStorage } from '@platformatic/job-queue'
const storage = new RedisStorage({ connectionString: 'redis://localhost:6379' })
const queue = new Queue({ storage })

RedisStorage leverages Redis data structures for atomic operations:

• Lists for job queues

• Sorted sets for delayed job scheduling

• Pub/sub for notifications across instances

• Lua scripts for atomic state changes

For high availability, RedisStorage also supports Sentinel and Cluster modes for failover and sharding.

Choosing the right backend

Start with MemoryStorage for development, use FileStorage for simple single-node deployments, and choose RedisStorage for production systems that need horizontal scaling.

Reliability features that matter in production

The library is designed around the real failure modes of job processing systems.

Visualize this: you deploy a routine patch, and one of your job workers crashes unnoticed. By the next day, 5,000 critical jobs piled up and could have vanished forever. But thanks to built-in recovery, every one of them was automatically rescued. Situations like this are exactly where background processing systems prove their worth, thanks to strong safeguards.

Recovering stalled jobs

If a worker crashes while processing a job, the Reaper can detect the stalled work and requeue it after visibilityTimeout.

import { Reaper } from '@platformatic/job-queue'
const reaper = new Reaper({
 storage,
 visibilityTimeout: 30_000
})
await reaper.start()

For high availability, the Reaper also supports leader election (with Redis storage), so multiple instances can run safely while only one acts as leader at a time. If the leader goes away, another instance takes over, which helps avoid manual control during incidents.

Controlled retries and terminal states

Failed jobs can retry automatically up to maxRetries. When retries are exhausted, errors are persisted as a terminal state so producers can inspect or react programmatically.

This gives you reliable behavior for flaky dependencies, such as third-party APIs: transient failures recover automatically, while permanent failures remain visible and actionable.

Graceful shutdown

When stopping a worker, queue.stop() waits for in-flight jobs to finish. This reduces dropped work during deploys and restarts and helps keep queue state consistent across gradual updates. In practice, this means you can safely perform blue/green or canary deployments without worrying about losing in-progress work. Teams can ship changes faster, with the confidence that jobs will complete and customer data will not go missing, even as new versions are rolled out.

Request/response without building custom plumbing

One particularly useful capability is enqueueAndWait(). Teams often build this pattern manually on top of queues, but it is already integrated here, including timeout handling and typed errors.

try {
 const result = await queue.enqueueAndWait('invoice-123', payload, { timeout: 10_000 })
 return result
} catch (error) {
 // handle TimeoutError / JobFailedError, etc.
}

This is a good fit when work should run in a worker context, but the caller still needs a bounded response path, such as document generation, webhook fan-out, or expensive validation that should not run on an HTTP thread.

You also get explicit queue errors (TimeoutError, JobFailedError, and others), so your application can distinguish among transport problems, worker failures, and business-level errors.

Getting started

Install the package:

npm install @platformatic/job-queue

Then choose a backend based on your environment:

1. Start with MemoryStorage for local development.

2. Move to RedisStorage (Redis 7+ or Valkey 8+) for production.

3. Add Reaper when running multiple workers or when stalled-job recovery is required.

If you already have queue infrastructure in place, one good migration approach is to move one bounded workflow first (for example, email delivery or report generation), validate behavior and observability, and then expand usage across other jobs.

We recommend separating responsibilities into dedicated processes:

• Producer services enqueue jobs from HTTP handlers or internal events.

• Worker services execute jobs with tuned concurrency.

• A Reaper instance handles stalled-job recovery (or multiple instances with leader election).

This setup lets you scale producers and workers independently. If incoming traffic spikes, add producers; if processing backlog grows, add workers.

Final thoughts

@platformatic/job-queue is a practical option for Node.js teams that want reliable background processing without having to assemble every reliability feature from scratch. The combination of deduplication, request/response semantics, retries, and pluggable storage makes it flexible enough for both simple jobs and more demanding production workloads. Most importantly, it lets you focus on what matters most: building features and generating value, knowing your background tasks are handled with care. Imagine deployments where you can sleep soundly, confident that every job is accounted for and that no critical work is lost, even during outages. With the right foundation, you are set up not just for peace of mind, but for lasting success as your systems and team continue to grow.

If you are evaluating queue systems for your next service, this is a good time to try it and share feedback with the team (us). Real-world feedback is especially valuable while the project is still young and evolving quickly. If you run into an unexpected edge case or a strange retry failure, please open an issue describing your scenario: we love to fix hard problems. Concrete examples help us improve reliability for everyone!

In this article, I want to break down some of the key design choices that made OpenClaw such a sensation, what the biggest friction points are for enterprises trying to adopt and run agents at scale, and how the open-source work we’ve been doing at Platformatic can bridge that gap. 

Developers and the Path of Least Resistance

OpenClaw racked up 196,000 GitHub stars, caught the eye of Meta and OpenAI, and got flagged by Gartner as an “unacceptable cybersecurity risk” for enterprises. So what’s really going on?

Let’s first take a look at what made OpenClaw so appealing to everyday developers. Namely, it brought the world of LLMs and agents to where developers were most excited to apply them, i.e., the data and apps on their own machines. (This, interestingly enough, is a common thread between consumers and enterprise teams, which I’ll touch on later.)

Second, it came with a fantastic developer experience out of the box. Because it was built on Node.js, OpenClaw shipped with a rich  ecosystem that let developers hook their agents up to … well, pretty much whatever they wanted, just with a few simple lines of code.  

Your Agents, Your System

So what does OpenClaw’s viral appeal teach us about bringing agents to the enterprise?

Well, it turns out, agents are most useful when you run them where they can do useful things. Again, your data, your files, all that good stuff. That’s greatly simplified if your agent runs on your own system, and it’s this simplicity that's largely been missing from most cloud-based Agentic Platforms. 

This is because enterprises need something that integrates with the infrastructure they’ve already invested in. 

When we talk about the sometimes ambiguous notion of “the enterprise”, what we are really referring to about are teams that have invested years of engineering effort and millions of dollars (both in terms of engineering hours and/or commercial licences) building heavily customized Kubernetes platforms for their teams, replete with observability stacks, CI/CD pipelines, security policies, and compliance systems; all heavily customized to the ergonomics of their developers and domain. So you can imagine how platform teams respond when a new vendor says,

“Great news, agentic AI is here. You just need to adopt this entirely new platform to run it.”

Here’s where Watt comes in: making your existing stack agent-ready.

Why Node.js Is the Runtime for Agents

OpenClaw’s architecture is a 390,000-line TypeScript codebase running on Node.js 22 or higher. Its Gateway, the control plane that manages every agent interaction across WhatsApp, Telegram, Slack, Discord, iMessage, and more, is written entirely in JavaScript and TypeScript. It works anywhere Node.js works. 

If you’ve ever looked closely at how agents work, this makes a lot of sense. Agents aren’t batch jobs; they are persistent, event-driven processes that keep long WebSocket connections open, respond to messages across multiple channels at once, call external APIs, and manage conversations over time. This is exactly what Node.js was built for. The event loop, the main feature that makes Node.js great for high-concurrency I/O, lets an agent handle many conversations, tool calls, and streaming LLM responses at the same time without needing a separate thread for each connection.

OpenClaw chose Node.js because no other runtime handles this pattern as smoothly. Python would struggle with concurrency. Go could work, but it lacks the rich ecosystem that lets Steinberger build integrations for every major messaging platform in just weeks. The npm ecosystem, such as  Baileys for WhatsApp, grammY for Telegram, discord.js, and Slack’s Bolt SDK, is why a single developer could build something in weeks that would take an enterprise team months.

Watt: The Primitive that makes your existing stack Agent-Ready

At its core, Watt implements ideas that are simple to grasp but challenging to execute (elegantly) from an engineering perspective. Namely, we wanted to 1) truly unlock the power of multi-threading for Node.js by running your application as a worker thread within Watt, and  2) provide a universal primitive to run your app across any infrastructure, while making all the NFRs (observability, thread management, etc) “out of the box”.

So - what are the benefits of using tools like Watt to run and manage your agents as isolated worker threads? Let’s do a quick reality check.

• Can you see every long-running, event-driven process in your stack right now? 

• Do you have automated visibility into which connections are open, what messages are moving, or how your agents scale during spikes in requests?

If you hesitate, you’re not alone. Most enterprise stacks aren’t built for persistent, event-driven workloads. That’s exactly where agentic AI exposes the cracks.

Long-running operations for agents. Agents are stateful, as they inherently operate in a “loop”. They that must remain active for hours, days, or even longer, maintaining state, holding connections, and reacting to events across multiple channels. Sub-agents can be spawned on demand to adapt the system on the fly. Watt allows your application to do so in isolated worker threads. Watt manages their full lifecycle of long-running Node.js agents on Kubernetes, including smooth restarts, health monitoring, and resource management, without losing agent state. 

For enterprise teams, this brings real improvements: Watt's ability to recycle and self-heal threads means agentic workflows keep running without interruption. 

Put another way, if your agent is in the middle of a conversation with a customer, coordinating across Slack and email, and your pod is rescheduled on Kubernetes, you lose your state and frustrate your users. With Watt, we automatically detect service degradation and act accordingly, gracefully hot-swapping threads before Kubernetes (or your customer) notices anything has gone awry. 

Out-of-the-Box Observability for Node.js. The OpenClaw security nightmare was as much about bad defaults as anything. Let’s be honest - configuring security and observability is going to be perceived as a distracting sidequest for an excited developer who wants to just ship (they are called ‘NFRs’ for a reason, after all).  Our workaround was to provide all of this “out-of-the-box” for both devs and the platform teams that look after them.

To this end, Watt’s Intelligent Command Center (and its companion Admin service) provides continuous profiling, event loop monitoring, and application-level metrics, giving DevOps teams and security leaders a clear view of every Node.js process in their cluster. You can’t secure what you can’t see.

Intelligent autoscaling tied to Node.js internals. Agents often have unpredictable workloads. One agent might be idle for hours, then suddenly need to handle dozens of LLM calls when a user starts a complex workflow. 

Watt’s autoscaler understands Node.js event loop metrics, not just CPU and memory, and scales based on real application-level demand. This kind of event-loop aware scaling can deliver strong business results. Application-level autoscaling strategies like this can cut cloud compute costs by 25 percent or more by avoiding overprovisioning during slow periods and preventing slowdowns during traffic spikes. 

Put another way: autoscaling on the wrong metrics is expensive, both financially and in terms of performance SLOs.

Enterprise-grade operations without rewrites. A big driver of adoption for us has been the fact that we don’t ask teams to rewrite their Node.js applications or give up their current infrastructure. 

Watt wraps your Node.js app and adds operational features such as profiling, logging, tracing, and scaling, all without code changes. It integrates with your current Kubernetes setup, works with your observability tools, and fits into your deployment workflows. If your team has been building agent features on Node.js, Watt makes those agents ready for production on the infrastructure you already have.

Watt and the Multi-agent-verse

Let’s imagine a multi-agent workflow you could put into production next quarter:

1. A sales agent gets a message from a customer about a delayed order. 

2. Instead of forwarding the ticket manually, the sales agent automatically works with a logistics-tracking agent to check the shipment status. 

3. If there’s a problem, an incident response agent opens a case in the ITSM system and notifies the customer proactively, all without human intervention. 

4. Your teams see faster response times, fewer dropped tickets, and a better customer experience, and the whole process is auditable from start to finish.

At its core, this is a distributed systems problem, and one that ties back to Node’s core strengths, with its event-driven architecture, streaming capabilities, and unmatched ecosystem for live communication. 

But distributed systems also need operational infrastructure. They require monitoring, lifecycle management, security boundaries, and proven operational tools that Matteo and I have spent the last decade building.

OpenClaw showed that a single developer using Node.js can build an agent platform that excites hundreds of thousands of people. Imagine what happens when enterprises bring the same capabilities and add proper security, observability, and operational controls.

What if you could deploy AI agents the same way you deploy microservices, with worker isolation, auto-scaling, health checks, and hot-reload, on infrastructure you already own? Watt could run each agent type as an isolated application with its own worker pool, sandboxed filesystem, and tool policy, while a single gateway handles authentication, role-based access control, and routing across Slack, Teams, Telegram, or any HTTP client through an OpenAI-compatible API. No vendor lock-in, no data leaving your network, and the same Node.js runtime your team already knows, just pointed at a harder problem.

That’s the world Watt is making real.

Time to take the lobster by the claws.

If you’re leading an enterprise and watching OpenClaw unfold, here’s my take:

Don’t ban agentic AI. The demand is real, and your teams will find workarounds if you try. Instead, invest in the infrastructure that ensures safety. The pull of the ecosystem is strong. Your agent strategy is really a Node.js strategy.

Get your operations in order. You need visibility into long-running Node.js processes, autoscaling that understands the event loop, and lifecycle management for processes that aren’t just stateless web servers.

Start with what you already have. If your teams are running Node.js (and most likely they are), the path to production-ready agents is shorter than you think. Watt is built to meet you where you are.

 The OpenClaw moment is just the beginning. Enterprises that build the right infrastructure now will be the ones to take advantage of agentic AI. Those who respond with bans and blocks will spend years trying to catch up.

Node.js made OpenClaw possible. Your cloud investment made your infrastructure real. Watt connects the two, turning them into enterprise-grade platforms that run secure, scalable, and durable agents.

Well, almost. Node.js does not enable Pointer compression by default for two historical reasons.

First, there was the '4 GB cage' limitation, which meant that enabling pointer compression required the entire Node.js process to share a single 4 GB memory space between the main thread and all the worker threads. This was a significant issue. Cloudflare and Igalia partner to solve it so that the cage can be per-isolate (an individual instance of the V8 engine).

Next, some worried that compressing and decompressing pointers on each heap access would introduce performance overhead. Cloudflare, Igalia, and the Node.js project collaborated to determine exactly what kind of overhead existed and assess whether it would impact real-world applications.

To test this, we created node-caged, a Node.js 25 Docker image with pointer compression turned on, and ran production-level benchmarks on AWS EKS.

In short, we achieved 50% memory savings with only a 2-4% increase in average latency across real-world workloads and reduced P99 latency by 7%. For most teams, this trade-off is an easy choice.

How Pointer Compression Works

Every JavaScript object is stored on V8’s heap. Inside, objects point to each other using 64-bit memory addresses on a 64-bit system. For example, an object like { name: "Alice", age: 30 } has several internal pointers: one to its hidden class (shape), one to where its properties are stored, and one to the string “Alice” on the heap.

As you might imagine, all these pointers can add up in a typical Node.js app, taking up a lot of valuable heap space. On a 64-bit system, each pointer uses 8 bytes, even though most V8 heaps are much smaller than the huge address space they could use.

Pointer compression takes advantage of this. Instead of saving full 64-bit memory addresses, V8 stores 32-bit offsets (relative distances from a fixed starting point, called the base address). When reading from the heap (the section of memory where objects are stored), it rebuilds the full pointer by adding the base and the offset. When writing, it compresses the pointer by subtracting the base from the full address.

The trade-off is simple:

• Memory: Each pointer goes from 8 bytes to 4 bytes. For structures with many pointers—such as objects, arrays, closures, Maps, and Sets—this can reduce memory consumption by around 50%

• CPU: Each heap access now needs one extra addition (for reads) or subtraction (for writes). To put it in perspective, this extra operation is akin to a Level 1 cache hit in terms of computational effort. These are incredibly fast operations, and although millions of them occur every second, their impact is minimal, akin to a gentle ripple in a vast ocean of processing tasks.

• Heap limit: 32-bit offsets can only reach 4GB of memory per V8 isolate (a separate instance of the JavaScript engine with its own memory and execution state). For most Node.js services, which usually use less than 1GB, this isn’t a problem.

Chrome has used pointer compression since 2020, but Node.js hasn't. Previously, using this feature required setting a flag (--experimental-enable-pointer-compression) at compile time, which often felt like an 'expert-only' option for many developers. However, the introduction of node-caged has transformed this, enabling pointer compression with a simple one-line Docker image swap. This substantial simplification opens the door for a much broader audience to experiment with the feature more immediately.

What Changed: IsolateGroups

Pointer compression has been part of V8 for years. Node.js didn’t use it before, not because of CPU overhead, but because of the memory cage limitation.

Originally, V8’s pointer compression made every isolate in a process share a single “pointer cage”—a 4GB block of memory for all compressed pointers. This meant the main thread and all worker threads had to fit into the same 4GB. In Chrome, where each tab runs in its own process, this worked fine. But for Node.js, where workers share a process, it was a big problem.

In November 2024, James Snell (Cloudflare, Node.js TSC) initiated the endeavor to address this challenge. Cloudflare sponsored Igalia engineers Andy Wingo and Dmitry Bezhetskov to introduce a new V8 feature, IsolateGroups, which gives each pointer its own compression cage. (You can read more about this feature and work at https://dbezhetskov.dev/multi-sandboxes/.)

The pivotal modification is that multiple IsolateGroups can now exist within a single process, each having its own 4GB cage, thus eliminating the process-wide memory constraint. This work symbolizes a significant collaboration between organizations, showcasing the strength of the open-source ecosystem. Thanks to this work, enabling pointer compression in Node.js changed from (shared cage):

to (IsolateGroups):

In V8, the C++ change is simple. Use v8::Isolate::New(group, ...) instead of v8::Isolate::New(...). Now, each worker thread gets its own 4GB heap. The only limit is the system’s available memory.

Snell’s Node.js integration landed in October 2025: 62 lines across 8 files. This represents less than one commit's worth of changes across most modules, underscoring the update's maintainability. The code was reviewed and approved by Joyee Cheung [Igalia], Michael Zasso [Zakodium], Stephen Belanger [Platformatic], and me [Platformatic]. Cheung also fixed the pointer compression build itself, which had been broken since Node.js 22. I tested with real-world Next.js SSR applications and confirmed a ~50% reduction in heap usage before approving.

This feature still requires a compile-time flag and isn’t in official Node.js builds yet. That’s why we made node-caged.

The Experiment

Two of our four configurations use Platformatic Watt, our open-source Node.js application server. Watt runs multiple Node.js applications as worker threads (separate execution threads) within a single process, using the Linux kernel's 'SO_REUSEPORT' (a system feature that allows multiple processes to listen on the same network port) to distribute connections directly to workers. No master process, no IPC (Inter-Process Communication) coordination. In previous benchmarks, this eliminated the ~30% performance tax imposed by PM2 and the 'cluster' module through IPC-based load balancing.

We set up a Next.js e-commerce app—a trading card marketplace with 10,000 cards, 100,000 listings, server-side rendering, search, and simulated database delays—on a Kubernetes cluster. We tested four setups, all using the same hardware and app code:

Infrastructure: We used AWS EKS with m5.2xlarge nodes (8 vCPUs, 32GB RAM), 6 replicas for plain Node and 3 replicas for Watt (each with 2 workers, for a total of 6 processes). Both images used the same Debian bookworm-slim base and Node.js 25, so the only difference was the use of pointer compression.

Workload: We used k6 with a ramping-arrival-rate executor, running 400 requests per second for 120 seconds after a 60-second ramp-up. The traffic was mixed as follows:

• 20% homepage (SSR with featured cards, recent listings)

• 25% search (full-text search with pagination)

• 20% card detail (individual product page SSR)

• 15% game category pages

• 10% games listing

• 5% sellers listing

• 5% set detail pages

Each request follows the server-side rendering path. It loads JSON data from disk, applies query filters, renders React components to HTML, and sends the response. We added a simulated 1-5ms database delay to mimic real data access.

The Results

Plain Node.js: Standard vs Pointer Compression

The average overhead was 2.5%. That translates to approximately 1 ms additional latency on our 40 ms median latency. This is a minor trade-off for cutting memory use in half. But if you look at p99 and max latency, they’re actually lower with pointer compression. A smaller heap means the garbage collector has less work to do, so there are fewer and shorter GC pauses. In these cases, pointer compression doesn’t just keep up—it performs better.

Platformatic Watt (2 workers): Standard vs Pointer Compression

A similar outcome appears here. Average overhead is slightly higher (4.2%), the median remains unchanged, and maximum latency drops by 20% due to reduced garbage collection pressure.

The Full Picture: Watt + Pointer Compression vs Baseline

This is the comparison that matters for production decisions. What do you get if you adopt both Watt and pointer compression?

Consider this: on average, it’s 15% faster, delivering significant speed gains without requiring code adjustments. This kind of improvement could be likened to the gains typically achieved by rewriting key parts of a system in a more optimized language, such as C++. Not only does it increase p99 latency by 43%, but it also halves memory usage, all for free with minimal effort.

Why the Hello-World Benchmarks Were Misleading

Initial tests of pointer compression on a basic Next.js starter app showed a 56% overhead. This outcome was unexpected.

But a simple hello-world SSR page mostly does V8 internal work: compiling templates, diffing the virtual DOM, and joining strings. There’s no I/O, no data loading, and no real app logic. Every operation goes through pointer decompression.

Real applications are different. A typical request spends most of its time on:

1. I/O wait: database queries, cache lookups, API calls to downstream services

2. Data marshaling: JSON parsing, response body construction

3. Framework overhead: routing, middleware chains, header processing

4. OS/network: TCP handling, TLS, kernel scheduling

The V8 heap access that triggers pointer decompression is only one component of the total request time. As the ratio of “real work” to “pure V8 pointer chasing” increases, the overhead of pointer compression shrinks proportionally.

Our e-commerce app includes simulated database delays of 1-5ms, JSON parsing of datasets with 10,000+ records, search filtering, pagination, and full SSR rendering with React. In that context, the pointer decompression overhead rounds to noise.

The takeaway: always use realistic workloads for benchmarking; asmicrobenchmarks can give you the wrong idea. As a challenge to validate these findings, we invite you to try your heaviest endpoint and share your results. This collaborative effort can transform observations into active participation, build trust, and foster community validation of the effectiveness of pointer compression.

The Technical Details: Why GC Gets Better

The improved tail latencies deserve a deeper explanation. V8’s garbage collector (Orinoco) performs several types of collection:

• Minor GC (Scavenge): Copies live objects from the young generation. Time is proportional to the number of live objects and their size.

• Major GC (Mark-Sweep-Compact): Marks all reachable objects, sweeps dead ones, and optionally compacts. Time depends on the total heap size and the level of fragmentation.

With pointer compression, every object is smaller. This has domino effects:

1. Objects fit in fewer cache lines. A compressed object that fits in a single 64-byte cache line instead of two means the GC’s marking phase generates half as many cache misses while traversing the object graph.

2. The young generation fills more slowly. Smaller objects mean more allocations before a minor GC is triggered. Fewer minor GCs per unit of work.

3. Major GC has less to scan. A 1GB heap with compressed pointers contains the same logical data as a 2GB heap without. The GC scans half the bytes to process the same application state.

4. Compaction moves fewer bytes. When the GC compacts the heap to reduce fragmentation, smaller objects mean less data to copy.

The end result is that GC pauses are both shorter and less frequent. This corresponds to what we saw in the p99 and max latency numbers. When a long-tail request lines up with a GC pause, the pause is now shorter.

What This Means for Your Business

Cut Your Kubernetes Bill

If you run Node.js on Kubernetes with 2GB memory limits per pod, pointer compression lets you cut that to 1GB. You get the same app and performance, but can run twice as many pods per node or use half as many nodes. What would halving pod memory do to your cluster bill? Take a moment to calculate the potential savings based on your current setup and see how much your organization could benefit from implementing pointer compression.

A 6-node m5.2xlarge EKS cluster (at $0.384 per hour per node) costs about $16,600 a year. Dropping to 3 nodes saves $8,300 a year. In a real production fleet with 50 or more nodes, the savings can reach $80,000 to $100,000 a year, all without changing your code.

For platform teams running hundreds of Node.js microservices, these savings add up. Each service has a baseline memory load from the V8 heap, framework, and modules. Pointer compression reduces the baseline across all services simultaneously.

Double Your Tenant Density

Multi-tenant SaaS platforms, where each tenant runs in an isolated Node.js process, hit memory as the binding constraint for density. If each tenant’s worker uses 512 MB, pointer compression reduces it to ~256 MB. That’s 2x tenants per host.

At scale, this changes your costs. If each tenant costs $5 per month for infrastructure and you have 10,000 tenants, cutting memory in half saves $25,000 a month, or $300,000 a year.

Unlock Edge Deployment

Edge runtimes like Lambda@Edge, Cloudflare Workers, and Deno Deploy have strict memory limits, typically 128MB to 512MB per isolate. Cloudflare sponsored the IsolateGroups work in V8 because their Workers runtime needed pointer compression to support more isolates. Pointer compression can be the difference between your app running at the edge or needing to go back to the origin server.

That matters for revenue. Every 100ms of latency measurably reduces conversion rates. An e-commerce site moving SSR to the edge shaves 50-200ms off TTFB, depending on user location. For a $50M/year business, that latency improvement can translate to hundreds of thousands in incremental annual revenue.

Handle More Concurrent Connections

For WebSocket-based applications (chat, collaboration, live dashboards, gaming), each persistent connection holds state in memory. A server handling 50,000 connections at ~10KB heap per connection uses 500MB. With pointer compression, that drops to ~250MB, allowing the same server to handle 100,000 connections, or halving your WebSocket server fleet.

Compatibility Constraints

There is one strict limit: each V8 IsolateGroup’s pointer cage is 4GB. 32-bit compressed pointers can only address 4GB. With IsolateGroups, this limit applies to each isolate, not the whole process. Your main thread gets 4GB, each worker thread gets 4GB, and the total is only limited by your system’s memory.

For most Node.js services, 4GB per isolate is irrelevant. The vast majority of production processes run well under 1GB of heap. If your service genuinely requires more than 4GB of heap per isolate (e.g., large ML model inference, massive in-memory caches, or heavy ETL pipelines), pointer compression is not an option. Note that only the V8 JavaScript heap lives inside the cage; native add-on allocations and ArrayBuffer backing stores do not count against the 4GB limit.

There is one more compatibility constraint: native addons built with the legacy NAN (Native Abstractions for Node.js) won't work with pointer compression enabled. NAN exposes V8 internals directly, and pointer compression changes the internal representation of V8 objects. When you recompile, the ABI is different. Addons built on [Node-API](https://nodejs.org/api/n-api.html) (formerly N-API) are unaffected because Node-API abstracts away V8's pointer layout entirely. The most popular native packages have already migrated: sharp, bcrypt, canvas, sqlite3, leveldown, bufferutil, and utf-8-validate all use Node-API today. The main holdout is nodegit, which still depends on NAN. If you're unsure, check your dependency tree with npm ls nan. If nothing shows up, you're good.

For everyone else—which is most Node.js deployments—there’s nothing to lose.

Try It

It’s a drop-in replacement. You don’t need to change any code.

# Before
FROM node:25-bookworm-slim

# After
FROM platformatic/node-caged:25-slim

The platformatic/node-caged image is built from the Node.js v25.x branch with --experimental-enable-pointer-compression. It’s the same Node.js, same APIs, and everything else—just with smaller heaps.

Available tags: latest, slim, 25, 25-slim.

Start by testing in staging. Watch your memory usage go down. Make sure your p99 latency stays within your SLO. Then deploy it.

As always, we want to hear from you! Share your results and experience by dropping us a note at hello@platformatic.dev or by engaging on social media if you’d like to chat about anything you’re building.

Benchmarks were run on AWS EKS (m5.2xlarge nodes, us-west-2) using k6 with ramping-arrival-rate at 400 req/s sustained. The application is a Next.js 16 e-commerce marketplace with server-side rendering and a JSON-based data layer. Full benchmark infrastructure and results are available in the node-caged repository. The upstream V8 IsolateGroups feature was implemented by Igalia, sponsored by Cloudflare. Node.js integration by James Snell, with build fixes by Joyee Cheung. See the tracking issue for the full history.

Watt 3.32 introduces first-class support for TanStack Start, the full-stack React framework from the creators of TanStack Query and TanStack Router. We benchmarked TanStack Start on AWS EKS under extreme load (10,000 req/s) and found that Watt matches single-process Node.js throughput and improves tail latency by 10%, consistently demonstrating measurable improvements.

Both configurations were tested under identical conditions at a 10,000 req/s target load. The following section details the full methodology and raw data.

We’re excited to announce that Watt 3.32 adds native support for TanStack Start, bringing the same performance benefits that Next.js users have enjoyed to this rapidly growing full-stack React framework.

What is TanStack Start?

TanStack Start is a modern full-stack React framework built on top of TanStack Router, Vinxi, and Nitro. It offers:

• Type-safe routing with first-class TypeScript support

• Server functions for seamless client-server communication

• SSR and streaming out of the box

• File-based routing with nested layouts

• Built-in data loading patterns from the TanStack Query team

For teams already using TanStack Query and TanStack Router, TanStack Start provides a natural progression to full-stack development with familiar patterns and excellent developer experience. Next, we'll explore why running TanStack Start with Watt is a strong architectural choice.

Why Watt for TanStack Start?

Like Next.js, TanStack Start uses server-side rendering (SSR), which is CPU-bound and poses familiar scaling challenges:

1. Node.js runs on a single CPU core by default, underutilizing multi-core servers.

2. SSR frameworks require the full request context to gauge load, preventing early request rejection.

3. Event loop blocking: CPU-intensive rendering can cause the event loop to block, leading to latency spikes.

Watt addresses these with SO_REUSEPORT, distributing connections at the kernel level across workers and removing IPC overhead. To validate this approach, our benchmark methodology is explained below.

Benchmark Methodology

Infrastructure

All benchmarks ran on AWS EKS (Elastic Kubernetes Service) with the following infrastructure:

• EKS Cluster: 4 nodes running m5.2xlarge instances (8 vCPUs, 32GB RAM each)

• Region: us-west-2

• Load Testing Instance: c7gn.2xlarge (8 vCPUs, 16GB RAM, network-optimized)

• Load Testing Tool: Grafana k6

The environment was ephemeral, created on demand via shell scripts and the AWS CLI, then torn down after each test run.

Software Versions

Resource Allocation

Each configuration received identical total CPU resources:

Pods were distributed evenly across all 4 cluster nodes using topologySpreadConstraints.

Load Test Configuration

We tested under extreme load to stress-test both configurations:

export const options = {
 scenarios: {
   ramping_load: {
     executor: 'ramping-arrival-rate',
     startRate: 100,
     timeUnit: '1s',
     preAllocatedVUs: 1000,
     maxVUs: 10000,
     stages: [
       { duration: '20s', target: 2000 },   // Ramp to 2,000 req/s
       { duration: '20s', target: 5000 },   // Ramp to 5,000 req/s
       { duration: '20s', target: 8000 },   // Ramp to 8,000 req/s
       { duration: '20s', target: 10000 },  // Ramp to 10,000 req/s
       { duration: '100s', target: 10000 }, // Hold at 10,000 req/s
     ],
   },
 },
};

This configuration ramps up to 10,000 requests per second and holds for 100 seconds, deliberately exceeding the capacity of both configurations to observe behavior under stress.

Test Protocol

1. NLB Warm-up Phase: All endpoints received a 60-second warm-up (ramping from 10 to 500 req/s) to ensure AWS Network Load Balancers were properly scaled

2. Pre-test Warm-up: Each runtime received a 20-second warm-up before its test

3. Test Execution: 180 seconds total (80s ramp + 100s hold at 10k req/s)

4. Cooldown: 480 seconds between each test to allow system recovery

Results

Performance Summary

Latency (Successful Requests Only)

Key Observations

1. Equivalent Throughput Under Extreme Load

Both Watt and single-process Node.js achieved nearly identical throughput (~5,958 req/s) under the 10,000 req/s target load. This demonstrates that Watt’s multi-worker architecture introduces no overhead compared to running Node.js directly.

2. Better Tail Latency with Watt

While average latencies were equivalent, Watt showed measurably better tail latency:

• p99: 263ms (Watt) vs 289ms (Node.js) - 9% improvement

• p95: 221ms (Watt) vs 250ms (Node.js) - 12% improvement

• p90: 196ms (Watt) vs 216ms (Node.js) - 9% improvement

This improvement comes from SO_REUSEPORT’s kernel-level load distribution, which prevents request pileup on any single worker.

3. Slightly Higher Success Rate

Watt achieved a 79.3% success rate compared to Node.js’s 78.6% - a small but consistent improvement under stress. Both configurations were pushed well beyond their sustainable capacity (the target was 10k req/s, but actual throughput was ~6k req/s), so the high failure rates are expected.

4. Test Was Deliberately Extreme

The 20%+ failure rate across both configurations indicates we successfully stress-tested beyond capacity. Under normal production loads (staying within throughput limits), both configurations would achieve near-100% success rates, as demonstrated in our Next.js benchmarks at 1,000 req/s.

Getting Started with TanStack Start on Watt

Adding Watt support to your TanStack Start application requires minimal configuration:

1. Install Dependencies

npm install wattpm @platformatic/tanstack

2. Create watt.json

{
 "$schema": "https://schemas.platformatic.dev/@platformatic/tanstack/3.32.0.json",
 "application": {
   "outputDirectory": ".output"
 },
 "runtime": {
   "logger": {
     "level": "info"
   },
   "server": {
     "hostname": "0.0.0.0",
     "port": 3000
   },
   "workers": {
     "static": 2
   }
 }
}

3. Update package.json Scripts

{
 "scripts": {
   "build": "vite build",
   "build:watt": "NODE_ENV=production wattpm build",
   "start:watt": "wattpm start"
 }
}

4. Build and Run

npm run build:watt

npm run start:watt

That’s it. Watt will automatically detect your TanStack Start application and configure the appropriate build and runtime settings.

Kubernetes Deployment

For Kubernetes deployments, the same principles from our Next.js guide apply. Here’s a sample deployment configuration:

apiVersion: apps/v1
kind: Deployment
metadata:
 name: tanstack-watt
spec:
 replicas: 4
 template:
   spec:
     topologySpreadConstraints:
       - maxSkew: 1
         topologyKey: kubernetes.io/hostname
         whenUnsatisfiable: DoNotSchedule
         labelSelector:
           matchLabels:
             app: tanstack-watt
     containers:
       - name: tanstack-watt
         image: your-registry/tanstack-app:latest
         env:
           - name: WORKERS
             value: "2"
         resources:
           requests:
             cpu: '2000m'
             memory: '4Gi'
           limits:
             cpu: '2000m'
             memory: '4Gi'
         ports:
           - containerPort: 3000

Key points:

• Use topologySpreadConstraints to distribute pods evenly across nodes.

• Set WORKERS to match your CPU allocation (2 workers for 2 CPUs)

• Watt’s health monitoring will automatically restart unhealthy workers without terminating the pod.

Conclusion

Watt 3.32 brings the same performance benefits to TanStack Start that Next.js users have enjoyed: kernel-level load distribution via SO_REUSEPORT, zero-overhead multi-worker scaling, and external health monitoring to improve throughput and tail latency.

Our benchmarks show that under extreme load (10,000 req/s), Watt matches Node.js throughput while delivering measurably better tail latency (p99 improved by 9%, p95 by 12%). In production deployments constrained by capacity, both approaches achieve near-complete reliability.

If you’re building with TanStack Start and deploying to Kubernetes or any multi-core environment, Watt provides a straightforward path to better resource utilization and improved tail latency with minimal configuration changes.

The complete benchmark code is available at: https://github.com/platformatic/k8s-watt-performance-demo.

To get started with Watt, visit: https://docs.platformatic.dev.

For questions or enterprise support, reach out to info@platformatic.dev or connect with us on Discord.

How often have you captured a CPU profile, stared at a flamegraph, and tried to make sense of thousands of stack frames? What if your AI assistant could help you understand exactly where your application is spending time?

Today, we’re releasing a new feature in @platformatic/flame that generates LLM-friendly markdown analysis alongside your flamegraphs. Now, when you profile your Node.js application, you get three outputs:

• Binary pprof data (.pb) - for tooling compatibility

• Interactive HTML flamegraph (.html) - for visual exploration

• Markdown analysis (.md) - for AI-assisted debugging

This means you can drop your profile analysis directly into Cursor, Claude Code, OpenCode, or any AI assistant and get intelligent insights about your application’s performance characteristics.

The Problem with Traditional Profiling

Flamegraphs are incredibly powerful visualization tools, but they have limitations:

1. They require expertise to interpret - Understanding which stack frames matter takes experience.

2. They don’t prioritize hotspots - You see everything, but the critical bottlenecks aren’t highlighted.

3. They’re not searchable by AI - You can’t paste an SVG into ChatGPT and ask “what’s slow?”

We built Flame to make profiling accessible, and this update takes it a step further by making profile data consumable by AI assistants.

How It Works

When you run Flame, it now automatically generates a markdown file with structured hotspot analysis:

# Profile your application
flame run server.js

# When you stop the app (Ctrl-C), you'll see:
# 🔥 CPU profile written to: cpu-profile-2025-01-21T12-00-00-000Z.pb
# 🔥 CPU flamegraph generated: cpu-profile-2025-01-21T12-00-00-000Z.html
# 🔥 CPU markdown generated: cpu-profile-2025-01-21T12-00-00-000Z.md
# 🔥 Heap profile written to: heap-profile-2025-01-21T12-00-00-000Z.pb
# 🔥 Heap flamegraph generated: heap-profile-2025-01-21T12-00-00-000Z.html
# 🔥 Heap markdown generated: heap-profile-2025-01-21T12-00-00-000Z.md

The markdown output contains a structured analysis of your profile:

# CPU Profile Analysis: cpu-profile-2025-01-21T12-00-00-000Z.pb

## Summary
- Total samples: 1,234
- Duration: 10.5s
- Sample rate: 99 Hz

## Top Hotspots

| Rank | Function | File | Self Time | Total Time |
|------|----------|------|-----------|------------|
| 1 | processRequest | src/handler.js:45 | 23.5% | 45.2% |
| 2 | parseJSON | node_modules/... | 12.3% | 12.3% |
| 3 | renderTemplate | src/views.js:123 | 8.7% | 15.4% |
...

This format is perfect for AI consumption. You can paste it directly into your AI assistant and ask questions like:

• “What are the main performance bottlenecks in this profile?”

• “How can I optimize the processRequest function?”

• “Is there anything unusual about this CPU usage pattern?”

• “Optimize all hot spots.”

Three Markdown Formats

We’ve included three output formats optimized for different use cases:

Summary (Default)

The summary format produces a compact hotspots table - ideal for quick AI triage:

flame run server.js
# or explicitly:
flame run --md-format=summary server.js

This is perfect for dropping into an AI chat and asking, “What should I focus on?” or even “Improve the performance of my application”.

Detailed

The detailed format includes full stack traces and comprehensive statistics:

flame run --md-format=detailed server.js

Use this when you need the AI to understand the complete call hierarchy and suggest architectural improvements.

Adaptive

The adaptive format automatically chooses based on profile complexity:

flame run --md-format=adaptive server.js

Simple profiles get the summary treatment; complex profiles get detailed analysis.

Works with Both CPU and Heap Profiles

Flame captures both CPU and heap profiles concurrently, and markdown analysis is generated for both:

flame run server.js
# Generates:
# cpu-profile-*.pb, cpu-profile-*.html, cpu-profile-*.md
# heap-profile-*.pb, heap-profile-*.html, heap-profile-*.md

For heap profiles, the markdown highlights memory allocation hotspots - perfect for asking your AI assistant to help identify memory leaks or excessive allocations.

Generate from Existing Profiles

Already have pprof files? Generate markdown analysis from them:

# Generate HTML and markdown from existing profile
flame generate cpu-profile.pb

# Use detailed format for comprehensive analysis
flame generate --md-format=detailed cpu-profile.pb

Programmatic API

The new generateMarkdown function is also available in the programmatic API:

const { generateMarkdown } = require('@platformatic/flame')

// Generate LLM-friendly markdown analysis
await generateMarkdown('profile.pb', 'analysis.md', { format: 'summary' })

AI Debugging Workflow

Here’s the workflow we recommend for AI-assisted performance debugging:

1. Profile your application during a realistic workload:

flame run server.js

1. Generate traffic that exercises the slow code paths.

2. Stop profiling (Ctrl-C) to generate all output files.

3. Open the markdown file and paste its contents into your AI assistant.

4. Ask

   • “What are the top 3 things I should optimize?”

   • “Is this JSON parsing overhead normal?”

   • “How can I reduce the time spent in renderTemplate?”

   • “Improve the performance of all the hotspots.”

5. Iterate based on AI changes and re-profile to verify improvements.

Requirements

This feature requires Node.js 22.6.0 or later. We’ve bumped the minimum version to take advantage of ES module interoperability improvements needed for the pprof-to-md integration.

Update flame to the latest version:

npm install -g @platformatic/flame@latest

LLM Performance Optimization Evals

We didn’t just build this feature and hope it works - we ran systematic evaluations to measure how well LLMs can identify and fix performance bottlenecks using pprof-to-md output.

The eval process used Claude Code with Claude Opus 4.5: an orchestrating agent ran benchmarks, collected profiles, spawned optimization subagents with the markdown analysis, applied suggested fixes, and measured results.

Results Summary

Key metrics:

• Correct fix identified: 5/5 (100%)

• Significant improvement achieved: 4/5 (80%)

Highlights

json-bottleneck (144x improvement): The app was parsing a 1MB JSON config file on every request. The profile showed the route handler at 71.1% and readFileSync at 10.7%. Claude immediately identified the issue and moved JSON parsing out of the request handler. Result: 8 req/s → 1,122 req/s.

n-plus-one (12.8x improvement): Sequential async calls in a loop - the classic N+1 query pattern. Claude recognized this from code analysis (CPU profiles don’t capture async wait time) and parallelized with Promise.all(). Result: 41 req/s → 526 req/s.

quadratic-algo (127x latency improvement): O(n²) deduplication using nested loops. Claude suggested using Set for O(1) lookups. Latency dropped from 4,686ms to 37ms with zero errors.

memory-churn (84x latency improvement): Creating 4 intermediate arrays with spread copies. Claude combined all operations into a single loop pass. Latency dropped from 5,239ms to 62ms.

What We Learned

1. Claude correctly identified all 5 performance issues by reviewing the analysis and then applying the necessary patches.

2. All fixes were idiomatic and correct - caching parsed config, pre-compiling regex, parallelizing async operations, single-pass array processing, using Set for O(1) lookups.

3. Latency is often a better success indicator than throughput for optimization evals.

4. The markdown format provides enough information for Claude to understand call paths and identify hotspots in the codebase.

The one failure (regex-hotpath) wasn’t because Claude suggested the wrong fix - it correctly moved the regex pattern outside the loop. The bottleneck was simply masked by I/O operations in that particular workload.

From Profile to Actionable Fix

The real power of LLM-friendly profiles is turning raw data into specific, prioritized recommendations. In one example, we profiled an application, and the AI identified that URL constructor calls accounted for 14.8% of CPU time, garbage collection overhead accounted for 6.7%, and route matching accounted for another 7.1%. But it didn't stop at identifying hotspots; it provided concrete fixes ranked by impact: replace expensive abstractions with simpler alternatives where possible, pre-compute values in loops instead of recalculating them, initialize resources at startup rather than on-demand, and memoize repeated computations. The estimated result? A 20-25% reduction in CPU time from straightforward changes. This is the workflow we envisioned: profile your app, paste the markdown into your AI assistant, and get back a prioritized list of exactly what to fix and how to fix it.

Specifically, this is about TanStack Start. We notified Tanner immediately - they are on it!

Built on pprof-to-md

The markdown generation is powered by our new pprof-to-md library, which we’ve also open-sourced. If you’re building profiling tools and want to add AI-friendly output, check it out.

Get Started

Update to the latest flame and start profiling:

npm install -g @platformatic/flame@latest

flame run your-app.js

Then paste your markdown analysis into your favorite AI assistant and start asking questions. Performance debugging just got a whole lot easier.

Have questions or feedback? Open an issue on GitHub or contact us on our DM.

While evaluating these options and the benchmarks that follow, it’s important to keep in mind what matters most for your context and use case, as there are no “one-size fits all” solutions in software: latency, consistency, or ease of adoption.

ll runtimes completed the benchmarks without any errors. You can find the complete methodology we followed below.

Benchmark Methodology

We benchmarked Next.js 15.5 on AWS EKS across four JavaScript runtimes, each allocated six CPU cores, and the results will be of interest to any engineer building or maintaining server-side Javascript applications with any sort of performance sensitivity.

Three test runs were conducted, rotating the test order, at 1,000 requests per second for 120 seconds each, to illustrate the practical demands these runtimes might face under heavy traffic (think a flash sale in eCommerce, etc).

Infrastructure

All benchmarks ran on AWS EKS (Elastic Kubernetes Service) with the following infrastructure:

• EKS Cluster: 4 nodes running m5.2xlarge instances (8 vCPUs, 32GB RAM each)

• Region: us-west-2

• Load Testing Instance: c7gn.large (2 vCPUs, 4GB RAM, network-optimized)

• Load Testing Tool: Grafana k6

Two critical but often overlooked aspects of effective benchmarking are 1) providing clean and reproducible conditions for each test run, and 2) providing a reliable set-up for others to replicate your experiment. This empowers researchers and developers to verify the results by reproducing them themselves.

To this end, we used shell scripts and the AWS CLI to create on-demand, ephemeral environments for each testing round:

Software Versions

The benchmarks used the following software versions:

All software versions were specified in the Dockerfile to ensure reproducible benchmarks.

Resource Allocation

Each runtime received identical total CPU resources (6 cores) with the following distribution:

Node.js, Bun, and Deno, which each operate as single-threaded processes, were distributed across six single-CPU pods. We configured Watt, our multi-threaded application service built on Node.js, to use two workers per pod across three x2 CPU pods.

Considering the AWS infrastructure costs, these six cores on an m5.2xlarge instance roughly translate to approximately $0.096 per hour. By understanding this cost, you can better evaluate how any latency improvements might affect your budget, as different runtimes could potentially lead to savings by requiring fewer instances to handle the same load (measured in requests per second).

Load Test Configuration

Each runtime was tested with the following k6 configuration:

import http from 'k6/http';
import { check } from 'k6';

export const options = {
 scenarios: {
   constant_arrival_rate: {
     executor: 'constant-arrival-rate',
     duration: '120s',
     rate: 1000,
     timeUnit: '1s',
     preAllocatedVUs: 1000,
     maxVUs: 20000,
   },
 },
};

export default function () {
 const res = http.get(__ENV.TARGET, {
   timeout: "5s",
 });
 check(res, {
   'status is 200': (r) => r.status === 200,
   'response has body': (r) => r.body && r.body.length > 0,
 });
}

This configuration maintained a constant arrival rate of 1,000 requests per second for 120 seconds, resulting in approximately 120,000 requests per test.

Test Protocol

Given that our benchmark harness runs on live cloud services, there is some inherent variability to the data we collected: to ensure a fair comparison and boost confidence in our results, we run them multiple times by rotating the order of service being tested, and we took the extra effort to ‘warm up’ each environment as a part of our test runs.

To start, the Network Load Balancer (NLB) went through a warm-up phase in which all four endpoints received a 60-second warm-up, starting at 10 and reaching up to 500 requests per second, ensuring that AWS Network Load Balancers were properly scaled. Each runtime also received a 20-second pre-test warm-up to stabilize the environment before its respective test.

Test execution spanned 120 seconds at a constant arrival rate of 1,000 requests per second, providing robust data for analysis. A cooldown period of 480 seconds was implemented between each test to allow the system to return to baseline conditions, further ensuring that subsequent tests commenced without residual impact from prior runs.

Finally, the tests were executed in three complete runs with different execution orders to detect positional bias and ensure that each run's performance was accurately assessed as part of our scientific rigor.

Test Orders

Runtime Configurations

Node.js: Standard Next.js standalone server

next start

Bun: Next.js with Bun runtime (requires --bun flag to override shebang)

bun run --bun next start

Without the --bun flag, Bun respects the shebang (#!/usr/bin/env node) in the Next.js binary and executes it with Node.js instead. The --bun flag overrides this behavior to use the Bun runtime.

Deno: Next.js via npm compatibility layer

deno run -A npm:next start

Deno runs Next.js via its npm compatibility layer (npm:next), which allows running npm packages in the Deno runtime.

Watt: Platformatic Watt with 2 workers per pod

wattpm start  # with WORKERS=2

Watt uses SO_REUSEPORT to distribute connections across multiple Node.js worker threads at the kernel level, eliminating the IPC overhead present in traditional cluster-based approaches. Each worker operates with its own event loop while sharing the same listening socket.

Results

Success Rate

All runtimes achieved a 100% success rate, with zero failed requests across all three runs. Each test processed approximately 120,000 requests at the target rate of 1,000 requests per second.

Observations

Latency Distribution

The runtimes fell into distinct performance tiers based on average latency:

• Tier 1 (~11-14ms): Deno and Watt

• Tier 2 (~20ms): Node.js

• Tier 3 (~246ms): Bun

Consistency Across Runs

Deno demonstrated the most consistent performance across different test positions, with a standard deviation of ±1.19ms, indicating minimal predictability risk. Watt exhibited similar consistency at ±1.03ms, offering low operational risk and high reliability. Node.js displayed moderate variance at ±2.42ms, posing a moderate predictability risk that decision-makers should consider when evaluating stability. Although Bun’s absolute variance was higher at ±4.72ms, this represented consistent behavior relative to its average latency, which could translate into higher predictability risk. Understanding these performance metrics in terms of predictability risk can help managers better assess the stability and reliability of deploying specific runtimes.

Test Order Impact

Rotating the test order across three runs helped identify whether the position affected the results. Of the frameworks we tested, all of them performed consistently regardless of where they fell in the testing order, with the notable exception of Node.js itself, which performed best when tested last (see "Run 3", above).

Tail Latency (p99)

The p99 latency provides insight into the worst-case user experience:

• Deno: 101.27ms average p99

• Watt: 114.78ms average p99

• Node.js: 173.84ms average p99

• Bun: 974ms average p99

Throughput

All runtimes successfully handled the target load of 1,000 requests per second with negligible dropped requests. The slight variations in reported requests per second, ranging from 997.94 to 999.96, are within normal measurement variance.

As we reflect on these results, it prompts us to consider future directions for our experiments. For example, which memory-intensive workloads might flip these rankings?

Part of our aim in our open source practice is not just to build products, but to build community, and we’d like to hear from you all: what frameworks and scenarios are most relevant to your work today that you think we should investigate next?

Reproducing these benchmarks

The complete benchmark infrastructure is available at: https://github.com/platformatic/runtimes-benchmarks.

To run the benchmarks:

AWS_PROFILE=<profile-name> ./benchmark.sh

The script creates an ephemeral EKS cluster, deploys all four runtime configurations, executes the load tests, and automatically tears down the infrastructure. Easy as that!

Let us know how this works for you (and perhaps more importantly, if anything doesn’t work for you or if you see results that surprise you…).

Conclusions

The benchmarks showed three distinct performance tiers: Deno and Watt had the lowest average latencies, at approximately 11 to 14 milliseconds; Node.js averaged 20 milliseconds; and Bun exhibited significantly higher latency at approximately 246 milliseconds. (I’m sure Bun’s showing here will surprise many - it surprised us as well.)

All configurations successfully handled the target throughput of 1,000 requests per second, achieving a 100% success rate. These results reflect performance characteristics under the specified test conditions and may vary depending on application workload, infrastructure configuration, and runtime versions. Teams prioritizing sub-15ms latency may shortlist Deno and Watt, with Watt being the natural choice for those who want to stay within the Node.js ecosystem.

What Next?

As we reflect on these results, we’re considering what future direction we’d like to take with our next round of experiments.

Part of our aim in our open source practice is not just to build products, but to build community, and we’d like to hear from you all: what frameworks and scenarios are most relevant to your work today that you think we should investigate next?

Don’t be shy - do drop us a comment here or on LinkedIn (DMs always open!) about what you’d like to see.

Could this approach apply to other Node.js frameworks as well? If so, does a general pattern begin to emerge that performance-sensitive Node.js applications benefit from being run with Watt?

While the jury is still out on if Watt can help every performance-sensitive Node.js app out there, we’ve done another round of benchmarks, this time with React Router, to see if our previous results with Next could be replicated with other frameworks. The answer?

Why yes, of course. I wouldn’t be writing an article about it if I didn’t have some compelling numbers to share 🙂

Methodology

We ran React Router (framework mode), the go-to library for handling server and client-side navigations in React applications, through an extreme load test: 10,000 HTTP requests per second for 120 seconds. The results confirm that Watt delivers notable performance improvements across the Node.js ecosystem.

Why 10x the Load?

Our Next.js benchmarks used 1,000 requests per second. For React Router, we cranked it up to 10,000.

Why? Because React Router is significantly more efficient than Next.js for server-side rendering (“SSR”) workloads. SSR with Next.js involves heavier processing: full React server components, complex routing logic, and more extensive middleware chains. React Router’s server rendering is leaner and faster. At 1,000 req/s, all three configurations (PM2, Watt, Single Node) handled React Router without breaking a sweat, and we couldn’t see meaningful differences.

To properly stress-test the systems and expose the architectural differences, we needed to crank up the dial by an order of magnitude. This extreme load is where the cracks appear and Watt’s advantages become stark.

The Benchmark Setup

We tested three deployment strategies on AWS EKS, all using identical total CPU resources (6 CPUs):

1. Single-CPU pods (6 replicas x 1000m CPU limit each)

2. PM2 multi-worker pods (3 replicas x 2000m CPU limit with 2 PM2 workers each)

3. Watt multi-worker pods (3 replicas x 2000m CPU limit with 2 Watt workers each)

Infrastructure:

• EKS Cluster: 3 nodes running m5.2xlarge instances (8 vCPUs, 32GB RAM each)

• Load Testing: c7gn.2xlarge instance (8 vCPUs, 16GB RAM, network-optimized)

• Load Pattern: k6 with a constant arrival rate of 10,000 requests/second for 120 seconds

• Virtual Users: Up to 20,000 VUs

• Request Timeout: 5 seconds

This is an extreme stress test - 10x the load we used in our Next.js benchmarks. All three configurations hit the 20,000 VU ceiling, confirming we pushed each system to its absolute limits.

Results Summary

Note that the average latency is lower in the single-node deployment due to lower number of errors that Watt has. The server are at saturation point, therefore any additional requests passing through would raise the latency.

Key Findings

Watt vs PM2: A Dramatic Difference

Under extreme load, Watt consistently outshines PM2:

• 45% higher throughput (6,032 vs 4,154 req/s)

• 45% lower failure rate (37.9% vs 69.2%)

• 2.9x more successful responses (467K vs 160K)

• 21% lower average latency (866ms vs 1.1s)

Moreover, Watt showed better tail latency performance. The P95 latency for Watt was significantly lower than that for PM2, reinforcing Watt's reliability under heavy traffic conditions. This emphasis on both average and tail latencies provides a more detailed view of Watts' advantage in real-world conditions.

Throughout our testing, PM2 struggled with the load we were putting it under, dropping over 680,000 iterations and failing on nearly 70% of requests. Watt, using the same CPU resources, maintained only a 37.9% failure rate while processing nearly 3x more successful requests.

The (Un)surprising Single Node Performance

The results from this benchmark support our study from the Next.js benchmarks: PM2’s cluster module architecture (which creates child processes and routes all incoming connections through a single master process via inter-process communication, or IPC) brings substantial overhead (30%) that becomes overwhelming under heavy load.

This observation challenges conventional assumptions about classic Node.js deployments, which often treat multiple processes as necessary to effectively handle high volumes of tasks. It encourages us to reexamine our deployment strategies in similar high-load situations, potentially simplifying architectures that may be plagued by process-management overhead.

Watt vs Single Node

Watt also edges out a Single Node running React Router, most notably when it comes to resilience:

• 3% higher throughput (6,032 vs 5,838 req/s)

• 10% lower failure rate (37.9% vs 42.1%)

• 11% more successful responses (467K vs 420K)

While the margin is smaller than what we’ve seen with Next, we will take a cool 10% reduction in failure rate back to the lab for the next sprint.

Why These Results Matter

It’s worth taking a moment to ground these numbers in what matters to your business (managers, shareholders, etc.) and users (presumably, customers). For example, most developers writing in Node.js, spend a lot of time thinking about how quickly they can load a given page for a user (“latency”, measured in milliseconds), how many users requests they might be able to serve simultaneously (requests per second), and how often those requests fail (failure rate).

Latency, requests per second, and failure rate all have very real business consequences, from user churn to abandoned carts, that have a very real and material cost on the bottom line for your business, especially for teams working at any sort of scale.

Why Watt Thrives at Scale

Watt’s provides the benefits of utilizing multiple workers across multiple cores while avoiding node:cluster and PM2’s management overhead by using SO_REUSEPORT to let the Linux kernel distribute connections directly to workers:

With Watt, there’s no need for master-worker coordination, IPC overhead, or serialization. Instead, worker processes accept connections directly from the OS, using the Linux kernel’s fast, hash-based algorithm to distribute connections evenly across workers.

Conclusion

This latest round of benchmarks with React Router and PM2 confirms what we saw with our previous benchmarks and Next.js: Watts’ SO_REUSEPORT-based architecture delivers significant performance gains over classic Node.js scaling approaches; particularly when compared to PM2 in this instance.

Under extreme load of 10,000 requests per second:

• Watt delivered 45% higher throughput than PM2 (6,032 vs 4,154 req/s)

• Watt processed 2.9x more successful responses than PM2 (467K vs 160K)

• Watt delivered 3% higher throughput than Single Node (6,032 vs 5,838 req/s)

• Watt achieved a 10% lower failure rate than Single Node (37.9% vs 42.1%)

Single Node outperformed PM2, confirming our assessment of cluster module overhead. Watt surpassed both by efficiently using multiple CPU cores without the management overhead of PM2, offering multi-core parallelism, operational benefits, and effective load distribution. Additionally, Watt's architecture complements existing service-mesh and autoscaling patterns, rendering it a strategic fit for modern distributed systems. By integrating smoothly, it increases performance while in accordance with the architects’ mental models and simplifying deployment strategies.

Crucially, Watt’s main thread continuously monitors all worker threads, detecting and recovering from catastrophic event loop situations - blocked loops, memory leaks, or unresponsive workers. When a worker becomes unhealthy, Watt gracefully restarts it without affecting other workers or requiring pod termination. With a Single Node, a blocked event loop means your entire pod is down until Kubernetes notices and restarts it. With Watt, the main thread catches the problem early, restarts just that worker, and your service stays available.

The takeaway: at scale, PM2 and single-process deployments underperform. Watt delivers multi-core power without traditional tradeoffs for Node.js apps.

Getting Started with Watt

Watt is open-source and easy to implement, and you don’t need to be serving 10,000 requests per second to benefit from adopting it, either. In fact, much of our time is spent on good, old-fashioned developer experience. One team cut its P95 latency by 30% in just one afternoon by simply integrating Watt into its existing system during an onsite architecture workshop with us. (If you have a particularly business-critical or thorny app you think Watt could, we love doing these - drop me a message on LinkedIn!)

When configuring Watt with your project, we recommend setting the number of workers to match your CPU allocation to start with. From there, you simply deploy using the same Kubernetes setup you're already using and you’re off to the races.

For a complete guide, see our documentation: https://docs.platformatic.dev/docs/guides/deployment/nextjs-in-k8s

The benchmark code is available at: https://github.com/platformatic/k8s-watt-performance-demo.

If you have questions or want help getting Watt set-up in your environment, contact us at info@platformatic.dev or connect with Luca or Matteo on LinkedIn. We always love hearing from the community!

The Challenge: Sharing Performance Insights

As Node.js developers, we've all been there: you've discovered a performance issue, captured some data, and now need to share it with your team. But how do you effectively communicate what you're seeing? Screenshots don't tell the whole story, and setting up monitoring dashboards for everyone isn't always practical.

Previously, Watt Admin provided real-time monitoring—perfect for live debugging. But what about post-mortem analysis? What if you need to capture a performance profile during a specific scenario, or share detailed metrics with a colleague who isn't running your application?

That's where Recording Mode comes in.

What's New: Record, Profile, Analyze

With the new recording capabilities, Watt Admin can now:

• 📹 Record complete sessions: Capture all metrics and performance data over time

• 🔥 Generate flame graphs: Profile CPU usage or heap allocation to identify bottlenecks

• 📦 Create offline bundles: Package everything into a single HTML file

• 🤝 Share effortlessly: Send the bundle to anyone—no setup required

How It Works

Metrics Recording

At the core of every recording session is comprehensive metrics collection. Watt Admin captures a complete picture of your application's health and performance:

Memory Metrics

• RSS (Resident Set Size): Total process memory usage

• Heap Usage: Total heap, used heap, new space, and old space—essential for tracking memory leaks and garbage collection behavior

CPU & Event Loop

• CPU Usage: Per-thread CPU utilization percentage

• Event Loop Utilization (ELU): How busy your event loop is the key indicator of Node.js application health

HTTP Performance

• Request Count & RPS: Total requests and throughput over time

• Latency Percentiles: P90, P95, and P99 response times to understand your tail latency

HTTP Client (Undici)

• Connection Pool Stats: Idle, open, pending, queued, and active sockets

• Track how your application communicates with external services

Additional Metrics

• WebSocket Connections: Active WebSocket connection count

• Kafka Metrics: Produced/consumed messages, producers, consumers, and DLQ stats (if using Kafka)

• Event Loop Resources: Active handles and requests in the event loop

All metrics are sampled every second and stored for the duration of your recording session (up to 600 data points). When you stop recording, this entire metrics history is bundled into the HTML file, giving you a complete timeline to analyze.

CPU Profiling

Identify performance bottlenecks by visualizing where your application spends CPU time:

watt-admin --record --profile cpu

Run your application through the scenario you want to analyze, then press Ctrl+C. Watt Admin will:

1. Stop profiling all services in your runtime

2. Collect CPU flame graph data

3. Bundle all metrics and the flame graph into a single HTML file

4. Automatically open it in your browser

The resulting flame graph shows you exactly which functions are consuming CPU cycles, making it easy to spot optimization opportunities.

Heap Profiling

Track down memory leaks and understand allocation patterns:

watt-admin --record --profile heap

Heap profiling reveals:

• Which parts of your code allocate the most memory

• Memory allocation patterns over time

• Potential memory leak sources

• Object retention paths

Perfect for debugging those mysterious memory issues that only appear under specific conditions.

Real-World Use Cases

Debugging Production Issues Locally

Reproduce a production issue in your local environment, record a session with CPU profiling, and share the complete analysis with your team. No need for everyone to set up the same environment—they can explore the flame graph and metrics directly from the HTML bundle.

Performance Reviews

Before merging a significant change, record a profiling session to demonstrate its performance characteristics. Attach the HTML file to your pull request so reviewers can see the real-world impact of your optimizations.

Team Knowledge Sharing

Found an interesting performance pattern? Record it and share the bundle in Slack or your team chat. Your colleagues can explore the interactive flame graph and metrics without any setup.

Client Reporting

Need to show a client why their application is slow? Generate a recording with clear flame graphs that visually demonstrate the bottlenecks. The self-contained HTML makes it easy to share professional performance analysis.

The Technical Details

Recording mode leverages Platformatic's built-in profiling capabilities:

• Automatic service discovery: Profiles all applications in your Platformatic runtime

• Standards-based profiling: Uses V8's built-in CPU and heap profilers

• pprof format: Stores profiling data in the industry-standard pprof format

• Interactive visualization: Uses react-pprof for exploring flame graphs

• Complete capture: Embeds metrics, logs, and profiling data in window.LOADED_JSON

The generated HTML bundle is truly self-contained—it includes:

• All JavaScript, CSS, and assets inlined

• Complete metrics history from the recording session

• Flame graph data for all profiled services

• Interactive UI for exploring the data

No external dependencies. No network requests. Just open and explore.

Getting Started

Prerequisites

Important: Watt Admin is designed specifically for applications running on Platformatic Watt. If you don't have a Watt application yet, follow the Quick Start Guide to create one.

Installation

Install Watt Admin globally:

npm i @platformatic/watt-admin -g

Or use it directly with npx:

npx wattpm admin --record --profile cpu

Basic Workflow

1. Start a recording session:

    watt-admin --record --profile cpu
   

2. Run your application through the scenario you want to analyze

3. Stop recording by pressing Ctrl+C

4. Analyze the automatically-opened HTML bundle

5. Share the bundle file with your team

Under the Hood: How Recording Works

When you start Watt Admin with --record, here's what happens:

1. Discovery: Watt Admin discovers your Platformatic runtime using the RuntimeApiClient

2. Connection: Connects to the runtime's admin API

3. Profiling start: Calls startApplicationProfiling() on each application

4. Metrics collection: Continuously collects metrics at regular intervals

5. User interaction: You use your application while profiling runs

6. SIGINT handling: When you press Ctrl+C, graceful shutdown begins

7. Profiling stop: Calls stopApplicationProfiling() to get the profiling data

8. Data bundling: Writes profiling data (.pb files) and embeds everything in HTML

9. Auto-launch: Opens the generated bundle in your default browser

The resulting file lives in web/frontend/dist/index.html and contains everything needed for offline analysis.

What's Next

Recording mode is just the beginning. We're exploring additional profiling capabilities:

• Custom time ranges: Record specific time windows

• Comparison mode: Compare multiple recordings side-by-side

• Export formats: Additional export options for different tools

• Annotations: Mark specific events during recording

We'd love to hear your feedback! Try recording mode and let us know what you think.

Try It Today

Recording and profiling are available now in Watt Admin. Update to the latest version:

npm i @platformatic/watt-admin -g

Then start profiling:

watt-admin --record --profile cpu

Happy profiling! 🔥

Resources

• Watt Admin on GitHub

• Platformatic Documentation

• Watt Admin Introduction

Get Involved

Have questions or feedback? Connect with us:

• GitHub Issues

• Discord Community

• Twitter

For teams running Python ASGI applications alongside Node.js services, this release enables new application types, including real-time dashboards, live data feeds, WebSocket-powered chat systems, progressive file uploads, and server-sent events—all while maintaining the Python-Node.js integration you've come to expect.

If you're new to @platformatic/python-node, it's a module that lets you run Python ASGI applications (such as FastAPI, Starlette, or Django) directly in Node.js processes. No separate Python server, no HTTP proxy overhead, no complex deployment setup.

What's New in v2.0.0

This release brings four major enhancements that align @platformatic/python-node with modern ASGI server capabilities:

HTTP Response Streaming

The new handleStream() method enables streaming of HTTP responses. Instead of buffering the entire response body before returning it to Node.js, chunks are processed incrementally as they arrive from your Python application. This reduces memory usage for large responses and provides immediate access to response headers before the body completes.

Each chunk will be pulled from Python when Node.js is ready to operate on it. Python will wait until a chunk is requested before continuing to process the ASGI handler. This architecture provides proper backpressure between the two languages and fully async on both ends so either language can do operate on other things while waiting for the other.

const res = await python.handleStream(req)

// Headers available immediately
console.log(res.status) // 200
console.log(res.headers.get('content-type'))

// Body consumed via AsyncIterator as chunks arrive
for await (const chunk of res) {
  console.log(chunk.toString())
}

HTTP Request Streaming

In addition to streaming responses, you can now stream request bodies to Python. This is essential for handling large file uploads, processing data progressively, or implementing custom streaming protocols.

Each write returns a promise to provide backpressure from Python so Node.js doesn't write too much data if Python is not consuming it fast enough. It uses an internal buffer, if the buffer has enough space for the write it will resolve immediately, otherwise it will wait for space to be made available.

const req = new Request({
  method: 'POST',
  url: '/upload',
  headers: { 'Content-Type': 'application/octet-stream' }
})

// Dispatch request and write body concurrently
const [res] = await Promise.all([
  python.handleStream(req),
  async () => {
    // Stream chunks to Python
    await req.write(chunk1)
    await req.write(chunk2)
    await req.write(chunk3)
    await req.end()
  }()
])

Bidirectional WebSocket Support

Full WebSocket support means your Python ASGI applications can now handle persistent, bidirectional connections. Whether you're building a chat application, live dashboard, or multiplayer game, you can implement the WebSocket logic in Python while integrating seamlessly with your Node.js infrastructure.

const req = new Request({
  url: '/ws',
  websocket: true
})

const res = await python.handleStream(req)

// Send messages to Python
await req.write('Hello from Node.js!')

// Receive messages from Python
for await (const chunk of res) {
  console.log('Received:', chunk.toString())
}

ASGI 3.0 Protocol Implementation

Under the hood, v2.0.0 implements the ASGI 3.0 protocol specification for both HTTP and WebSocket communication. This ensures compatibility with the entire Python async ecosystem, including FastAPI's StreamingResponse, Starlette's WebSocket endpoints, and any other ASGI-compliant framework.

Key Benefits

• Lower Memory Footprint: Stream large responses without buffering everything in memory
• Faster Time-to-First-Byte: Access response headers immediately, before body writing even begins
• Real-Time Capabilities: Build WebSocket applications with bidirectional communication
• Better Resource Utilization: Process chunks as they arrive instead of waiting for completion
• Backward Compatible: Existing code using handleRequest() continues to work almost completely unchanged -- the single breaking change is no more req.body setter/getter

HTTP Streaming in Action: Server-Sent Events with FastAPI

One of the most powerful use cases for HTTP streaming is Server-Sent Events (SSE), which enable servers to push real-time updates to clients over a standard HTTP connection. Let's build a live monitoring dashboard that streams system metrics from Python to Node.js.

Here's a FastAPI application that generates streaming metrics:

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
import asyncio
import json
import random
from datetime import datetime

app = FastAPI()

async def generate_metrics():
    """Generate fake system metrics as server-sent events"""
    while True:
        # Simulate collecting system metrics
        metrics = {
            'timestamp': datetime.now().isoformat(),
            'cpu_usage': random.uniform(20, 80),
            'memory_usage': random.uniform(40, 90),
            'active_connections': random.randint(10, 100),
            'requests_per_second': random.randint(50, 500)
        }

        # Format as SSE event
        data = f'data: {json.dumps(metrics)}\n\n'
        yield data.encode()

        # Send update every second
        await asyncio.sleep(1)

@app.get('/metrics/stream')
async def stream_metrics():
    """Endpoint that streams real-time metrics"""
    return StreamingResponse(
        generate_metrics(),
        media_type='text/event-stream',
        headers={
            'Cache-Control': 'no-cache',
            'Connection': 'keep-alive'
        }
    )

@app.get('/health')
async def health_check():
    """Standard health check endpoint"""
    return {'status': 'healthy', 'version': '1.0.0'}

Now let's consume this stream from Node.js:

import { Python, Request } from '@platformatic/python-node'

const python = new Python({
  docroot: './python-apps',
  appTarget: 'metrics_app:app'
})

async function monitorMetrics() {
  const req = new Request({
    method: 'GET',
    url: 'http://localhost/metrics/stream'
  })

  console.log('Connecting to metrics stream...')
  const res = await python.handleStream(req)

  console.log(`Status: ${res.status}`)
  console.log(`Content-Type: ${res.headers.get('content-type')}`)
  console.log('\nReceiving metrics:\n')

  // Process metrics as they arrive
  for await (const chunk of res) {
    const lines = chunk.toString().split('\n')

    for (const line of lines) {
      if (line.startsWith('data: ')) {
        const data = JSON.parse(line.slice(6))

        console.log(`[${data.timestamp}]`)
        console.log(`  CPU: ${data.cpu_usage.toFixed(1)}%`)
        console.log(`  Memory: ${data.memory_usage.toFixed(1)}%`)
        console.log(`  Connections: ${data.active_connections}`)
        console.log(`  RPS: ${data.requests_per_second}`)
        console.log()
      }
    }
  }
}

monitorMetrics().catch(console.error)

How It Works

The streaming implementation leverages the interaction between Python's async generators and Node.js's AsyncIterator pattern:

1. Python Side: FastAPI's StreamingResponse accepts an async generator that yields chunks. Each yield dispatches data to Rust via a tokio DuplexStream.

2. ASGI Bridge: The Rust-based ASGI implementation receives http.response.body events with the more_body flag, queuing chunks as they arrive.

3. Node.js Side: The handleStream() method returns a Response object that implements the AsyncIterator protocol. Each iteration of for await...of receives the next chunk.

This architecture means Node.js can start processing data the moment Python sends the first chunk—no waiting for the complete response. But it also means bidirectional backpressure, so each side can only operate as fast as the other and will yield back to its respective event loop when there's no work to be done.

Real-World Use Cases for HTTP Streaming

Beyond metrics dashboards, HTTP streaming enables:

• Large File Downloads: Stream files from Python (e.g., generated reports, media files) without loading them entirely into memory
• AI/ML Model Outputs: Stream generated content from language models or other AI systems
• Progressive Data Processing: Stream database query results or CSV processing as rows are processed
• Video/Audio Streaming: Deliver media content with Python processing (transcoding, filtering) and Node.js delivery

WebSocket Support: Building Real-Time Applications

While HTTP streaming works well for server-to-client communication, WebSockets provide full bidirectional real-time channels. This is useful for chat applications, collaborative editing, live gaming, and scenarios where both client and server need to send messages independently.

Let's build a conversational AI assistant with FastAPI WebSockets:

from fastapi import FastAPI, WebSocket, WebSocketDisconnect
import json

app = FastAPI()

@app.websocket('/ws/assistant')
async def assistant_endpoint(websocket: WebSocket):
    await websocket.accept()

    # Send welcome message
    await websocket.send_text('Hello! I am your AI assistant. Ask me anything or try /help for commands.')

    try:
        while True:
            # Receive message from client
            message = await websocket.receive_text()

            # Simple routing based on message content
            if message.startswith('/help'):
                response = 'Available commands: /help, /status, /about, or ask any question'
            elif message.startswith('/status'):
                response = 'System status: All services operational'
            elif message.startswith('/about'):
                response = 'AI Assistant v1.0 - Powered by Python and Node.js'
            elif message.lower() in ['hi', 'hello', 'hey']:
                response = 'Hello! How can I help you today?'
            elif 'time' in message.lower():
                from datetime import datetime
                response = f'The current time is {datetime.now().strftime("%H:%M:%S")}'
            else:
                # Echo back with a simulated AI response
                response = f'You said: "{message}". I am processing your request...'

            # Send response back to client
            await websocket.send_text(response)

    except WebSocketDisconnect:
        pass  # Client disconnected

Now let's interact with this assistant from Node.js:

import { Python, Request } from '@platformatic/python-node'

const python = new Python({
  docroot: './python-apps',
  appTarget: 'assistant_app:app'
})

async function runAssistant() {
  // Create WebSocket request
  const req = new Request({
    url: 'http://localhost/ws/assistant',
    websocket: true
  })

  console.log('Connecting to AI Assistant...\n')
  const res = await python.handleStream(req)

  // Messages to send to the assistant
  const messages = [
    'Hello',
    '/help',
    '/status',
    'What is the time?',
    'Tell me about yourself'
  ]

  let messageIndex = 0

  // Clean for-await loop: read response, then write next message
  for await (const chunk of res) {
    const response = chunk.toString()
    console.log(`Assistant: ${response}\n`)

    // Send next message if we have more
    if (messageIndex < messages.length) {
      const nextMessage = messages[messageIndex]
      console.log(`You: ${nextMessage}`)
      await req.write(nextMessage)
      messageIndex++
    } else {
      // No more messages, close the connection
      console.log('Closing connection...')
      await req.end()
      break
    }
  }
}

runAssistant().catch(console.error)

How WebSocket Communication Works

The example above demonstrates the clean request-response pattern enabled by WebSockets:

1. Connection Establishment: Node.js creates a Request with websocket: true. Python receives a scope with type: 'websocket' and sends websocket.accept to establish the connection.

2. Bidirectional Messaging: The for-await loop reads from the Python server, and each iteration writes a new message back:

   • Python → Node.js: Python's await websocket.send_text() delivers data to Node.js via the AsyncIterator
   • Node.js → Python: await req.write(data) sends data to Python via websocket.receive_text()

3. Clean Loop Pattern: Unlike the background task pattern often seen in WebSocket examples, this approach uses a single synchronous-style loop where each receive is followed by a send, making the flow easy to understand and debug.

4. Connection Lifecycle: Either side can close the connection. Python receives websocket.disconnect events, while Node.js closes via req.end().

Real-World WebSocket Use Cases

WebSocket support enables full-stack teams to build:

• Conversational AI Assistants: Build chatbots and AI assistants in Python, exposing them via WebSocket for real-time conversations
• Real-Time Chat and Messaging: Interactive chat backends where Python handles message routing and business logic
• Live Data Feeds: Stream stock prices, sports scores, IoT sensor data, or live metrics with bidirectional control
• Interactive Commands: Command-line style interfaces where users send commands and receive structured responses
• Gaming: Real-time multiplayer game state synchronization with player actions and server updates
• Live Customer Support: Real-time support chat with Python AI integration for automated responses

Real-World Integration Scenarios for Full-Stack Teams

The combination of streaming and WebSocket support enables several integration patterns for teams using both Python and Node.js:

Python ML Models with Real-Time Inference

Run machine learning inference in Python (using PyTorch, TensorFlow, or transformers) and expose results via WebSocket for real-time predictions. Your Node.js API gateway can manage authentication and routing while Python handles the heavy computation.

@app.websocket('/ws/inference')
async def ml_inference(websocket: WebSocket):
    await websocket.accept()
    model = load_model()  # Load your ML model

    while True:
        data = await websocket.receive_json()
        result = model.predict(data['input'])
        await websocket.send_json({'prediction': result})

Progressive Data Processing

Stream large dataset processing results back to Node.js as they're computed, enabling progress bars, partial result display, or early termination:

@app.get('/process/dataset')
async def process_dataset():
    async def process_stream():
        for batch in dataset.batches(size=1000):
            result = process_batch(batch)
            yield json.dumps(result).encode() + b'\n'

    return StreamingResponse(process_stream())

Hybrid API Gateway

Use Node.js as your API gateway for authentication, rate limiting, and routing, while leveraging Python's rich ecosystem for specific endpoints that need streaming or WebSocket capabilities.

Existing Python Tools with WebSocket Interfaces

Wrap existing Python command-line tools or libraries with FastAPI WebSocket endpoints, making them accessible to your Node.js infrastructure without rewriting them.

Getting Started and Migration Guide

Installation

Upgrade to v2.0.0 via npm:

npm install @platformatic/python-node@latest

Or with yarn:

yarn add @platformatic/python-node@latest

Choosing Between handleRequest() and handleStream()

The API now offers two methods for handling requests:

Use handleRequest() when:

• Response bodies are small and fit comfortably in memory
• You would need to buffer the complete response body anyway
• Backward compatibility with existing code is required

const res = await python.handleRequest(req)
console.log(res.body.toString()) // Body available immediately

Use handleStream() when:

• Responses are large or potentially unbounded
• You need access to headers before the body completes
• Implementing Server-Sent Events or streaming protocols
• Building WebSocket applications
• Memory efficiency is important

const res = await python.handleStream(req)
console.log(res.body) // `undefined` for streams!
console.log(res.status) // Headers available immediately
for await (const chunk of res) {
  // Process chunks incrementally
}

Migration Checklist

Existing applications using handleRequest() should mostly all continue to work without changes. The one exception being that the req.body setter and getter are no longer available. To adopt streaming:

1. Identify endpoints that would benefit from streaming (large responses, real-time data, WebSockets)
2. Switch those specific endpoints to use handleStream()
3. Update response handling to use for await...of iteration
4. Test thoroughly, especially error handling during streaming
5. Monitor memory usage to verify streaming benefits

Documentation and Resources

• Full documentation: python-node GitHub repository
• ASGI specification: asgi.readthedocs.io
• FastAPI streaming: FastAPI Advanced User Guide
• FastAPI WebSockets: FastAPI WebSockets

Conclusion

The addition of HTTP streaming and WebSocket support in @platformatic/python-node v2.0.0 expands what's possible for full-stack teams building with Python and Node.js. The existing integration now extends to real-time, high-performance use cases that previously required complex workarounds.

Whether you're building live dashboards, chat applications, progressive data processing pipelines, or exposing Python ML models via WebSocket APIs, v2.0.0 provides the necessary foundation while maintaining backward compatibility with existing code.

The implementation follows the ASGI 3.0 specification, ensuring compatibility with the Python async ecosystem, including FastAPI, Starlette, Django Channels, and other ASGI-compliant frameworks. Combined with running Python directly in Node.js processes, this release provides a practical option for teams looking to leverage both ecosystems.

Try out v2.0.0 today and share your feedback. If you encounter issues or have questions, please open an issue on our GitHub repository.

We continued running the tests and analyzing the results, but they consistently failed to match our production experience. The variance was high, the sample sizes were small, and we lacked confidence that we were measuring what we intended to measure.

We decided to revisit our approach fundamentally. Not just to make @platformatic/kafka faster, but to ensure we were testing it correctly in the first place.

It turned out our methodology was flawed. Correcting that led us down a path that resulted in substantial performance improvements.

Performance Summary

Here's where we ended up with v1.21.0:

• Producer (Single Message): 92,441 op/sec—48% faster than KafkaJS

• Producer (Batch): 4,465 op/sec—53% faster than KafkaJS

• Consumer: 159,828 op/sec—9% faster than our previous version

That single-message producer number represents a 223% improvement over v1.16.0.

Benchmark Methodology Issues

When we initially ran benchmarks for our first blog post, we used what appeared to be a standard approach: send messages, measure elapsed time, and calculate operations per second.

The problem was that we were only capturing timing measurements every 100 messages. For the rdkafka-based libraries, we weren't properly waiting for delivery reports. We were essentially sending messages without tracking when they were actually acknowledged. The timing measurements were inconsistent and unreliable.

Our initial results reflected these methodological flaws:

┌─────────────────────────────────────────────┬─────────┬────────────────┬───────────┐
│ Library                                     │ Samples │         Result │ Tolerance │
├─────────────────────────────────────────────┼─────────┼────────────────┼───────────┤
│ node-rdkafka                                │     100 │  68.30 op/sec  │ ± 67.58 % │
│ @confluentinc/kafka-javascript (rdkafka)    │     100 │ 220.26 op/sec  │ ±  1.24 % │
│ KafkaJS                                     │     100 │ 383.82 op/sec  │ ±  3.91 % │
│ @platformatic/kafka                         │     100 │ 582.59 op/sec  │ ±  3.97 % │
└─────────────────────────────────────────────┴─────────┴────────────────┴───────────┘

Consider the variance on node-rdkafka: ±67.58%. This level of variance indicates unreliable measurements. Additionally, only 100 samples provided insufficient statistical confidence.

We completely rewrote the benchmark suite with the following improvements:

Per-operation timing: Instead of sampling every 100 messages, we now measure timing for each individual operation. This provides significantly more granular data and much lower variance.

Proper delivery tracking: For rdkafka-based libraries, we now send a message and wait for its specific delivery report before timing the next operation. This ensures accurate per-message timing.

Substantially larger sample sizes: We increased from 100 samples to 100,000 for most tests. While this increases execution time, the results are statistically meaningful.

When we re-ran the tests with the corrected methodology, the numbers improved dramatically across all libraries—particularly for the rdkafka-based ones:

The libraries themselves hadn't changed—we had simply started measuring them accurately.

However, these improved benchmarks revealed performance issues in our own implementation that required attention.

Identifying and Addressing Performance Bottlenecks

With proper measurements in place, we could precisely identify where @platformatic/kafka was spending its time and where optimization opportunities existed.

Our v1.16.0 performance numbers were respectable—28,596 op/sec for single messages—but the ±34.18% variance was concerning. In production environments, variance of this magnitude translates to unpredictable latency spikes, which contradicts our design goals.

We began systematic profiling. The first bottleneck that became apparent was CRC32C computation. We were calculating checksums for every message (as required by the Kafka protocol) using a pure JavaScript implementation. While functional, it exhibited both low throughput and high variance.

We integrated @node-rs/crc32, a native Rust implementation (#126). The improvement was immediate and substantial—not just in throughput, but in consistency. The timing became significantly more predictable.

@baac0 contributed a pull request that refactored error handling in request serialization (#154). Initially, we viewed this primarily as code cleanup. This assessment proved incorrect. By handling errors asynchronously rather than blocking the serialization path, we eliminated an entire category of event loop blockages. Throughput increased substantially.

@jmdev12 identified a subtle bug in our metadata request handling (#144). We were improperly mixing callbacks in kPerformDeduplicated, which occasionally caused requests to hang or retry unnecessarily. Resolving this issue significantly improved connection handling reliability.

We also introduced a handleBackPressure option (#127) to provide users with control over flow control behavior. While the Kafka protocol includes back-pressure mechanisms where brokers can signal clients to slow down, we weren't handling this consistently. The new option allows fine-tuning of how the client responds to back-pressure signals.

After implementing these changes, we re-ran the benchmarks.

From 28,596 to 92,441 op/sec—a 223% improvement. More significantly, observe the variance reduction to ±1.05%.

Batch Processing Performance

Single-message performance is important for real-time event streaming, but many Kafka workloads involve bulk data pipelines sending hundreds or thousands of messages in batches.

Our batch performance was already competitive in v1.16.0—3,779 op/sec for batches of 100 messages. With the same optimizations applied, we observed improvements here as well:

This represents an 18% improvement to 4,465 op/sec. More significantly, we now outperform KafkaJS by 53% in batch scenarios. This performance difference becomes substantial when processing millions of messages daily.

Consumer Performance Improvements

Our consumer implementation was already performing well in initial tests, but we discovered several bugs. Partition assignment logic had issues (#138), and lag computation had edge cases that could produce incorrect results (#153).

Addressing these bugs improved performance from 146,862 to 159,828 op/sec:

The 9% throughput improvement is valuable, but the ±1.75% variance is more significant. This compares favorably to node-rdkafka's ±19.16% and the Confluent clients' ±18-24% variance. Consistent performance is often more valuable than peak throughput in production environments.

Performance Architecture

We frequently receive questions about how a pure JavaScript implementation can outperform native bindings to librdkafka. The answer lies not in a single optimization, but in the cumulative effect of multiple architectural decisions:

Minimal buffer copying: Every buffer allocation and copy adds garbage collection pressure. We designed the entire protocol handling layer to work with buffer slices and views wherever possible. When processing 90,000+ messages per second, avoiding unnecessary allocations has significant impact on both throughput and latency consistency.

Direct protocol implementation: There is no abstraction layer between application code and the wire protocol. Less indirection means fewer function calls, reduced stack manipulation, and more predictable performance characteristics. This also allows us to optimize hot paths without architectural constraints.

Non-blocking event loop usage: Node.js performs optimally when used according to its design principles—specifically, with async operations that don't block. The error handling refactor was particularly impactful. We had been blocking on error serialization in several code paths, and eliminating these blocks substantially reduced latency spikes.

Proper stream implementation: Node.js streams provide built-in back-pressure management when used correctly. When network sockets fill up, the stream pauses writes. When consumers cannot keep up, the fetch loop pauses. This keeps memory usage predictable and prevents unbounded memory growth.

Hot path optimization: Operations like CRC32C checksums, murmur2 partition hashing, and varint encoding execute for every single message. We profiled these operations extensively, optimized them, and profiled again. The migration to native CRC32C via Rust was the largest single improvement, but numerous smaller optimizations compound significantly at scale.

It's worth noting that librdkafka implements similar optimizations—it's exceptionally well-optimized C code. However, it must cross the Node.js/C++ boundary for every operation, and that boundary crossing carries measurable overhead. By remaining in JavaScript, we avoid that overhead entirely.

The Journey Continues

What started as a nagging doubt about our benchmark methodology turned into something far more valuable: a comprehensive understanding of our library's performance characteristics and a 223% improvement in single-message throughput.

The lessons from this experience are worth highlighting. First, measurement matters—flawed benchmarks don't just waste time, they obscure real performance issues. By fixing our methodology, we exposed bottlenecks we hadn't even known existed. Second, community contributions matter tremendously. The PRs from our contributors didn't just fix bugs—they fundamentally improved our throughput and reliability. Third, consistency matters as much as peak performance. Reducing variance from ±34% to ±1% means your p99 latencies become predictable, which is what production systems actually need.

The results speak for themselves: @platformatic/kafka v1.21.0 now delivers 92,441 op/sec for single messages and 159,828 op/sec for consumption, with variance under ±2% across all scenarios. It's 99% pure JavaScript library and yet it outperforms libraries built on highly optimized C code.

If you're building Node.js applications where Kafka performance matters, we encourage you to evaluate @platformatic/kafka:

npm install @platformatic/kafka

Run the benchmarks on your own infrastructure—we've published the complete test suite in BENCHMARKS.md. Test it against your workload patterns. And if you find issues or have optimization ideas, we welcome contributions at github.com/platformatic/kafka. After all, that's how we got here.

All benchmarks executed on an M2 Max MacBook Pro with Node.js 22.19.0 against a three-broker Kafka cluster. Results may vary based on hardware and network configurations, though relative performance characteristics should remain comparable.

We'll start by examining the complications of running this powerful framework in your own environment, and get under the hood (and I mean, down to the kernel) about why they happen.

Then, we'll walk you through the approach we took with Watt to solve them, and what it means for you if you happen to run Next.js on any other Node.js CPU-bound workload on-prem.

(And if you're just here for the benchmarks, feel free to scroll past all the technical bits 🙂.)

The Fundamental Problems of Scaling Next (and Node) in Kubernetes

If you're running Node.js at any modicum of scale, specifically in containers, you may know some version of this story:

Traffic spikes hit during peak hours, and suddenly, some pods are maxed out at 100% CPU while others sit at 30% utilization.

Requests start timing out. Your monitoring dashboard lights up red. Users see loading spinners instead of content, and your error rate climbs to 8%.

So you over-provision. You add 50% more pods than you need to handle the uneven load distribution. Your cloud bill grows, but at least most requests succeed.

Except now you're paying for infrastructure that sits idle most of the time, and during the next traffic surge, you're back to the same problem - just with more pods experiencing the same uneven distribution.

Meanwhile, your median latency hovers around 182ms. That doesn't sound terrible until you realize each page load makes multiple API calls. Three sequential calls at 180ms each mean users wait over half a second for basic interactions. In e-commerce, that's the difference between a sale and an abandoned cart. In SaaS, it's the difference between a delighted user and a churned customer.

This point I'm trying to (not so subtly) emphasize is that this isn't just a performance problem. It's a revenue problem. Each failed request during peak traffic is a lost transaction. Every 100ms of latency measurably reduces conversion rates. And all that over-provisioned infrastructure? That's profit margin burning in idle CPU cycles.

And yes, at the end of this, I'm going to tell you how Watt can fix it all, and show you some numbers to prove it. Strap in, it's time to go spelunking into the internals of Node.js and Linux.

Under the Hood

Ok. Time to get into the technical details.

For over a decade, the Node.js community has relied on two main approaches for scaling applications across multiple CPU cores:

1. The cluster module (and PM2)

When Node.js introduced the cluster module in 2011, it seemed perfect: fork multiple worker processes, share a server port, and let the master process distribute connections using round-robin load balancing. Tools like PM2 made this even easier.

But there's a hidden cost. The cluster architecture requires the master process to act as an internal load balancer - accepting every connection and transferring it to workers via IPC (inter-process communication). This introduces approximately 30% overhead as every request passes through this coordination layer.

2. Horizontal scaling with single-CPU pods

In Kubernetes environments, the conventional wisdom became: deploy single-CPU pods behind a load balancer. Simple, stateless, easy to scale. But this approach creates a critical problem for frameworks that can't implement early request rejection.

The Early Rejection Problem

Here's the issue: Once a request enters Node.js's event loop queue, it cannot be rejected until processing begins:

Request → TCP Accept → Event Loop Queue → [Wait] → Process → 503 (Too Late)
                     ↑
                Cannot reject here

This causes requests to pile up during overload, consuming memory and increasing latency for everyone. An ideal server would reject new requests immediately with a 503 before accepting them, allowing load balancers to route traffic elsewhere. But Node.js's event loop architecture makes this remarkably difficult.

Why Next.js Makes This Worse

Frameworks like Next.js that rely on React server-side rendering (SSR) fundamentally cannot implement early 503 responses. They need the full request context before they can even determine what to do:

1. Request Context Required: SSR needs headers, cookies, and query params before rendering

2. Dynamic Route Matching: Next.js must accept the connection to determine which page to execute

3. Data Fetching Dependencies: Server components require the request to be in-flight, parallelizing I/O but postponing the CPU-bound activity (learn more why this cause Event Loop blocking).

4. Middleware Execution: Next.js middleware runs after request acceptance, not before

By the time Next.js knows it's overloaded, the request is already queued and consuming resources.

The Compounding Effect

When you combine this with traditional scaling approaches, the problems multiply:

• With cluster/PM2: Every request pays the ~30% IPC overhead, even when the server isn't overloaded.

• With single-CPU pods: Round-robin distribution creates isolated queues where load imbalances compound. One pod might be drowning while another sits idle, but they can't share work.

What we needed was a way to scale across multiple cores without the coordination overhead of cluster, while also enabling better statistical distribution of load than isolated single-CPU pods.

How We Solved This (with Watt)

We built Watt as an "application server" for Node.js to fix these fundamental issues of scaling Node.js in containerized environments.

Here's what we achieved running Next.js with Watt on AWS EKS under sustained load of 1,000 requests per second:

That's 93.6% faster median latency and near-perfect reliability compared to the standard approaches for scaling Node.js applications.

These results come from production-grade benchmarks on real Next.js applications running on Kubernetes, comparing three common deployment strategies with identical total CPU resources (6 CPUs each). All configurations were tested under the same sustained load pattern, and Watt consistently outperformed both traditional approaches.

Under the Hood (again)

Watt leverages a feature of the Linux kernel that allows us to distribute connections across multiple Node.js processes with zero coordination overhead: SO_REUSEPORT. This listen() option allows us to eliminate the ~30% performance tax that PM2 and the cluster module impose through IPC-based load balancing.

The core idea is elegantly simple: instead of having a master process coordinate workers, let the Linux kernel handle load distribution directly via SO_REUSEPORT to run multiple Node.js applications with zero coordination overhead.

So, for teams running Node with any sort of performance sensitivity, here's what that means for you:

1. Zero-Overhead Load Balancing

Each Watt worker accepts connections directly from the OS using SO_REUSEPORT. There's no master process, no IPC coordination, no serialization overhead. The kernel distributes incoming connections using an efficient hash-based algorithm, and workers handle them independently.

2. Process Orchestration

While workers run independently, Watt manages the process lifecycle:

• Automatic restart of crashed processes

• Graceful shutdown handling

• Health monitoring and metrics

• Coordinated deployments

3. Shared HTTP Cache

Watt includes a shared HTTP cache layer across workers, reducing redundant work and improving cache hit rates. See our post on bringing HTTP caching to Node.js for details. (Oh, and for you all working in Next.js, you can now do component caching inside Watt as well.)

4. Automatic Health Restarts

Event loop or heap catastrophic failures trigger graceful worker restarts without pod termination, maintaining service availability. These checks are executed from the main thread without triggering the worker thread's event loop; we can perform them even if the application event loop is blocked.

How It Works

Watt runs multiple Node.js applications as separate threads within a single Node.js process. Each thread operates independently with its own event loop, but they all share the same listening socket using SO_REUSEPORT:

server.listen({
  host: '127.0.0.1',
  port: 3000,
  reusePort: true
})

This single flag - reusePort: true - is what enables the kernel to distribute connections efficiently. But rather than managing this yourself, Watt handles the entire orchestration for you while adding process management, health monitoring, and caching on top.

The result? The same performance as manually using SO_REUSEPORT, but with all the operational features you'd expect from a production application server.

Now, I know we are already 'under the hood' of Watt, but let's get a bit closer to the kernel, shall we?

(Even Deeper) Under the Hood:: How SO_REUSEPORT Works

The magic behind Watt's performance comes from a Linux kernel feature called SO_REUSEPORT, available since kernel 3.9 (April 2013). This socket option fundamentally changes how the operating system distributes incoming connections to processes.

Kernel-Level Load Distribution

When you set the reusePort: true option on Node.js's HTTP server, it calls this under the hood:

setsockopt(socket, SOL_SOCKET, SO_REUSEPORT, &opt, sizeof(opt));

This tells the Linux kernel to distribute incoming connections across all processes listening on the same port using a two-stage hash-based algorithm:

Stage 1: Listen Socket Lookup

• All processes using SO_REUSEPORT on the same port are grouped together in a shared array structure

• The destination port determines which bucket in the kernel's LISTEN hash table to search

Stage 2: Connection Distribution

• For each incoming connection, the kernel calculates a hash from the 4-tuple:

    hash(source_ip, source_port, dest_ip, dest_port)
  

• This hash selects which worker process receives the connection

• The worker accepts the connection directly - no coordination needed

Key Properties:

• Connection affinity: Same client IP:port always reaches the same worker

• Even distribution: Hash function provides balanced load across workers

• Zero coordination: No IPC, no shared state, no serialization

• Deterministic: Based purely on connection parameters, not current load

This is fundamentally different from PM2/cluster, where the master process must accept each connection and then forward it to a worker via Unix domain sockets - adding ~30% overhead.

For more technical details on SO_REUSEPORT, see:

• Cloudflare: The Sad State of Linux Socket Balancing

• NGINX: Socket Sharding in NGINX 1.9.1

Practical Applications: Two-Layer Architecture in Kubernetes

When you deploy Watt with multiple workers on a Kubernetes pod with multiple CPUs, you get a two-layer load balancing system (with two layers of resiliency):

Layer 1: Kubernetes Service

• Distributes new TCP connections across pod IPs

• Uses round-robin or other configured algorithms

• Example: 3 pods become 3 endpoints

Layer 2: Watt and Worker Threads within each pod

• Kernel distributes connections across workers in the same pod using Watt and SO_REUSEPORT

• Hash-based selection ensures balanced distribution

• Example: Each pod has 2 Watt workers for a total of 6 total workers

This creates better statistical multiplexing (combining multiple signals or data streams to share a single resource) than the traditional approach of 6 single-CPU pods.

Deeper under the hood we go…

1. Independent Event Loops

Each worker has its own event loop on its own CPU core. When Worker 1 is busy processing a slow Next.js SSR request, Worker 2's event loop continues processing its connections independently. Variance in request processing times affects fewer connections.

2. Resource Sharing Within Pods

Workers in the same pod share:

• Kernel page cache (file system operations)

• Memory for binary code (lower memory footprint)

• Single network namespace (lower context switching)

3. Better Failure Characteristics

With Watt's orchestration:

• Single worker crash: Only 1/6 capacity lost temporarily

• Automatic restart without pod termination

• Health checks at worker level, not pod level

• Graceful failover for catastrophic failures

4. Statistical Load Distribution

Hash-based distribution at both layers provides better statistical properties than round-robin to isolated pods. Connections are more evenly distributed, and the two-layer approach reduces the impact of connection-level variance.

Production Benchmarks: Next.js on AWS EKS

Ok. Now for the fun part. The part where I show you what all this means for your applications, with numbers. (And welcome, to those of you who scrolled here from the beginning of the article.)

To validate the theoretical and simulation results, we ran production-grade benchmarks using Next.js on AWS EKS (Elastic Kubernetes Service), i.e. testing a real-world framework that cannot implement early request rejection.

The tests compared three Kubernetes deployment strategies:

1. Single-CPU pods (6 replicas × 1000m CPU limit, 2GB RAM each = 6 total CPUs)

2. PM2 multi-worker pods (3 replicas × 2000m CPU limit with 2 PM2 workers, 4GB RAM each = 6 total CPUs)

3. Watt multi-worker pods (3 replicas × 2000m CPU limit with 2 Watt workers, 4GB RAM each = 6 total CPUs)

Given that Next.js Server-Side Rendering is CPU-bound, using 6 CPUs provides a like-for-like comparison.

Infrastructure:

• EKS Cluster: 3 nodes running m5.2xlarge instances (8 vCPUs, 32GB RAM each)

• Load Testing: c7gn.large instance (2 vCPUs, 4GB RAM, network-optimized)

• Load Pattern: k6 with constant arrival rate of 1,000 requests/second for 120 seconds

• Virtual Users: 1,000 pre-allocated VUs

The environment is totally ephemeral and created on-demand via a shell script and the aws CLI.

All configurations used identical total CPU resources (6 CPUs) for fair comparison. For testing, we used Grafana's K6, configured as follows:

k6 Load Test Configuration:

import http from 'k6/http';
import { check } from 'k6';

export const options = {
  scenarios: {
    constant_arrival_rate: {
      executor: 'constant-arrival-rate',
      duration: '120s',
      rate: 1000,
      timeUnit: '1s',
      preAllocatedVUs: 1000,
    },
  },
};

export default function () {
  const res = http.get(__ENV.TARGET, {
    timeout: "5s"
  });
  check(res, {
    'status is 200': (r) => r.status === 200,
  });
}

This configuration maintains a constant arrival rate of 1,000 requests/second for 120 seconds, with 1,000 pre-allocated virtual users and a 5-second timeout per request.

You can find the complete source code for these benchmarks at: https://github.com/platformatic/k8s-watt-performance-demo.

Benchmark Results

Key Findings

1. Throughput and Reliability

• Watt vs PM2: +9.6% more throughput (997 vs 910 req/s)

• Watt vs Single-CPU: +2.5% more throughput (997 vs 972 req/s)

• Watt success rate: 99.8% vs 91.9% (PM2) and 93.7% (single-CPU pods)

Under sustained load of 1,000 req/s, Watt maintained near-perfect reliability while both PM2 and single-CPU pod architectures experienced significant request failures (8.1% and 6.3% failure rates respectively).

2. Latency Performance

Watt delivers dramatically better latency across all percentiles:

• Median (P50): 11.6ms vs 182ms (PM2) = 93.6% faster

• Median (P50): 11.6ms vs 155ms (single-CPU) = 92.5% faster

• P95: 235ms vs 1,260ms (PM2) = 81.3% faster

• P95: 235ms vs 1,000ms (single-CPU) = 76.5% faster

3. Why Single-CPU Pods Underperform

Single-CPU pods suffer from two compounding issues: blind load distribution and limited self-healing capability.

Kubernetes distributes connections via round-robin without visibility into each pod's actual load. When one pod starts struggling—perhaps processing a slow SSR request—it keeps receiving new connections at the same rate as healthy pods.

The deeper problem: a Node.js process with a blocked event loop cannot effectively monitor itself. Health checks run on the same event loop, so by the time the process can report it's unhealthy, requests have already queued up and timed out. Kubernetes only sees the problem after users have already experienced failures.

4. Why PM2 Underperforms

PM2's lower throughput and higher latency validate our analysis of the cluster module overhead:

• Master process acts as internal load balancer, adding IPC overhead

• Every request requires socket transfer via Unix domain sockets

• Lower success rate (91.9%) indicates the coordination overhead impacts reliability under load

5. Watt's Advantages

Watt's key advantage is external health monitoring combined with SO_REUSEPORT.

Because Watt monitors workers from outside their event loops, it can detect when a worker is struggling and restart it before the situation cascades—without terminating the entire pod. This directly addresses the fundamental problem that a blocked Node.js process cannot effectively monitor itself.

The SO_REUSEPORT architecture eliminates the ~30% IPC overhead imposed by PM2 and the cluster module. Workers accept connections directly from the kernel with zero coordination. Our benchmark demonstrates results: a 99.8% success rate and a 93% faster median latency under sustained load.

Benchmark Configuration

The tests used Kubernetes deployments on AWS EKS with explicit resource limits:

# Single-CPU pods - 6 replicas with 1000m CPU limit each
apiVersion: apps/v1
kind: Deployment
metadata:
  name: next
spec:
  replicas: 6  # 6 pods × 1 CPU = 6 total CPUs
  template:
    spec:
      containers:
        - name: next
          resources:
            requests:
              cpu: '1000m'
              memory: '2Gi'
            limits:
              cpu: '1000m'
              memory: '2Gi'

# PM2 multi-worker pods - 3 replicas with 2000m CPU limit, 2 workers each
apiVersion: apps/v1
kind: Deployment
metadata:
  name: next-pm2
spec:
  replicas: 3  # 3 pods × 2 CPUs = 6 total CPUs
  template:
    spec:
      containers:
        - name: next-pm2
          env:
            - name: WORKERS
              value: "2"
          resources:
            requests:
              cpu: '2000m'
              memory: '4Gi'
            limits:
              cpu: '2000m'
              memory: '4Gi'

# Watt multi-worker pods - 3 replicas with 2000m CPU limit, 2 workers each
apiVersion: apps/v1
kind: Deployment
metadata:
  name: next-watt
spec:
  replicas: 3  # 3 pods × 2 CPUs = 6 total CPUs
  template:
    spec:
      containers:
        - name: next-watt
          env:
            - name: WORKERS
              value: "2"
          resources:
            requests:
              cpu: '2000m'
              memory: '4Gi'
            limits:
              cpu: '2000m'
              memory: '4Gi'

These results confirm that SO_REUSEPORT-based multi-worker pods (Watt) outperform both single-CPU pod scaling and PM2-based multi-worker approaches in real-world production scenarios on Kubernetes.

Getting Started with Watt

What's more fun than reading about our results? Replicating them with your own apps in your own environment, of course.

As you might already know, Watt is open source, and straightforward to implement. Simply follow these steps to deploy Next.js in Kubernetes with Watt: https://docs.platformatic.dev/docs/guides/deployment/nextjs-in-k8s.

Implementation and Configuration Tips

From PM2:

• Remove PM2 ecosystem files

• Replace pm2 start with your Watt-enabled entry point

• Set workers to match your previous PM2 instance count

• Update health checks to target individual workers if needed

From Single-CPU Pods:

• Reduce pod count and increase CPU per pod (maintain total CPU)

• Example: 6 × 1-CPU pods → 3 × 2-CPU pods with workers: 2

• Update resource limits to match worker count

• Monitor and adjust based on your traffic patterns (and check out our Intelligent Command Center if you want to make your life even easier 🙂)

For complete documentation and advanced features like shared HTTP caching, visit the Watt GitHub repository.

Conclusion

The 93% latency improvement we showed at the start of this post isn't magic - it's the result of eliminating unnecessary coordination overhead and letting the Linux kernel do what it does best: efficiently distributing network connections.

Traditional Node.js scaling approaches, whether PM2's cluster module or horizontally scaled single-CPU pods, impose architectural constraints that hurt performance:

• PM2/cluster: Every new connections pays a ~30% IPC tax for master-worker coordination

• Single-CPU pods: Isolated queues compound load imbalances, leading to higher failure rates

Watt takes a different approach: leverage SO_REUSEPORT to let multiple Node.js workers accept connections directly from the kernel, with zero coordination overhead. Then, we add all the needed control systems like healthchecks. The result is consistent and dramatic:

• 93.6% faster median latency than PM2

• 99.8% reliability under sustained load

• 9.6% more throughput with the same CPU resources

What makes this especially compelling is the simplicity of implementation. You don't need specialized hardware, complex infrastructure changes, or extensive code refactoring. The path from PM2 or single-CPU pods to Watt is straightforward, and the performance gains are immediate. (Believe me, compared to some of the things I've teams do to achieve even a quarter of these improvements, this is pretty incredible ROI for your time here.)

If you're running Node.js applications - especially frameworks like Next.js that can't implement early request rejection - on Kubernetes or with PM2, you now have a proven alternative. The benchmarks speak for themselves, and the implementation is open source.

Give Watt a try on your next deployment and measure the difference yourself. Your p95 latency will thank you.

If you want to have a chat with us about any of this, or are interested in professional support or architecture guidance with Watt or any of our other projects, feel free to send an email to info@platformatic.dev or add either Luca or me on LinkedIn.

This is a game-changer because Next.js 16 fundamentally reimagines React caching. For the first time, you can cache individual React components and functions with a simple use cache directive, giving you surgical precision over what gets cached and when. No more guessing about implicit caching behavior or wrestling with complex cache invalidation strategies. You now have explicit, fine-grained control over your application's performance.

However, here's where it gets even more exciting: while Next.js 16 provides the caching primitives, it requires custom configuration to enable component caching in production environments. That's where Watt 3.18.0 comes in. With Watt's Redis/Valkey cache adapter, you get instant, zero-configuration distributed caching. This means your cached React components are automatically shared across all application instances, eliminating cache inconsistencies and dramatically improving performance at scale.

The combination unlocks what was previously difficult to achieve: component-level caching that works seamlessly in distributed, production-scale Next.js deployments. Add a single line (cacheComponents: true) to your config, sprinkle use cache directives where you need them, and Watt handles all the complexity behind the scenes.

The Self-Hosting Challenge: Why Distributed Caching Matters

When deploying Next.js applications outside of Vercel's platform—whether on your own infrastructure, Kubernetes clusters, or other cloud providers—you face a significant caching challenge that can impact both performance and data consistency. Next.js defaults to file-system-based caching, which works perfectly fine for single-server deployments but creates serious issues when scaling horizontally.

Here's the problem: in a typical self-hosted production environment, such as a Kubernetes deployment with multiple replicas, you run multiple instances of your Next.js application behind a load balancer for high availability and scalability. When each instance relies on local disk caching, every server maintains its own separate cache. This means:

• Cache Inconsistencies: Different users may see different cached data depending on which server handles their request. User A hits Server 1 and sees cached data from an hour ago, while User B hits Server 2 and sees data cached just seconds ago.

• Wasted Resources: Each server independently fetches and caches the same data, multiplying your database/API load and memory usage across all instances.

• Cache Invalidation Complexity: When data changes, you need to invalidate caches across all servers simultaneously, which becomes a distributed systems problem without proper tooling.

Vercel's platform solves this through its Data Cache infrastructure, which provides a distributed cache shared across all function invocations. But when self-hosting, you're on your own to implement this critical piece of infrastructure.

This is precisely why Watt's Redis/Valkey adapter is essential for production Next.js deployments: it gives self-hosted applications the same distributed caching capabilities that Vercel provides as a platform feature. With a centralized cache store, all your application instances share the same cached data, ensuring consistency while dramatically improving performance and resource efficiency.

Understanding Next.js 16's use cache Directive

Next.js 16 introduces a fundamental shift in how caching works. Unlike previous versions, where caching was implicit, Next.js 16 makes caching explicit through the use cache directive. This gives you fine-grained control over what gets cached and when.

You can use the directive at different levels:

// Cache an entire component
export async function ProductList() {
  "use cache";
  const products = await fetchProducts();
  return <div>{/* render products */}</div>;
}

// Cache a specific function
export async function getProductData(id) {
  "use cache";
  const data = await fetch(`/api/products/${id}`);
  return data.json();
}

// Cache at the file level
("use cache");
export default async function Page() {
  return <div>{/* page content */}</div>;
}

This explicit approach provides more flexibility and control compared to the automatic caching behavior in earlier versions. Learn more about Next.js caching strategies in the official documentation.

What is Watt?

Watt is an extensible Node.js application server designed to simplify how you build, deploy, and scale applications. Whether you're creating simple APIs, microservices, or full-stack applications, Watt acts as a powerful orchestration layer that composes multiple services into a single cohesive system.

With Watt, you can seamlessly integrate frontend frameworks like Next.js, Astro, or Remix with backend microservices, all while benefiting from:

• Zero-configuration deployment - Applications run instantly without complex setup

• Service orchestration - Coordinates multiple applications and services seamlessly

• Production-ready features - Built-in monitoring, logging, and operational best practices

• Multi-threading support - Leverages Node.js worker threads for improved performance

• Shared caching - Centralized caching strategies like the Redis/Valkey adapter for distributed cache across all instances

Watt handles the complexity of inter-service communication, caching strategies, and deployment, allowing you to focus on building your application logic. Learn more in our Watt 3 introduction post.

The latest version of Watt adds first-class support for Next.js 16.0, including the new Cache Components feature. When you enable Watt's Redis/Valkey adapter and set the new cacheComponents option to true, you unlock the full potential of Next.js 16's use cache directive.

Key Features

• Component-Level Caching: Use the use cache directive to cache individual React components, functions, or entire route handlers

• Distributed Caching with Valkey: Share cached components across all your application instances using Redis or Valkey.

• Simple Configuration: Enable component caching with a single cacheComponents: true setting in your Watt configuration

• Seamless Integration: Watt handles all the complexity of configuring Next.js 16's cache handler behind the scenes

Enabling Component Caching in Watt

To take advantage of Next.js 16's component caching with Watt, you need to configure the Redis/Valkey adapter and enable component caching in your watt.json:

{
  "cache": {
    "adapter": "valkey",
    "url": "valkey://redis.example.com:6379",
    "cacheComponents": true
  }
}

With these settings in place, Watt automatically configures Next.js 16's cache handler to use Redis or Valkey for storing cached components, making them available across all instances of your application.

Important: ISR Cache Behavior

When you enable the Redis/Valkey adapter and set cacheComponents: true, there's an important architectural consideration: Next.js 16 does not support running both the new component caching and the legacy Incremental Static Regeneration (ISR) cache simultaneously.

As a result, when component caching is enabled in Watt 3.18.0 with Next.js 16, the traditional ISR cache is automatically disabled. This is a limitation of Next.js 16's architecture, not Watt. The new use cache directive provides more explicit and flexible caching capabilities that supersede the older ISR approach.

If your application relies heavily on ISR, you can continue using Next.js 16 with cacheComponents disabled (or omitted) and avoid using the use cache directive. This way, you can benefit from other Next.js 16 features while maintaining your existing ISR-based caching strategy.

Why Redis/Valkey for Next.js Caching?

Redis and Valkey are high-performance, in-memory data stores optimized for caching workloads. When scaling Next.js applications horizontally, local caching solutions fall short because each instance maintains its own cache, leading to inconsistent user experiences.

By using Watt's Redis/Valkey adapter, you get:

• Shared cache across instances: All application replicas access the same cached data

• Automatic cache key management: Watt handles cache key generation using Next.js 16's compiler

• Production-ready performance: Autopipelining support ensures minimal latency

• Simple configuration: No need to manually configure cache handlers or manage cache lifecycle

Learn more about Redis and Valkey in their official documentation.

Getting Started

To try Next.js 16 component caching with Watt 3.18.0:

1. Update to Watt 3.18.0 or later

2. Upgrade your Next.js dependency to version 16.0 or higher

3. Configure the Redis/Valkey adapter in your watt.json

4. Enable cacheComponents: true in your Next.js service configuration

5. Add use cache directives to the components and functions you want to cache

That's it! Watt takes care of all the underlying complexity, allowing you to focus on building great applications.

Learn More

• Platformatic Watt Documentation

• Next.js 16 Announcement

• Next.js Cache Components Guide

• use cache Directive Reference

• Valkey-Based Caching for Next.js

• Valkey Project

• PR #4397: Next.js 16.0 Support

We're committed to making Platformatic the best platform for building and deploying modern web applications. This update represents another step forward in providing seamless, production-ready tooling for the latest web technologies.

Try it out and let us know what you think!