README.md

# Learn Node.js 🟩

<div align="center">

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Node.js Version](https://img.shields.io/badge/Node.js-24%20LTS+-5FA04E.svg?logo=nodedotjs&logoColor=white)](https://nodejs.org/)
[![CI](https://github.com/TheLearningHubOrg/learn-nodejs/actions/workflows/ci.yml/badge.svg)](https://github.com/TheLearningHubOrg/learn-nodejs/actions/workflows/ci.yml)
[![Release](https://img.shields.io/github/v/release/TheLearningHubOrg/learn-nodejs)](https://github.com/TheLearningHubOrg/learn-nodejs/releases)
[![TypeScript](https://img.shields.io/badge/TypeScript-type%20stripping-3178C6.svg?logo=typescript&logoColor=white)](https://nodejs.org/api/typescript.html)
[![libuv](https://img.shields.io/badge/libuv-event%20loop-purple.svg)](https://docs.libuv.org/)

**From your first `node` command to staff-level Node.js: how the runtime works, how to write it well, and how to run it in production.**  
V8 and libuv internals, the event loop and its queues, CommonJS and ES modules, native TypeScript, async error handling and cancellation, buffers, streams and backpressure, HTTP, worker threads, testing, diagnostics and deployment.

[Getting Started](#1-stage-1-runtime-architecture-toolchain--the-event-loop) • [Modules & Tooling](#2-stage-2-modules-packages--tooling) • [Async & Streams](#3-stage-3-async-programming-events-buffers-files--streams) • [Networking & HTTP](#4-stage-4-networking-http--web-frameworks) • [Concurrency & Testing](#5-stage-5-concurrency-request-context--testing) • [Performance & Security](#6-stage-6-performance-security--operations) • [Interview Prep](#7-stage-7-staff-nodejs-engineer-interview-handbook--cheatsheet)

<br/>

<a href="https://www.buymeacoffee.com/manthanank">
  <img src="public/bmc-button.svg" alt="Buy Me A Coffee" width="180">
</a>

</div>

---

## 🗺️ 7-Stage Pedagogical Roadmap

```mermaid
flowchart LR
    S1["Stage 1: Runtime & Event Loop"] --> S2["Stage 2: Modules & Tooling"]
    S2 --> S3["Stage 3: Async, Buffers & Streams"]
    S3 --> S4["Stage 4: Networking & HTTP"]
    S4 --> S5["Stage 5: Concurrency & Testing"]
    S5 --> S6["Stage 6: Performance & Security"]
    S6 --> M["Masterclasses M1-M3"]
    M --> S7["Stage 7: Staff Interview Handbook"]
```

| Stage | Focus Domain | Core Concepts & Engineering Outcomes |
| :--- | :--- | :--- |
| **Stage 1** | **Runtime Architecture, Toolchain & the Event Loop** | Install and pin versions with `fnm`/`nvm` and `engines`; V8 tiers (Ignition, Sparkplug, Maglev, TurboFan), hidden classes, heap spaces and GC; the six libuv phases; `process.nextTick` vs Promise jobs in CommonJS and ESM; the 4-thread libuv pool and `UV_THREADPOOL_SIZE`. You can predict the output order of any scheduling script and explain why. |
| **Stage 2** | **Modules, Packages & Tooling** | `.mjs`/`.cjs`/`"type"`, resolution rules, `require(esm)` and `ERR_REQUIRE_ASYNC_MODULE`, `createRequire`, `exports`/`imports` maps and conditions, semver and `npm ci`, `node --run`, type stripping with `erasableSyntaxOnly`, `--watch`, `--env-file`, `util.parseArgs`. You can publish a dual-consumer package and run TypeScript with no build step. |
| **Stage 3** | **Async Programming, Events, Buffers, Files & Streams** | Error-first callbacks, `util.promisify`, Promise combinators, `Error.cause`, `unhandledRejection`, `AbortController`/`AbortSignal.timeout()`/`AbortSignal.any()`, `EventEmitter` and `events.once`, `Buffer` vs `Uint8Array`, `StringDecoder`, `fs/promises`, `stream/promises` `pipeline()` and backpressure, zlib. You can process a file of any size in constant memory and cancel any async operation. |
| **Stage 4** | **Networking, HTTP & Web Frameworks** | `node:http` request/response streams, body limits, TCP with `node:net`, UDP with `node:dgram`, Express 5 vs Fastify vs NestJS. |
| **Stage 5** | **Concurrency, Request Context & Testing** | `worker_threads` and `SharedArrayBuffer`, `cluster`, `AsyncLocalStorage`, `node:test` with mocks and fake timers. |
| **Stage 6** | **Performance, Security & Operations** | `perf_hooks`, heap snapshots and `node:v8`, memory leak hunting, `node:crypto` (AES-256-GCM, `timingSafeEqual`), graceful shutdown on `SIGTERM`. |
| **Masterclasses** | **Streaming file server, queue-backed gateway, WebSocket gateway** | End-to-end builds that combine streams, HTTP Range requests, BullMQ, `AsyncLocalStorage` and `ws`. |
| **Stage 7** | **Staff Interview Handbook** | 49 interview Q&As in four groups, scenario questions, one-page cheatsheet. |

---

## 🆕 Node.js Versions, What's New & Migration

Node.js ships a new major every six months. Even-numbered majors (20, 22, 24) are promoted to **Long-Term Support** in October of their release year and are what you run in production. Odd-numbered majors (23, 25) are short-lived "Current" lines for trying features early. A line moves through three states: *Current* (about six months), *Active LTS* (about twelve months) and *Maintenance* (security and critical fixes only) until end-of-life roughly three years after its first release.

This guide targets **Node.js 24 LTS** and keeps everything runnable on **Node.js 22.18+**. The starter project's CI runs on both. Check [the official release schedule](https://nodejs.org/en/about/previous-releases) for the newest even-numbered line and its exact LTS dates; dates below cover only lines already released.

### V.1 Release Timeline

```mermaid
timeline
    title Node.js release lines that matter for this guide
    section 20 Iron (April 2023)
      Experimental permission model, stable test runner : end-of-life April 2026
    section 22 Jod (April 2024)
      require(esm), --watch stable, WebSocket client : Maintenance LTS, end-of-life April 2027
      Type stripping on by default from 22.18 : run .ts files without a build
    section 23 (October 2024)
      require(esm) unflagged, type stripping unflagged in 23.6 : short-lived Current line
    section 24 Krypton (May 2025)
      V8 13.6, npm 11, URLPattern global : LTS since October 2025
      AsyncLocalStorage on AsyncContextFrame : faster context propagation
    section 25 (October 2025)
      V8 14.x : short-lived Current line
```

### V.2 Compatibility Matrix

| Node.js | First release | Status (November 2026) | V8 | npm (bundled) | Notes |
| --- | --- | --- | --- | --- | --- |
| 20 "Iron" | April 2023 | End-of-life since April 2026 | 11.3 | 9 / 10 | Upgrade now. No security fixes. `node:20` images fail this repo's Dockerfile check. |
| 22 "Jod" | April 2024 | Maintenance LTS until April 2027 | 12.4 | 10 | Minimum for this guide is **22.18.0**, the first 22.x that strips types without a flag. |
| 23 | October 2024 | End-of-life | 12.9 | 10 | First line with `require(esm)` on by default. |
| 24 "Krypton" | May 2025 | Active LTS, end-of-life April 2028 | 13.6 | 11 | Target of this guide. The Dockerfile uses `node:24-alpine`. |
| 25 | October 2025 | Current, short-lived | 14.x | 11 | Use for previewing features, not for production. |

Check what your machine runs:

```bash
node -p "process.versions.node + ' V8 ' + process.versions.v8 + ' libuv ' + process.versions.uv"
node -p "process.release.lts || 'not an LTS line'"
```

### V.3 What's New in Node.js 24

#### Running TypeScript directly

Node 22.18+ and 24 remove type annotations at load time and run the remaining JavaScript. Only syntax that can be erased is accepted, so `enum`, `namespace` with values and constructor parameter properties are rejected.

```bash
# Before: a build step or a loader
npx tsc && node dist/server.js
npx tsx src/server.ts

# After (Node 22.18+, 24): no extra tooling for development
node src/server.ts
node --watch src/server.ts
```

Section [2.6](#26-running-typescript-natively-with-type-stripping) covers the rules, the `tsconfig.json` options that enforce them, and why the production image still compiles with `tsc`.

#### `require()` of ES modules

CommonJS code can `require()` an ES module as long as the module graph has no top-level `await`. This removes the most common reason libraries shipped two builds.

```js
// Before: CommonJS had to switch to an async dynamic import
const { greet } = await import('./esm-lib.mjs'); // only inside an async function

// After (20.19+, 22.12+, 24): synchronous
const { greet } = require('./esm-lib.mjs');
```

See [2.3](#23-interop-requireesm-import-of-commonjs-and-createrequire) and `examples/modules/`.

#### `URLPattern` as a global

```ts
// Before: a routing library or a hand-written regex
const match = /^\/users\/([^/]+)$/.exec(pathname);

// After (24): the WHATWG URLPattern API, no import needed
const pattern = new URLPattern({ pathname: '/users/:id' });
const result = pattern.exec('https://example.test/users/42');
console.log(result?.pathname.groups.id); // "42"
```

#### `AsyncLocalStorage` built on `AsyncContextFrame`

Node 24 switched `AsyncLocalStorage` to an implementation that stores context on V8's continuation frame instead of using async hooks. The API is unchanged; propagation is cheaper and context now follows `await` more predictably.

```ts
import { AsyncLocalStorage } from 'node:async_hooks';

const requestContext = new AsyncLocalStorage<{ requestId: string }>();

// Same code on 22 and 24; on 24 it no longer relies on async_hooks under the hood.
await requestContext.run({ requestId: 'req-1' }, async () => {
  await new Promise((r) => setTimeout(r, 10));
  console.log(requestContext.getStore()?.requestId); // "req-1"
});
```

#### The permission model is stable

```bash
# Before (20.x): experimental flag
node --experimental-permission --allow-fs-read=./data app.js

# After (22.13+, 23.5+, 24): stable flag name
node --permission --allow-fs-read=./data app.js
```

A process started with `--permission` cannot read or write files, spawn children or start workers unless a matching `--allow-*` flag grants it. Stage 6 shows how to use it as a defence-in-depth layer.

#### Explicit resource management with `using`

V8 13.6 supports `using` declarations, which call `[Symbol.dispose]()` (or `await using` with `[Symbol.asyncDispose]()`) when a block exits. Node's own handles are gaining dispose methods across the 24 and 25 lines; check each API's docs before relying on one.

```ts
import { open } from 'node:fs/promises';

// Before: try/finally for every resource
const handle = await open('data.bin');
try {
  /* ... */
} finally {
  await handle.close();
}

// After: the FileHandle closes when the block exits, even on throw
{
  await using handle = await open('data.bin');
  /* ... */
}
```

### V.4 Migration Guide (Node.js 20/22 → 24)

```bash
# 1. Install 24 next to your current version and pin it for the project
fnm install 24 && fnm use 24          # or: nvm install 24 && nvm use 24
echo "24" > .nvmrc

# 2. Declare the supported range so npm warns on older runtimes
npm pkg set engines.node=">=22.18.0"

# 3. Reinstall from the lockfile with the new npm, then look for native addons that need a rebuild
rm -rf node_modules && npm ci
npm ls --all 2>/dev/null | grep -i -E "node-gyp|prebuild" || true

# 4. Surface every deprecation your code or dependencies hit
NODE_OPTIONS="--throw-deprecation" npm test

# 5. Update CI and the container base image
#    .github/workflows/ci.yml  -> node-version: [22, 24]
#    Dockerfile                -> FROM node:24-alpine
```

After the upgrade, look at these three areas first:

1. **Flags that changed name.** `--experimental-permission` is now `--permission`. `--experimental-strip-types` is no longer needed on 22.18+ and 24.
2. **Native addons.** Anything built with `node-gyp` against the old ABI must be rebuilt (`npm rebuild`). Addons that use Node-API keep working without a rebuild.
3. **Timing-sensitive tests.** `AsyncLocalStorage` and test-runner changes can shift the order in which async callbacks observe state. A test that depended on an accidental order will show up here.

### V.5 Deprecations & Breaking Changes

| Version | Deprecated / removed | Replacement | What breaks and how to fix it |
| --- | --- | --- | --- |
| 20 | End-of-life (April 2026) | 22 or 24 | No more security releases. Move CI matrices and images off `node:20`. |
| 22.13 / 24 | `--experimental-permission` | `--permission` | Old flag name is not recognised; update start scripts. |
| 22.18 / 24 | Need for `--experimental-strip-types` | Nothing: on by default | Leaving the flag in is harmless; remove it to keep scripts short. |
| 24 | `tls.createSecurePair()` removed | `tls.TLSSocket` | Throws `TypeError` on call. Wrap sockets with `new tls.TLSSocket()`. |
| 24 | `url.parse()` (DEP0169, legacy API) | `new URL(input, base)` | `url.parse()` accepts malformed input that WHATWG `URL` rejects; migrate and handle `TypeError` for invalid URLs. |
| Long-standing | `process._getActiveHandles()` / `_getActiveRequests()` (undocumented) | `process.getActiveResourcesInfo()` | Private APIs may change without notice. The starter's diagnostics endpoint uses the public API. |
| Long-standing | `new Buffer(n)` (DEP0005) | `Buffer.alloc(n)`, `Buffer.from(x)` | Emits a deprecation warning; the old constructor could expose uninitialised memory. |
| Long-standing | `fs.exists()` (DEP0034) | `fs.access()` or just open the file and handle `ENOENT` | Check-then-act races; open and handle the error instead. |

The full list lives in the [deprecations reference](https://nodejs.org/api/deprecations.html). Run your test suite with `--throw-deprecation` on every upgrade to turn warnings into failures you can see.

---

## 📋 Table of Contents

- [🆕 Node.js Versions, What's New & Migration](#-nodejs-versions-whats-new--migration)
- [1. Stage 1: Runtime Architecture, Toolchain & the Event Loop](#1-stage-1-runtime-architecture-toolchain--the-event-loop)
  - [1.1 What Node.js Is and the Problem It Solves](#11-what-nodejs-is-and-the-problem-it-solves)
  - [1.2 Installing Node.js and Managing Versions](#12-installing-nodejs-and-managing-versions)
  - [1.3 First Programs: the REPL, Scripts and the process Object](#13-first-programs-the-repl-scripts-and-the-process-object)
  - [1.4 Line-by-Line Breakdown of the Starter Project](#14-line-by-line-breakdown-of-the-starter-project)
  - [1.5 Inside V8: Parsing, Tiered Compilation and Deoptimisation](#15-inside-v8-parsing-tiered-compilation-and-deoptimisation)
  - [1.6 Hidden Classes and Inline Caches](#16-hidden-classes-and-inline-caches)
  - [1.7 V8 Memory: Stack, Heap and Garbage Collection](#17-v8-memory-stack-heap-and-garbage-collection)
  - [1.8 The libuv Event Loop: Phases in Order](#18-the-libuv-event-loop-phases-in-order)
  - [1.9 Microtasks: the process.nextTick and Promise Queues](#19-microtasks-the-processnexttick-and-promise-queues)
  - [1.10 The libuv Thread Pool and UV_THREADPOOL_SIZE](#110-the-libuv-thread-pool-and-uv_threadpool_size)
  - [1.11 Pitfalls](#111-pitfalls)
  - [1.12 Best Practices](#112-best-practices)
  - [1.13 Hands-on Exercise](#113-hands-on-exercise)
- [2. Stage 2: Modules, Packages & Tooling](#2-stage-2-modules-packages--tooling)
  - [2.1 CommonJS vs ES Modules](#21-commonjs-vs-es-modules)
  - [2.2 How Node Picks a Module Type and Resolves a Specifier](#22-how-node-picks-a-module-type-and-resolves-a-specifier)
  - [2.3 Interop: require(esm), Import of CommonJS and createRequire](#23-interop-requireesm-import-of-commonjs-and-createrequire)
  - [2.4 package.json: type, exports, imports and Conditions](#24-packagejson-type-exports-imports-and-conditions)
  - [2.5 npm: Semantic Versioning, Lockfiles and Scripts](#25-npm-semantic-versioning-lockfiles-and-scripts)
  - [2.6 Running TypeScript Natively with Type Stripping](#26-running-typescript-natively-with-type-stripping)
  - [2.7 Built-in Dev Tooling: --watch, --env-file, parseArgs and styleText](#27-built-in-dev-tooling---watch---env-file-parseargs-and-styletext)
  - [2.8 Essential CLI Flags and Environment Variables](#28-essential-cli-flags-and-environment-variables)
  - [2.9 The node:util Toolbox](#29-the-nodeutil-toolbox)
  - [2.10 Pitfalls](#210-pitfalls)
  - [2.11 Best Practices](#211-best-practices)
  - [2.12 Hands-on Exercise](#212-hands-on-exercise)
- [3. Stage 3: Async Programming, Events, Buffers, Files & Streams](#3-stage-3-async-programming-events-buffers-files--streams)
  - [3.1 From Callbacks to Promises to async/await](#31-from-callbacks-to-promises-to-asyncawait)
  - [3.2 Error Handling Across Async Boundaries](#32-error-handling-across-async-boundaries)
  - [3.3 Cancellation and Deadlines with AbortController](#33-cancellation-and-deadlines-with-abortcontroller)
  - [3.4 Events and EventEmitter](#34-events-and-eventemitter)
  - [3.5 Buffers and Binary Data](#35-buffers-and-binary-data)
  - [3.6 The File System: node:fs and node:fs/promises](#36-the-file-system-nodefs-and-nodefspromises)
  - [3.7 Paths, URLs and the Operating System](#37-paths-urls-and-the-operating-system)
  - [3.8 Streams and Backpressure](#38-streams-and-backpressure)
  - [3.9 Compression with node:zlib](#39-compression-with-nodezlib)
  - [3.10 Pitfalls](#310-pitfalls)
  - [3.11 Best Practices](#311-best-practices)
  - [3.12 Hands-on Exercise](#312-hands-on-exercise)
- [4. Stage 4: Networking, HTTP & Web Frameworks](#4-stage-4-networking-http--web-frameworks)
  - [4.1 HTTP Servers with node:http](#41-http-servers-with-nodehttp)
  - [4.2 Raw Networking: TCP and UDP (node:net, node:dgram)](#42-raw-networking-tcp-and-udp-nodenet-nodedgram)
  - [4.3 Framework Comparison: Express 5 vs Fastify vs NestJS](#43-framework-comparison-express-5-vs-fastify-vs-nestjs)
- [5. Stage 5: Concurrency, Request Context & Testing](#5-stage-5-concurrency-request-context--testing)
  - [5.1 Worker Threads (node:worker_threads)](#51-worker-threads-nodeworker_threads)
  - [5.2 Multi-Process Clustering (node:cluster)](#52-multi-process-clustering-nodecluster)
  - [5.3 Request Context with AsyncLocalStorage](#53-request-context-with-asynclocalstorage)
  - [5.4 The Built-in Test Runner (node:test, node:assert)](#54-the-built-in-test-runner-nodetest-nodeassert)
- [6. Stage 6: Performance, Security & Operations](#6-stage-6-performance-security--operations)
  - [6.1 Performance Hooks and High-Resolution Timing (node:perf_hooks)](#61-performance-hooks-and-high-resolution-timing-nodeperf_hooks)
  - [6.2 Heap Statistics and Snapshots (node:v8)](#62-heap-statistics-and-snapshots-nodev8)
  - [6.3 Memory Leak Profiling with Heap Snapshots](#63-memory-leak-profiling-with-heap-snapshots)
  - [6.4 Cryptography (node:crypto)](#64-cryptography-nodecrypto)
  - [6.5 Graceful Shutdown and Zero-Downtime Signal Handling](#65-graceful-shutdown-and-zero-downtime-signal-handling)
- [M1. Masterclass: Streaming File Server with Range Requests](#m1-masterclass-streaming-file-server-with-range-requests)
  - [M1.1 Reference Implementation](#m11-reference-implementation)
- [M2. Masterclass: Queue-Backed Order Gateway](#m2-masterclass-queue-backed-order-gateway)
  - [M2.1 Architecture and Implementation](#m21-architecture-and-implementation)
- [M3. Masterclass: Real-Time WebSocket Gateway](#m3-masterclass-real-time-websocket-gateway)
  - [M3.1 Architecture and Implementation](#m31-architecture-and-implementation)
- [7. Stage 7: Staff Node.js Engineer Interview Handbook & Cheatsheet](#7-stage-7-staff-nodejs-engineer-interview-handbook--cheatsheet)
  - [7.1 Core Concepts (Q1-Q20)](#71-core-concepts-q1-q20)
  - [7.2 Internals & Performance (Q21-Q32)](#72-internals--performance-q21-q32)
  - [7.3 Architecture & Design (Q33-Q43)](#73-architecture--design-q33-q43)
  - [7.4 Scenario & Debugging Questions (Q44-Q49)](#74-scenario--debugging-questions-q44-q49)
  - [7.5 Staff Cheatsheet](#75-staff-cheatsheet)
- [🧪 Example Projects](#-example-projects)
- [📝 Cheat Sheets](#-cheat-sheets)
- [📚 Resources](#-resources)
- [🤝 Contributing](#-contributing)
- [📄 License](#-license)
- [🙏 Acknowledgments](#-acknowledgments)
- [🌐 Connect with me](#-connect-with-me)
- [☕ Support](#-support)

---

## 1. Stage 1: Runtime Architecture, Toolchain & the Event Loop

This stage explains what actually runs when you type `node server.ts`: the V8 engine that executes JavaScript, the libuv library that does I/O, and the event loop that ties them together. You need basic JavaScript (functions, objects, promises) and a terminal. By the end you can install and pin Node versions, read the starter server line by line, and predict the exact order in which timers, I/O callbacks, `process.nextTick` and promises run.

Code examples in this guide are TypeScript that uses only erasable syntax, so Node 22.18+ and 24 run them directly with `node file.ts`. The [Example Projects](#-example-projects) section explains the choice.

```mermaid
flowchart TD
    subgraph App["Your code"]
        JS["JavaScript / TypeScript"]
        Lib["node: built-in modules (fs, http, crypto, stream)"]
    end
    subgraph Bindings["Node.js C++ bindings"]
        B["node_file.cc, tcp_wrap.cc, crypto bindings"]
    end
    subgraph Deps["Bundled dependencies"]
        V8["V8: parse, compile, run JS, manage the heap"]
        UV["libuv: event loop, sockets, timers, thread pool"]
        Other["llhttp, undici, OpenSSL, zlib, c-ares, ICU, simdutf, ada"]
    end
    subgraph OS["Operating system"]
        Poll["epoll / kqueue / IOCP"]
        Threads["libuv worker threads (default 4)"]
    end
    JS --> Lib --> B
    B --> V8
    B --> UV
    B --> Other
    UV --> Poll
    UV --> Threads
```

### 1.1 What Node.js Is and the Problem It Solves

Node.js is a JavaScript **runtime**, not a language and not a framework. It embeds Google's V8 engine (the same one Chrome uses) and adds what a browser does not give you: files, sockets, processes, environment variables, and a module system. Its design choice is that one thread runs all your JavaScript, and every slow operation (network, disk, timers) is handed to the operating system or a small thread pool. When the operation finishes, its callback is queued and the single JavaScript thread runs it.

That model fits servers that spend most of their time waiting: an API that calls a database, a gateway that proxies requests, a worker that reads a queue. One process can hold tens of thousands of open connections because a waiting connection costs a few kilobytes of memory, not a thread with its own stack. The same model is a poor fit for long CPU-bound work on the main thread, because while one callback computes, every other request waits. Stage 5 covers `worker_threads` for that case.

| Component | What it does in Node.js | Where you meet it |
| :--- | :--- | :--- |
| **V8** | Parses and compiles JavaScript, runs it, owns the garbage-collected heap | `process.versions.v8`, `--max-old-space-size`, heap snapshots |
| **libuv** | Cross-platform event loop, TCP/UDP/pipes, timers, file I/O, a worker thread pool | Event loop phases, `UV_THREADPOOL_SIZE` |
| **llhttp** | HTTP/1.1 parser written for speed and strictness | `node:http` servers, `HPE_*` parse errors |
| **undici** | HTTP client behind the global `fetch()` | `fetch`, `Request`, `Response`, `WebSocket` |
| **OpenSSL** | TLS and cryptographic primitives | `node:tls`, `node:https`, `node:crypto` |
| **c-ares** | Asynchronous DNS queries | `dns.resolve*()` (not `dns.lookup()`) |
| **zlib, brotli, zstd** | Compression codecs | `node:zlib` |
| **ICU** | Unicode, time zones, `Intl` | `Intl.DateTimeFormat`, `toLocaleString()` |
| **amaro** | Wraps the SWC parser to strip TypeScript types | `node file.ts` |

Print the versions your binary bundles:

```bash
node -p "process.versions"
```

Node is not the only server-side JavaScript runtime. The comparison helps you understand which properties are Node's own choices:

| Runtime | Engine | Implemented in | TypeScript | Permissions | Package story |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **Node.js** | V8 | C++ | Type stripping (22.18+, 24) | Opt-in `--permission` | npm registry, `node_modules`, CommonJS and ESM |
| **Deno** | V8 | Rust | Full type checking built in | Deny by default, `--allow-*` | URL and `npm:` specifiers, ESM first |
| **Bun** | JavaScriptCore | Zig | Transpiles TS and JSX | None by default | npm-compatible, built-in bundler and test runner |

Node's advantage is the size and stability of its ecosystem and its LTS policy. Almost every npm package is tested against it first.

### 1.2 Installing Node.js and Managing Versions

Install Node with a **version manager**, not a system package. You will work on projects that need different majors, and CI must run the same version you do.

```bash
# fnm: cross-platform (Windows, macOS, Linux), reads .nvmrc and .node-version
winget install Schniz.fnm            # Windows
brew install fnm                     # macOS
curl -fsSL https://fnm.vercel.app/install | bash   # Linux

fnm install 24
fnm use 24
fnm default 24

# nvm: POSIX shells only (nvm-windows is a separate project with different commands)
nvm install 24
nvm use 24
nvm alias default 24
```

Pin the version per project in two places. `.nvmrc` tells version managers and `actions/setup-node` what to install; `engines` makes npm warn when someone runs an unsupported version.

```bash
echo "24" > .nvmrc
npm pkg set engines.node=">=22.18.0"
```

```json
{
  "engines": {
    "node": ">=22.18.0"
  }
}
```

The starter project declares `>=22.18.0` because that is the oldest release that runs `.ts` files without a flag. Running `npm install` on 22.16 prints `npm warn EBADENGINE` and continues; add `engine-strict=true` to `.npmrc` if you want it to fail instead.

Check the install:

```bash
node --version          # v24.x.y
npm --version           # 11.x
node -p "process.release.lts"   # "Krypton" on 24, undefined on Current lines
which node              # (where node on Windows) should point into the version manager's directory
```

Corepack (the shim that runs `pnpm` and `yarn` at the version pinned in `package.json#packageManager`) ships with Node 24. The project decided to stop bundling it from Node 25; on those versions install it with `npm install --global corepack`.

### 1.3 First Programs: the REPL, Scripts and the `process` Object

Node runs code four ways. Each is useful for a different job:

```bash
node                                  # REPL: experiment, inspect objects, .help for commands
node -e "console.log(1 + 1)"          # evaluate a string
node -p "os.cpus().length"            # evaluate and print; built-in modules are pre-loaded as globals
node hello.ts                         # run a file (ESM or CJS decided by extension and package.json)
```

The `process` global connects your program to the operating system. This script uses the parts you need on day one:

```ts
// hello.ts — run with: node hello.ts Ada --shout
const [, , name = 'world', ...flags] = process.argv;
const shout = flags.includes('--shout');

const greeting = `Hello, ${name}! Node ${process.version} on ${process.platform}/${process.arch}`;
console.log(shout ? greeting.toUpperCase() : greeting);

console.log(`cwd: ${process.cwd()}`);
console.log(`NODE_ENV: ${process.env.NODE_ENV ?? '(unset)'}`);
console.log(`pid ${process.pid}, uptime ${process.uptime().toFixed(3)} s`);

// Set the exit code and let the process end naturally, instead of calling process.exit()
// in the middle of pending work.
if (!name) process.exitCode = 2;
```

| Line / API | Role | What happens at runtime |
| :--- | :--- | :--- |
| `process.argv` | Command-line arguments | Index 0 is the `node` binary, 1 is the script path, the rest are yours. |
| `process.version`, `.platform`, `.arch` | Runtime identity | Strings such as `v24.11.0`, `linux`, `x64`; use them in diagnostics, not for feature detection. |
| `process.cwd()` | Current working directory | The directory the shell was in, not the script's directory. Use `import.meta.dirname` for that. |
| `process.env` | Environment variables | Always strings or `undefined`. `process.env.PORT = 8080` stores the string `"8080"`. |
| `process.exitCode` | Exit status | Node exits with this code when the event loop has nothing left to do. |
| `process.exit(n)` | Immediate exit | Stops now; pending writes to pipes, timers and `finally` blocks in other callbacks do not run. |

Node also provides the web platform globals, so code written for browsers often runs unchanged: `fetch`, `Request`, `Response`, `URL`, `URLSearchParams`, `AbortController`, `TextEncoder`, `structuredClone`, `performance`, `crypto.randomUUID()`, `WebSocket`, `navigator`, and web streams (`ReadableStream`).

Exit codes are how containers and CI know whether you succeeded:

| Code | Meaning in Node |
| :--- | :--- |
| `0` | Normal completion |
| `1` | Uncaught exception or unhandled promise rejection |
| `9` | Invalid command-line argument |
| `13` | ESM: top-level `await` never settled (the loop emptied while waiting) |
| `128 + n` | Killed by signal `n` (`137` = SIGKILL, often the OOM killer; `143` = SIGTERM unhandled) |

### 1.4 Line-by-Line Breakdown of the Starter Project

The repository root is a small Express 5 server written in TypeScript that exposes the runtime: `/health`, `/api/diagnostics` (memory, CPU, active handles) and four sandboxes that run event-loop, stream, worker-thread and cancellation experiments. Run it:

```bash
git clone https://github.com/TheLearningHubOrg/learn-nodejs.git
cd learn-nodejs
npm ci
npm run dev                # node --watch src/server.ts, no build step
# open http://localhost:8080 for the dashboard
curl -s localhost:8080/health
npm test                   # node:test, 30+ tests including every example script
```

```text
learn-nodejs/
├── src/
│   ├── server.ts                 process lifecycle: listen, log, graceful shutdown
│   ├── app.ts                    createApp(): routes, static files, error middleware
│   ├── config.ts                 PORT/HOST parsing, package version via import.meta.url
│   ├── errors.ts                 HttpError and input validation helpers
│   ├── controllers/              thin HTTP handlers
│   └── services/                 event loop, streams, worker thread and deadline demos
├── test/                         node:test suites (HTTP, services, examples/)
├── examples/                     one runnable script per Stage 1-3 concept
├── public/                       dashboard served by express.static
├── tsconfig.json                 type-check everything, erasable syntax only
└── tsconfig.build.json           emit dist/ for the production image
```

`src/server.ts` owns the process. It is the only file that reads the environment, binds a port and handles signals:

```ts
import { once } from 'node:events';
import { createApp } from './app.ts';
import { loadConfig } from './config.ts';

const config = loadConfig();
const app = createApp(config.version);
const server = app.listen(config.port, config.host);

await once(server, 'listening');
console.log(
  `learn-nodejs ${config.version} listening on http://${config.host}:${config.port} ` +
    `(Node ${process.version}, pid ${process.pid})`
);

let shuttingDown = false;

function shutdown(signal: NodeJS.Signals): void {
  if (shuttingDown) return;
  shuttingDown = true;
  console.log(`${signal} received: refusing new connections and draining in-flight requests`);

  const deadline = setTimeout(() => {
    console.error(`Shutdown exceeded ${config.shutdownTimeoutMs} ms, exiting with code 1`);
    process.exit(1);
  }, config.shutdownTimeoutMs);
  deadline.unref();

  server.close((err) => {
    if (err) {
      console.error(err);
      process.exitCode = 1;
    }
  });
  server.closeIdleConnections();
}

process.once('SIGTERM', shutdown);
process.once('SIGINT', shutdown);
```

| Line / API | Role | What happens at runtime |
| :--- | :--- | :--- |
| `import ... from './app.ts'` | Local import with the real extension | Node's ESM loader needs a full file name. Type stripping loads the `.ts` file; `tsc` rewrites it to `./app.js` in `dist/` (`rewriteRelativeImportExtensions`). |
| `loadConfig()` | Read and validate `PORT`, `HOST`, `SHUTDOWN_TIMEOUT_MS` | A bad `PORT=80a` throws `RangeError` before anything listens, so the container fails fast with exit code 1. |
| `app.listen(port, host)` | Bind the socket | Express creates an `http.Server`; libuv registers a listening TCP handle with epoll/kqueue/IOCP. The call returns before the bind completes. |
| `await once(server, 'listening')` | Top-level await | The module pauses until the event fires. If `'error'` fires first (`EADDRINUSE`), the promise rejects and the process exits with the error. |
| `host = '0.0.0.0'` | Listen on every interface | Inside a container, `127.0.0.1` would only accept connections from the container itself and the health check from outside would fail. |
| `process.once('SIGTERM', ...)` | Graceful shutdown | Docker and Kubernetes send SIGTERM before SIGKILL. Without a handler Node dies immediately and drops in-flight requests. |
| `server.close(cb)` | Stop accepting connections | Existing requests finish; the callback runs when the last connection closes. |
| `server.closeIdleConnections()` | Close idle keep-alive sockets | A browser's idle keep-alive connection would otherwise keep `close()` waiting until it times out. |
| `deadline.unref()` | A timer that does not keep the process alive | If everything closes in time, the loop is empty and Node exits with code 0 without waiting for the timer. |

`src/app.ts` builds the application without listening, which is what lets tests start it on an ephemeral port:

```ts
export function createApp(version: string): express.Express {
  const app = express();
  app.disable('x-powered-by');
  app.use(express.json({ limit: '16kb' }));

  app.get('/health', health);
  app.post('/api/sandbox/event-loop', runEventLoopSimulation);
  // ... more routes ...

  app.use('/api', (_req, _res, next) => next(new HttpError(404, 'Route not found')));
  app.use(express.static(fileURLToPath(new URL('../public', import.meta.url))));

  app.use((err: unknown, _req: Request, res: Response, _next: NextFunction) => {
    const status = statusOf(err);
    if (status >= 500) console.error(err);
    const message = status >= 500 ? 'Internal server error' : (err as Error).message;
    res.status(status).json({ success: false, error: message });
  });
  return app;
}
```

| Line / API | Role | What happens at runtime |
| :--- | :--- | :--- |
| `express.json({ limit: '16kb' })` | Parse JSON bodies with a size cap | Larger bodies get `413`, malformed JSON gets `400`; both reach the error middleware with a `status` property. |
| `new URL('../public', import.meta.url)` | Path relative to this file, not to `cwd` | Works from `src/` in development and from `dist/` in the image because both sit one level below the root. |
| Four-argument middleware | Express error handler | Express 5 forwards errors thrown or rejected by `async` handlers here, so controllers contain no `try/catch`. |
| `status >= 500 ? 'Internal server error'` | Do not leak internals | Client errors explain themselves; server errors are logged and replaced with a generic message. |

The tests in `test/app.test.ts` show the pattern used throughout the guide: listen on port `0`, read the assigned port, call the server with the built-in `fetch`.

```ts
before(async () => {
  server = createApp('9.9.9').listen(0, '127.0.0.1');
  await once(server, 'listening');
  const { port } = server.address() as AddressInfo;
  baseUrl = `http://127.0.0.1:${port}`;
});
```

### 1.5 Inside V8: Parsing, Tiered Compilation and Deoptimisation

V8 does not compile everything to machine code up front. It starts cheap and spends compile time only on code that proves to be hot. Functions move through four tiers:

```mermaid
flowchart LR
    Src["Source text"] --> Parse["Parser: lazy pre-parse, full parse on first call"]
    Parse --> Ign["Ignition: bytecode interpreter"]
    Ign -->|"warm"| Spark["Sparkplug: baseline compiler, no optimisation"]
    Spark -->|"hot, stable types"| Mag["Maglev: fast optimising compiler"]
    Mag -->|"very hot"| TF["TurboFan: full optimising compiler"]
    Ign -.->|"collects type feedback"| FB[("Feedback vector")]
    FB -.-> Mag
    FB -.-> TF
    Mag -.->|"assumption broken: deopt"| Ign
    TF -.->|"assumption broken: deopt"| Ign
```

| Tier | Input | Output | Why it exists |
| :--- | :--- | :--- | :--- |
| **Ignition** | AST | Compact bytecode | Starts executing quickly with low memory; records which types each operation sees. |
| **Sparkplug** | Bytecode | Unoptimised machine code | Removes interpreter dispatch overhead without spending time on analysis. |
| **Maglev** | Bytecode + feedback | Optimised machine code | Gets most of the speed of TurboFan at a fraction of its compile time. On by default since V8 11.x (Node 22+). |
| **TurboFan** | Bytecode + feedback | Highly optimised machine code | Inlining, escape analysis and type specialisation for the hottest functions. |

Optimised code is built on **assumptions** taken from the feedback vector: "`order.price` has always been a small integer", "this call site has always seen objects of one shape". When an assumption fails, V8 throws the optimised code away and resumes in Ignition. This is a **deoptimisation**. A few are normal; a function that deoptimises repeatedly is slow and V8 may stop trying to optimise it.

You can watch it happen:

```ts
// deopt-demo.ts — run with: node --trace-opt --trace-deopt deopt-demo.ts | grep -E "total|optimiz|deoptimiz"
function total(order: { price: number; quantity: number }): number {
  return order.price * order.quantity;
}

let sum = 0;
for (let i = 0; i < 200_000; i++) sum += total({ price: i, quantity: 2 });

// A double where only small integers were seen: the optimised code's assumption breaks.
sum += total({ price: 1.5, quantity: 2 });
console.log(sum);
```

The trace shows `total` being marked and compiled for optimisation, then a deoptimisation line that names the failed check (with this input, a check that the value is a *Smi*, V8's small-integer representation). The exact wording differs between V8 versions; the pattern of optimise, deoptimise, re-optimise is what to look for.

### 1.6 Hidden Classes and Inline Caches

JavaScript objects look like dictionaries, but V8 stores most of them as compact structs. Each object points to a **hidden class** (V8 calls it a *Map*) that records property names and their offsets. Objects created the same way share a hidden class.

```mermaid
flowchart LR
    C0["Map C0: empty"] -->|"add name"| C1["Map C1: name at offset 0"]
    C1 -->|"add email"| C2["Map C2: name 0, email 1"]
    C2 -->|"add role later"| C3["Map C3: name 0, email 1, role 2"]
    C1 -->|"add email first, then name, elsewhere"| CX["Different Map: email 0, name 1"]
```

At every property access V8 keeps an **inline cache** (IC) that remembers the hidden classes it has seen there:

| IC state | Shapes seen at the site | Cost of `obj.name` |
| :--- | :--- | :--- |
| Monomorphic | 1 | One map check plus a fixed-offset load |
| Polymorphic | 2-4 | A short chain of map checks |
| Megamorphic | More than 4 | A slow generic lookup; optimisers give up on this site |

```ts
class User {
  name: string;
  email: string;
  role: string | null;

  constructor(name: string, email: string) {
    this.name = name;
    this.email = email;
    this.role = null; // declare every field in the constructor, even if it starts empty
  }
}

const a = new User('Ada', 'ada@example.test');
const b = new User('Linus', 'linus@example.test');
a.role = 'admin'; // same hidden class: the field already exists

// Anti-patterns that create extra hidden classes:
const p = { x: 1, y: 2 };
const q = { y: 2, x: 1 };          // different insertion order -> different Map
const r: Record<string, number> = { x: 1 };
delete r.x;                        // delete often turns the object into slow dictionary mode
```

In application code the rule is simple: create objects of one kind in one place (a constructor or factory) with all fields present, and do not `delete` properties on hot objects. Set them to `undefined` or `null` instead. You rarely need to think further than that; measure with `--cpu-prof` (Stage 6) before reshaping code for the optimiser.

### 1.7 V8 Memory: Stack, Heap and Garbage Collection

Each thread running JavaScript has a **call stack** (frames for active function calls, a fixed size of about 1 MB by default) and shares the isolate's **heap**, where objects, closures, strings and arrays live. Too-deep recursion overflows the stack (`RangeError: Maximum call stack size exceeded`); too much live data exhausts the heap (`FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory`).

```mermaid
flowchart TD
    subgraph Heap["V8 heap of one isolate"]
        subgraph Young["Young generation"]
            Nursery["Semi-space A: new allocations"]
            Inter["Semi-space B: survivors of one scavenge"]
        end
        Old["Old space: objects that survived two scavenges"]
        Code["Code space: compiled machine code"]
        LO["Large object space: objects too big for a page"]
        RO["Read-only space: immutable built-ins"]
    end
    Ext["Outside the heap: ArrayBuffer and Buffer backing stores"]
    Nursery -->|"scavenge: copy live objects"| Inter
    Inter -->|"promote"| Old
```

V8's collector is **generational**: most objects die young, so new objects go into a small young generation that is collected often and cheaply.

| Collector | Collects | How | Pause |
| :--- | :--- | :--- | :--- |
| **Scavenger** (minor GC) | Young generation | Copies live objects to the other semi-space; dead objects cost nothing. Parallel across helper threads. | Usually under a few milliseconds |
| **Mark-Compact** (major GC) | Whole heap | Marks reachable objects starting from roots, sweeps the rest, compacts fragmented pages. Marking runs concurrently and incrementally. | Short main-thread pauses; longer when the heap is huge and nearly full |

The old-generation limit is derived from the machine's memory and capped by V8 (often 2-4 GB on 64-bit systems). Read it instead of guessing:

```bash
node -p "Math.round(v8.getHeapStatistics().heap_size_limit / 1024 / 1024) + ' MB'"
node --max-old-space-size=1536 -p "Math.round(v8.getHeapStatistics().heap_size_limit / 1024 / 1024) + ' MB'"
```

In a container, set `--max-old-space-size` to roughly 75% of the memory limit. The remaining 25% covers Buffers (stored outside the heap), thread stacks, native libraries and code. If you skip this and the V8 default is larger than the container limit, the kernel's OOM killer ends the process with exit code 137 and no JavaScript stack trace.

```dockerfile
ENV NODE_OPTIONS="--max-old-space-size=384"   # for a 512 MiB container limit
```

`process.memoryUsage()` shows the split; the starter returns it from `/api/diagnostics`:

| Field | Meaning | Grows when |
| :--- | :--- | :--- |
| `rss` | Resident set size: all memory the OS has given the process | Anything grows: heap, buffers, native memory |
| `heapTotal` | Heap reserved by V8 | V8 expands the heap for more objects |
| `heapUsed` | Heap occupied by live (or not yet collected) objects | You retain JavaScript objects |
| `external` | Memory of C++ objects tied to JS objects, including Buffers | You hold Buffers or native handles |
| `arrayBuffers` | Subset of `external` for `ArrayBuffer`/`Buffer` backing stores | You read files or sockets into memory |

### 1.8 The libuv Event Loop: Phases in Order

The event loop is a `while` loop inside libuv. Each iteration visits a fixed sequence of phases; each phase has its own queue of callbacks. When there are no timers, no open handles (servers, sockets, intervals) and no pending requests left, the loop exits and so does the process.

```mermaid
flowchart TD
    Start(["Loop iteration"]) --> T["timers: expired setTimeout / setInterval callbacks"]
    T --> P["pending callbacks: some deferred system errors, e.g. TCP ECONNREFUSED"]
    P --> I["idle, prepare: internal to libuv and Node"]
    I --> Poll["poll: wait for I/O, run I/O callbacks (sockets, files done, http requests)"]
    Poll --> C["check: setImmediate callbacks"]
    C --> Close["close callbacks: 'close' events, e.g. socket.on('close')"]
    Close --> Alive{"Any timers, handles or requests left?"}
    Alive -- yes --> Start
    Alive -- no --> Exit(["process exits with process.exitCode"])
```

| Phase | Runs | Typical sources | Detail worth knowing |
| :--- | :--- | :--- | :--- |
| **timers** | Callbacks whose delay has elapsed | `setTimeout`, `setInterval` | A delay is a minimum, not a promise; `0` is clamped to `1` ms. |
| **pending callbacks** | I/O callbacks deferred from the previous iteration | Certain TCP errors | Rarely visible from JavaScript. |
| **idle, prepare** | Internal hooks | Node internals | Not exposed. |
| **poll** | Completed I/O callbacks; blocks waiting for I/O when nothing else is scheduled | Incoming connections, socket data, finished `fs` operations | The block time is computed from the nearest timer, so timers still fire. |
| **check** | `setImmediate` callbacks | `setImmediate` | Always runs right after poll. |
| **close callbacks** | `'close'` handlers | `socket.destroy()`, `server.close()` | Cleanup code lives here. |

Two orderings follow directly from this picture and are worth memorising:

1. **Inside an I/O callback** (a request handler, a `fs.readFile` callback), `setImmediate` always runs before `setTimeout(fn, 0)`: check comes right after poll, timers only on the next iteration.
2. **In the main module**, the order of those two is not defined. It depends on whether 1 ms has passed by the time the loop starts.

```ts
// examples/02-io-callback-order.ts
import { readFile } from 'node:fs';

readFile(import.meta.filename, () => {
  setTimeout(() => console.log('timers: setTimeout 0'), 0);
  setImmediate(() => console.log('check: setImmediate'));
});
```

```text
check: setImmediate
timers: setTimeout 0
```

The starter's `POST /api/sandbox/event-loop` schedules one callback on each queue from inside a request handler and returns the observed order. `test/app.test.ts` asserts it is always `sync, nextTick, microtask, check, timer`.

libuv 1.45 (bundled since Node 20) moved the timers step to the end of each iteration, after the close callbacks. The cyclic order in the diagram is the same, so code that follows the two rules above behaves identically; only code that measured exact timer latency around `setImmediate` noticed the change.

### 1.9 Microtasks: the `process.nextTick` and Promise Queues

Two more queues sit outside libuv and run **between every callback**, not only between phases:

- the **nextTick queue**, owned by Node (`process.nextTick(fn)`);
- the **microtask queue**, owned by V8 (Promise reactions, `queueMicrotask(fn)`, `await` continuations).

When a callback returns, Node drains the nextTick queue completely, then V8 drains the microtask queue completely, repeating until both are empty. Only then does the loop move on. A callback that keeps scheduling microtasks therefore starves I/O exactly like an infinite loop would.

```mermaid
flowchart TD
    CB["A macrotask callback returns"] --> NT{"nextTick queue empty?"}
    NT -- no --> RunNT["Run all nextTick callbacks"] --> NT
    NT -- yes --> MQ{"microtask queue empty?"}
    MQ -- no --> RunMQ["Run all Promise jobs and queueMicrotask callbacks"] --> NT
    MQ -- yes --> Next["Next callback in the current phase, or the next phase"]
```

The order between the two queues depends on **where your code is running**. This is the most common wrong answer in interviews and in older tutorials, so the repository tests both cases:

```ts
// examples/01-event-loop-order.ts (an ES module) and examples/01-event-loop-order.cjs (CommonJS)
console.log('sync: start');

setTimeout(() => console.log('timers: setTimeout 0'), 0);
setImmediate(() => console.log('check: setImmediate'));

process.nextTick(() => {
  console.log('nextTick: first');
  process.nextTick(() => console.log('nextTick: scheduled by nextTick'));
});

Promise.resolve()
  .then(() => console.log('promise: first then'))
  .then(() => console.log('promise: second then'));

queueMicrotask(() => console.log('promise: queueMicrotask'));

console.log('sync: end');
```

| Position | CommonJS (`.cjs`) output | ES module (`.ts`, `.mjs`) output |
| :--- | :--- | :--- |
| 1 | `sync: start` | `sync: start` |
| 2 | `sync: end` | `sync: end` |
| 3 | `nextTick: first` | `promise: first then` |
| 4 | `nextTick: scheduled by nextTick` | `promise: queueMicrotask` |
| 5 | `promise: first then` | `promise: second then` |
| 6 | `promise: queueMicrotask` | `nextTick: first` |
| 7 | `promise: second then` | `nextTick: scheduled by nextTick` |
| 8-9 | timers and check, order not defined | timers and check, order not defined |

Why they differ: a CommonJS module body runs as a plain synchronous call, so when it returns Node drains nextTick first. An ES module body is evaluated by V8 as part of a Promise job. When the body returns you are still inside the microtask queue, and V8 keeps draining it (including the `.then` chains you just created) before control comes back to Node and its nextTick queue. The same thing happens in any code that runs after an `await`: `test/services.test.ts` hits it because test bodies are async functions.

Practical consequences:

- Do not write code whose correctness depends on nextTick-before-promise ordering. Use explicit `await` or callbacks instead.
- Prefer `queueMicrotask()` over `process.nextTick()` in new code: it is a web standard and its ordering is the same everywhere.
- Use `setImmediate()` when you want to yield to I/O, for example between batches of CPU work.

### 1.10 The libuv Thread Pool and `UV_THREADPOOL_SIZE`

"Node is single-threaded" is true only for your JavaScript. Network sockets use the operating system's non-blocking APIs (epoll on Linux, kqueue on macOS, IOCP on Windows) and need no threads. Operations that the OS cannot do asynchronously run on libuv's **thread pool**:

```mermaid
flowchart TD
    Call["Async API call"] --> Kind{"What kind of work?"}
    Kind -->|"TCP, UDP, pipes, HTTP"| Kernel["Kernel readiness API: epoll, kqueue, IOCP"]
    Kind -->|"fs.*, dns.lookup, crypto.pbkdf2/scrypt/randomBytes, zlib async"| Pool["libuv thread pool: 4 threads by default, max 1024"]
    Kernel --> Q["Callback queued for the poll phase"]
    Pool --> Q
    Q --> Main["Main thread runs the callback"]
```

| Uses the pool | Does not use the pool |
| :--- | :--- |
| `fs.readFile`, `fs.stat`, every async `fs` call except `fs.watch` | `net`, `http`, `https`, `tls` socket I/O |
| `dns.lookup()` (calls the blocking `getaddrinfo`) | `dns.resolve*()` (c-ares, non-blocking sockets) |
| `crypto.pbkdf2`, `scrypt`, `randomBytes`, `randomFill`, `generateKeyPair` (async forms) | `crypto.createHash()` (runs synchronously on the main thread) |
| Async `zlib` (`gzip`, `brotliCompress`, streams) | Timers, `setImmediate` |

With four threads, the fifth concurrent pool job waits. A slow disk or a burst of password hashes can therefore delay unrelated work, such as DNS lookups for outgoing HTTP calls, that also needs the pool. `examples/03-threadpool.ts` makes the waves visible:

```ts
import { pbkdf2 } from 'node:crypto';

const start = performance.now();
console.log(`thread pool size: ${process.env.UV_THREADPOOL_SIZE ?? '4 (default)'}`);

for (let i = 1; i <= 8; i++) {
  pbkdf2('password', 'salt', 100_000, 64, 'sha512', (err) => {
    if (err) throw err;
    console.log(`hash ${i} finished at ${Math.round(performance.now() - start)} ms`);
  });
}
```

```text
$ node examples/03-threadpool.ts
thread pool size: 4 (default)
hash 4 finished at 164 ms
hash 3 finished at 173 ms
hash 2 finished at 179 ms
hash 1 finished at 213 ms
hash 6 finished at 353 ms
hash 7 finished at 369 ms
hash 5 finished at 374 ms
hash 8 finished at 398 ms

$ UV_THREADPOOL_SIZE=8 node examples/03-threadpool.ts
thread pool size: 8
hash 4 finished at 251 ms
...                                    all eight finish together, each a little slower
```

The two waves of four are the pool at work. With eight threads they finish together, but each one takes longer because they now compete for the same CPU cores. Raising the pool size helps when jobs **wait** (disk, `getaddrinfo`), not when they **compute** on a machine with few cores.

Set the size before the pool starts, which in practice means in the environment:

```bash
UV_THREADPOOL_SIZE=16 node dist/server.js
```

Setting `process.env.UV_THREADPOOL_SIZE` inside your code only works if no pool work has happened yet, which is hard to guarantee once any dependency touches the file system at import time.

### 1.11 Pitfalls

| Pitfall | Symptom | Fix |
| --- | --- | --- |
| CPU-heavy work in a request handler | Every other request stalls; p99 latency jumps; health checks time out and the orchestrator restarts the pod | Move the work to `worker_threads` (Stage 5) or chunk it with `setImmediate` between batches. |
| Assuming `setTimeout(fn, 0)` runs before `setImmediate(fn)` | Order changes between runs or machines | Inside I/O callbacks `setImmediate` is first; at top level the order is undefined. Do not depend on it. |
| Recursive `process.nextTick` | Process at 100% CPU, no I/O served, timers never fire | Use `setImmediate` to yield, or restructure as a loop with an exit condition. |
| Relying on nextTick running before Promise jobs | Works in a `.cjs` file, breaks after moving to ESM or after an `await` | Express ordering with `await`. |
| No `--max-old-space-size` in a container | Pod killed with exit code `137`, `OOMKilled`, no stack trace | Set it to about 75% of the memory limit through `NODE_OPTIONS`. |
| Listening on `localhost` in Docker | `curl: (52) Empty reply from server` or connection refused from the host | Bind `0.0.0.0` (the starter's default `HOST`). |
| Thread pool exhaustion by `dns.lookup` | Outgoing HTTP calls slow down when disk or crypto is busy | Raise `UV_THREADPOOL_SIZE`, cache DNS, or use `dns.resolve*()` for your own lookups. |
| Calling `process.exit()` after writing to stdout | Last log lines missing when stdout is a pipe | Set `process.exitCode` and let the loop drain. |
| Installing Node from the OS package manager | Old major, global installs need `sudo`, CI uses another version | Use `fnm` or `nvm` plus `.nvmrc` and `engines`. |

### 1.12 Best Practices

- **Pin Node per project with `.nvmrc` and `engines`**: local, CI and container then run the same major.
- **Run only LTS lines in production**: odd majors lose support about two months after the next even major becomes LTS.
- **Keep each callback short**: the loop serves everyone only between callbacks, so a 200 ms callback is a 200 ms outage for every connection.
- **Separate building the app from starting it** (`createApp()` vs `server.ts`): tests can start it on port `0` and the process lifecycle lives in one place.
- **Handle `SIGTERM` and close the server**: rolling deploys then drop no requests.
- **Size the heap explicitly in containers**: an OOM kill gives you no diagnostics; a V8 heap limit gives you an error and a chance to write a heap snapshot.
- **Prefer `queueMicrotask` and `setImmediate` over `process.nextTick`**: their behaviour is standard and easier to reason about.
- **Read the runtime instead of guessing**: `process.versions`, `v8.getHeapStatistics()` and `process.getActiveResourcesInfo()` answer most "what is Node doing" questions.

### 1.13 Hands-on Exercise

**Task.** Predict, then verify, the output of the scheduling scripts.

1. Without running anything, write down the output of `examples/01-event-loop-order.cjs` and `examples/01-event-loop-order.ts`.
2. Run both. Explain each difference using section 1.9.
3. Add a `setImmediate` inside the first `.then` callback in the `.cjs` file. Predict where its line appears, run it, and check.
4. Run `examples/03-threadpool.ts` with `UV_THREADPOOL_SIZE` set to 1, 4 and 8. Record the finish time of the last hash for each and explain the shape of the results with the number of CPU cores on your machine (`node -p "os.availableParallelism()"`).
5. Start the starter server, open `http://localhost:8080`, run the event-loop sandbox, and compare its order with step 2.

**Expected output.** Steps 1-2 match the table in 1.9. In step 3 the new line prints directly after `check: setImmediate`: both immediates sit in the same check queue in the order they were scheduled. Whether that pair comes before or after the timer line still varies between runs. In step 4, size 1 is roughly eight times the single-hash time; size 4 shows two waves; size 8 helps only if you have eight or more cores.

**Solution.** The expected orders are asserted in `test/examples.test.ts`; run `npm test` to check your understanding against the runtime.

[Back to top ⤴️](#-table-of-contents)

---

## 2. Stage 2: Modules, Packages & Tooling

Stage 1 ran single files. Real programs are many files plus dependencies, and Node has two module systems that must work together. This stage covers how Node decides whether a file is CommonJS or an ES module, how specifiers resolve, how the two systems interoperate, how `package.json` controls what a package exposes, how npm installs reproducibly, and the built-in tooling (type stripping, `--watch`, `--env-file`, `node --run`) that replaces several dev dependencies. Afterwards you can structure a project, publish a package that both module systems can consume, and run TypeScript with no build step in development.

```mermaid
flowchart LR
    Spec["Specifier in import or require()"] --> Kind{"Kind?"}
    Kind -->|"node:fs"| Builtin["Built-in module"]
    Kind -->|"./x.js, ../y.ts"| Rel["Relative file"]
    Kind -->|"#internal/db"| Imports["package.json imports map"]
    Kind -->|"express, @scope/pkg/sub"| Bare["Bare specifier"]
    Bare --> NM["Walk up node_modules directories"]
    NM --> Exports["Apply the package's exports map and conditions"]
    Rel --> Type{"Module type?"}
    Imports --> Type
    Exports --> Type
    Type -->|"ESM"| ESMLoad["Parse, link, evaluate (async graph)"]
    Type -->|"CommonJS"| CJSLoad["Wrap in a function, run synchronously"]
```

### 2.1 CommonJS vs ES Modules

**CommonJS** (CJS) is Node's original system: `require()` is an ordinary function that reads a file, wraps it in a function, runs it and returns `module.exports`. **ECMAScript modules** (ESM) are the language standard: `import` and `export` are syntax, the whole dependency graph is parsed and linked before any code runs, and imports are live read-only bindings.

```mermaid
sequenceDiagram
    participant Loader as ESM loader
    participant A as main.mjs
    participant B as db.mjs
    Loader->>A: 1. Parse: find static imports
    Loader->>B: 2. Fetch and parse dependencies
    Loader->>Loader: 3. Link: bind every import to its export slot
    Loader->>B: 4. Evaluate db.mjs (may use top-level await)
    Loader->>A: 5. Evaluate main.mjs
    Note over A,B: CommonJS instead runs each file the moment require() is called
```

| Dimension | CommonJS | ES modules |
| :--- | :--- | :--- |
| Syntax | `const x = require('x')`, `module.exports = ...` | `import x from 'x'`, `export const ...` |
| When dependencies load | At the `require()` call, synchronously | Before evaluation, as a graph |
| Top-level `await` | Not allowed | Allowed |
| Exports | A mutable object copied at `require()` time | Live bindings: importers see updates |
| Conditional loading | `if (x) require('y')` | `await import('y')` |
| File and directory names | `__filename`, `__dirname` | `import.meta.filename`, `import.meta.dirname` (20.11+), `import.meta.url` |
| Extension in relative paths | Optional (`./util` finds `./util.js`) | Required (`./util.js`) |
| JSON | `require('./data.json')` | `import data from './data.json' with { type: 'json' }` |
| `this` at top level | `module.exports` | `undefined` |
| Strict mode | Opt-in (`'use strict'`) | Always |

New code should be ESM. It is the standard, it is what browsers, Deno and Bun run, and since `require(esm)` (2.3) it no longer cuts you off from CommonJS consumers. The starter project sets `"type": "module"` and every file under `src/` is ESM.

A worked example of live bindings versus copies:

```ts
// counter.ts (ESM)
export let count = 0;
export function increment(): void {
  count++;
}

// main.ts
import { count, increment } from './counter.ts';
increment();
console.log(count); // 1: the import is a live view of the exporting module's variable
// count = 5;       // TypeError: Assignment to constant variable. Imports are read-only.
```

In CommonJS, `const { count } = require('./counter.cjs')` copies the number at that moment and would still print `0`.

### 2.2 How Node Picks a Module Type and Resolves a Specifier

For every file Node first decides which loader to use:

```mermaid
flowchart TD
    F["File to load"] --> Ext{"Extension"}
    Ext -->|".mjs / .mts"| ESM["ES module"]
    Ext -->|".cjs / .cts"| CJS["CommonJS"]
    Ext -->|".js / .ts"| Pkg{"Nearest package.json has type?"}
    Pkg -->|"type: module"| ESM
    Pkg -->|"type: commonjs"| CJS
    Pkg -->|"no type field"| Detect{"Source contains import/export or top-level await?"}
    Detect -->|"yes (warns MODULE_TYPELESS_PACKAGE_JSON)"| ESM
    Detect -->|"no"| CJS
```

Syntax detection (the last box) is on by default since 22.7 and 20.19, but it costs an extra parse and prints a warning. Always set `"type"` explicitly.

**Resolution** then turns a specifier into a file:

| Specifier | CommonJS `require()` | ESM `import` |
| :--- | :--- | :--- |
| `node:fs` | Built-in | Built-in |
| `fs` (no prefix) | Built-in | Built-in. Prefer `node:` so a package named `fs` can never shadow it. |
| `./util` | Tries `./util`, `./util.js`, `./util.json`, `./util.node`, then `./util/index.js` | `ERR_MODULE_NOT_FOUND`: the extension is required |
| `./util.js` | That file | That file |
| `express` | Walks `./node_modules`, `../node_modules`, ... to the root; applies `exports`, else `main`, else `index.js` | Same walk; applies `exports`, else `main` |
| `#internal/db` | Package's `imports` map | Package's `imports` map |
| `/abs/path.js`, `file:///abs/path.js` | Absolute path | Absolute path or URL |

Ask the resolver instead of guessing:

```bash
node -p "require.resolve('express')"                       # CommonJS view
node --input-type=module -e "console.log(import.meta.resolve('express'))"   # ESM view
```

`import.meta.resolve()` is synchronous and returns a `file://` URL. Both answers can differ for the same package when its `exports` map has separate `import` and `require` conditions.

### 2.3 Interop: `require(esm)`, Import of CommonJS and `createRequire`

Most projects mix both systems: your code is ESM, some dependencies are still CommonJS, and some consumers of your library are CommonJS. The rules:

```mermaid
flowchart LR
    subgraph FromESM["From an ES module"]
        I1["import cjs from './legacy.cjs'"] --> R1["default = module.exports"]
        I2["import { add } from './legacy.cjs'"] --> R2["named exports found by static analysis"]
        I3["createRequire(import.meta.url)"] --> R3["a require() for JSON, .node addons, CJS"]
    end
    subgraph FromCJS["From a CommonJS module"]
        Q1["require('./lib.mjs')"] --> S1["module namespace object, if the graph is synchronous"]
        Q2["require('./tla.mjs')"] --> S2["ERR_REQUIRE_ASYNC_MODULE"]
        Q3["await import('./tla.mjs')"] --> S3["always works, returns a promise"]
    end
```

The repository has a runnable version of each case in `examples/modules/`:

```js
// examples/modules/cjs-consumer.cjs — CommonJS loading an ES module
const lib = require('./esm-lib.mjs');

console.log(lib.greet('commonjs'));   // hello, commonjs
console.log(lib.default);             // default export value

try {
  require('./tla.mjs');               // this module uses top-level await
} catch (err) {
  console.log(`require of a top-level-await module: ${err.code}`);
  // -> ERR_REQUIRE_ASYNC_MODULE
}
```

```js
// examples/modules/esm-consumer.mjs — an ES module loading CommonJS
import { createRequire } from 'node:module';
import legacy, { add } from './legacy.cjs';

console.log(`named import from CJS: ${add(2, 3)}`);                 // 5
console.log(`default import is module.exports: ${legacy.kind}`);    // commonjs

const require = createRequire(import.meta.url);
console.log(`createRequire loads JSON: ${require('../../package.json').name}`);
```

| Line / API | Role | What happens at runtime |
| :--- | :--- | :--- |
| `require('./esm-lib.mjs')` | `require(esm)` | Node loads and evaluates the ES module graph synchronously and returns its namespace object (`{ greet, default }`). Available without a flag in 20.19+, 22.12+ and 23+; early releases print an `ExperimentalWarning`. |
| `ERR_REQUIRE_ASYNC_MODULE` | Refusal for async graphs | A module with top-level `await` cannot finish synchronously, so `require()` throws. Use `await import()` or remove the top-level `await`. |
| `import legacy from './legacy.cjs'` | Default import of CJS | Always the whole `module.exports` value. |
| `import { add } from './legacy.cjs'` | Named import of CJS | Node's `cjs-module-lexer` scans the source for patterns like `exports.add =` and `module.exports = { add }`. Exports assigned dynamically (inside loops, through computed keys) are invisible; fall back to the default import. |
| `createRequire(import.meta.url)` | A CommonJS `require` inside ESM | Useful for `.node` native addons, for `require.resolve()`, and for packages that only work through `require`. |

`process.features.require_module` is `true` when the running Node supports `require(esm)`; libraries can test it before relying on the feature.

**The dual package hazard.** If a package ships separate CJS and ESM builds and an application loads both (one dependency requires it, another imports it), the process ends up with two copies of the module, each with its own state: two singletons, two caches, `instanceof` checks that fail across them. With `require(esm)` available on every supported Node line, the simplest fix for library authors is to ship one ESM build and let CommonJS consumers `require()` it.

### 2.4 `package.json`: `type`, `exports`, `imports` and Conditions

`exports` is the public API of a package. Once it exists, only the paths it lists can be imported; everything else is private, even though the files are on disk. `imports` gives the package private aliases that start with `#`.

`examples/packages/greeter` is a complete package:

```json
{
  "name": "greeter",
  "version": "1.0.0",
  "private": true,
  "type": "module",
  "exports": {
    ".": "./src/index.js",
    "./mode": {
      "development": "./src/mode-development.js",
      "default": "./src/mode-production.js"
    },
    "./package.json": "./package.json"
  },
  "imports": {
    "#internal/*": "./src/internal/*.js"
  }
}
```

```js
// examples/packages/greeter/src/index.js
import { shout } from '#internal/format';

export function greet(name) {
  return shout(`hello, ${name}`);
}
```

```js
// examples/packages/greeter/check.mjs — a package can import itself by name ("self-reference")
import { greet } from 'greeter';
import { mode } from 'greeter/mode';

console.log(greet('esm'));          // HELLO, ESM
console.log(`mode: ${mode}`);       // production, or development with -C development

try {
  await import('greeter/src/internal/format.js');
} catch (err) {
  console.log(`deep import blocked: ${err.code}`);   // ERR_PACKAGE_PATH_NOT_EXPORTED
}
```

```bash
node examples/packages/greeter/check.mjs
node --conditions=development examples/packages/greeter/check.mjs
node examples/packages/greeter/check.cjs          # same exports map, loaded through require(esm)
```

Conditions are matched **in the order they appear in the object**, and the first match wins:

```mermaid
flowchart TD
    Req["import 'pkg' or require('pkg'), with active conditions"] --> C1{"types?"}
    C1 -->|"TypeScript only"| T["./dist/index.d.ts"]
    C1 -->|"Node skips it"| C2{"development? (only with -C development)"}
    C2 -->|"active"| D["./src/index.ts"]
    C2 -->|"not active"| C3{"import (ESM) or require (CJS)?"}
    C3 -->|"import"| E["./dist/index.js"]
    C3 -->|"require"| R["./dist/index.cjs"]
    C3 -->|"neither matched"| Def["default: ./dist/index.js"]
```

| Condition | Matches when | Notes |
| :--- | :--- | :--- |
| `types` | TypeScript resolves types | Put it first; Node ignores it. |
| `node` | Running in Node | Use for Node-only builds next to a `browser` entry. |
| `import` | Loaded with `import` or `import()` | |
| `require` | Loaded with `require()` | |
| `module-sync` | Loader can `require()` a synchronous ES module | Lets one ESM file serve both `import` and `require` on modern Node. |
| custom (`development`, `production`) | `node -C name` / `--conditions=name` | Tooling such as bundlers can set them too. |
| `default` | Always | Must be last; anything after it is unreachable. |

Rules that prevent most packaging bugs:

- Always export `"./package.json"` if tools need to read it; `exports` hides it otherwise.
- Use subpath patterns (`"./features/*": "./dist/features/*.js"`) instead of listing hundreds of files.
- Adding `exports` to an existing package is a **breaking change** for anyone who imported a deep path. Release it in a major version.
- Use `imports` (`#internal/*`) instead of `../../../` chains inside your own code. They resolve the same way under ESM, CommonJS and TypeScript (`moduleResolution: "nodenext"`).

### 2.5 npm: Semantic Versioning, Lockfiles and Scripts

npm installs dependencies into `node_modules` and records the exact resolved tree in `package-lock.json`.

| Range | Matches | Use for |
| :--- | :--- | :--- |
| `5.1.0` | Exactly 5.1.0 | Tools where any change must be reviewed |
| `~5.1.0` | `>=5.1.0 <5.2.0` (patches) | Conservative libraries |
| `^5.1.0` | `>=5.1.0 <6.0.0` (minors and patches) | The npm default; trusts semver |
| `^0.4.2` | `>=0.4.2 <0.5.0` | In 0.x, the minor is treated as breaking |
| `>=22.18.0` | Any newer version | `engines` fields |

```mermaid
sequenceDiagram
    participant Dev as npm install (developer)
    participant Lock as package-lock.json
    participant CI as npm ci (CI / Docker)
    participant Reg as npm registry
    Dev->>Reg: resolve ranges from package.json
    Reg-->>Dev: newest versions that match
    Dev->>Lock: write exact versions, integrity hashes
    CI->>Lock: read exact tree
    CI->>Reg: download exactly those tarballs, verify integrity
    Note over CI: fails if package.json and the lockfile disagree
```

| Command | What it does | When |
| :--- | :--- | :--- |
| `npm install` | Resolves ranges, may update the lockfile | Adding or upgrading dependencies locally |
| `npm ci` | Deletes `node_modules`, installs exactly the lockfile, never writes it | CI, Docker builds, reproducing a bug |
| `npm install express@^5` | Adds a dependency | |
| `npm install -D typescript` | Adds a dev dependency (not installed with `--omit=dev`) | Build and test tools |
| `npm ci --omit=dev` | Production-only install | The runtime stage of the Dockerfile |
| `npm outdated` | Shows current, wanted and latest versions | Planning upgrades |
| `npm ls <pkg>` / `npm explain <pkg>` | Why a package is in the tree | Investigating duplicates or a vulnerable transitive dependency |
| `npm audit` | Checks the tree against the advisory database | CI gate, with judgement about dev-only findings |
| `npm pkg set key=value` | Edits `package.json` from the shell | Scripts and setup docs |
| `npx tsc` / `npm exec tsc` | Runs a binary from `node_modules/.bin` (or downloads it) | One-off tools |

Scripts in `package.json` are the project's command interface. The starter defines:

```json
{
  "scripts": {
    "dev": "node --watch src/server.ts",
    "lint": "tsc --noEmit",
    "test": "node --test \"test/**/*.test.ts\"",
    "build": "tsc -p tsconfig.build.json",
    "start": "node dist/server.js"
  }
}
```

`node --run <script>` (added in Node 22) runs a script without starting npm. It starts faster, adds `node_modules/.bin` to `PATH`, but does not run `pre`/`post` hooks and does not set `npm_*` environment variables.

```bash
npm run lint          # through npm: npm's own startup adds noticeable time
node --run lint       # same script, no npm process
```

Lifecycle scripts (`preinstall`, `install`, `postinstall`) of **dependencies** run arbitrary code at install time and are a common supply-chain attack path. Install with `npm ci --ignore-scripts` where you can, and review which dependencies need install scripts (usually native addons).

### 2.6 Running TypeScript Natively with Type Stripping

Node 22.18+ and 24 run `.ts`, `.mts` and `.cts` files directly. Node does not type-check; it replaces type annotations with whitespace and runs what is left. Because positions do not move, stack traces point at the right line and column without source maps.

```mermaid
flowchart LR
    TS["server.ts with types"] --> Amaro["amaro (SWC): replace types with spaces"]
    Amaro --> JS["Same-length JavaScript"]
    JS --> V8["V8 runs it; stack traces match the .ts file"]
    TS -.-> TSC["tsc --noEmit: the only type checker"]
```

What it accepts and rejects:

| Syntax | Type stripping | Why |
| :--- | :--- | :--- |
| Annotations, interfaces, `type` aliases, generics, `as`, `satisfies`, `!` | Accepted | Erasable: deleting them leaves valid JavaScript |
| `import type { X }`, `import { type X }` | Accepted, removed | |
| `import { X }` where `X` is only a type | Runtime `SyntaxError: The requested module ... does not provide an export named 'X'` | Node cannot know `X` is a type; mark it with `type` |
| `enum`, `namespace` with values | `ERR_UNSUPPORTED_TYPESCRIPT_SYNTAX: TypeScript enum is not supported in strip-only mode` | Needs code generation |
| Constructor parameter properties (`constructor(private x: number)`) | Rejected | Generates assignments |
| Relative imports without an extension (`./app`) | `ERR_MODULE_NOT_FOUND` | ESM needs the real file name: `./app.ts` |
| `tsconfig.json` `paths` aliases | Ignored | Node never reads `tsconfig.json`; use `imports` (`#src/*`) instead |
| `.tsx` / JSX | Not supported | Use a bundler or `tsx` |
| `.ts` files inside `node_modules` | `ERR_UNSUPPORTED_NODE_MODULES_TYPE_STRIPPING` | Packages must publish JavaScript |

The compiler options in the starter's `tsconfig.json` make `tsc` reject exactly what Node would reject, so you find out at lint time instead of at runtime:

```jsonc
{
  "compilerOptions": {
    "target": "ES2023",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "types": ["node"],
    "strict": true,
    "noUncheckedIndexedAccess": true,
    "erasableSyntaxOnly": true,              // TS 5.8+: errors on enum, namespace, parameter properties
    "verbatimModuleSyntax": true,            // errors on type-only imports without `type`
    "rewriteRelativeImportExtensions": true, // TS 5.7+: lets you write ./x.ts and emits ./x.js
    "skipLibCheck": true,
    "noEmit": true
  },
  "include": ["src", "test", "examples"]
}
```

The `HttpError` class in `src/errors.ts` shows the one change you notice in day-to-day code: fields are declared and assigned explicitly instead of with parameter properties.

```ts
export class HttpError extends Error {
  readonly status: number;

  constructor(status: number, message: string, options?: ErrorOptions) {
    super(message, options);
    this.name = 'HttpError';
    this.status = status;
  }
}
```

The production image still compiles with `tsc -p tsconfig.build.json`. Compiling once at build time catches type errors in CI, avoids parsing TypeScript on every container start, and keeps the runtime independent of a feature that is still evolving (`--experimental-transform-types`, which adds `enum` support, remains experimental).

| Approach | Type checks | Startup cost | Supports enums, JSX, paths | Good for |
| :--- | :--- | :--- | :--- | :--- |
| `node file.ts` (type stripping) | No (run `tsc --noEmit` separately) | Small | No | Development, scripts, tests |
| `tsx file.ts` | No | Small (esbuild) | Yes | Projects that need full TS syntax without a build |
| `tsc` then `node dist/x.js` | Yes, at build | None at runtime | Enums yes, paths need rewriting | Production images, published packages |
| Bundler (esbuild, Rollup) | No | None | Yes | Single-file deploys, serverless |

### 2.7 Built-in Dev Tooling: `--watch`, `--env-file`, `parseArgs` and `styleText`

Four built-ins cover what `nodemon`, `dotenv`, `yargs` and `chalk` used to do for most projects.

**Watch mode** restarts the process when the entry file or anything it imports changes:

```bash
node --watch src/server.ts                         # what `npm run dev` runs
node --watch-path=src --watch-path=config src/server.ts   # watch extra directories
node --watch --watch-preserve-output src/server.ts # do not clear the console on restart
```

**Environment files** load `KEY=value` pairs before your code runs. Variables already set in the real environment take precedence over the file, so production secrets injected by the orchestrator are never overwritten by a stale file.

```bash
cat > .env.example <<'EOF'
PORT=8080
SHUTDOWN_TIMEOUT_MS=10000
EOF

node --env-file=.env src/server.ts
node --env-file-if-exists=.env.local --env-file=.env src/server.ts   # optional file first
```

```ts
import { parseEnv } from 'node:util';

process.loadEnvFile('.env');                           // same as --env-file, from code
const parsed = parseEnv('A=1\nB="two words"\n');       // { A: '1', B: 'two words' }
console.log(parsed.B);
```

**Argument parsing** with `util.parseArgs`:

```ts
// cli.ts — node cli.ts report.csv --limit 5 -v
import { parseArgs, styleText } from 'node:util';

const { values, positionals } = parseArgs({
  allowPositionals: true,
  options: {
    limit: { type: 'string', short: 'l', default: '10' },
    verbose: { type: 'boolean', short: 'v', default: false }
  }
});

const limit = Number(values.limit);
if (!Number.isInteger(limit) || limit <= 0) {
  console.error(styleText('red', `--limit must be a positive integer, got ${values.limit}`));
  process.exitCode = 2; // the usual exit code for a usage error
} else {
  console.log(styleText('green', `processing ${positionals[0] ?? 'stdin'}`), `limit=${limit}`);
  if (values.verbose) console.log(styleText('dim', JSON.stringify({ values, positionals })));
}
```

| Line / API | Role | What happens at runtime |
| :--- | :--- | :--- |
| `parseArgs({ options })` | Declarative flag parsing | Unknown flags throw `ERR_PARSE_ARGS_UNKNOWN_OPTION` unless `strict: false`. |
| `type: 'string'` | Value flags | Values are always strings; convert and validate yourself. |
| `allowPositionals: true` | Bare arguments | Without it, `report.csv` would throw `ERR_PARSE_ARGS_UNEXPECTED_POSITIONAL`. |
| `styleText(format, text)` | ANSI colours | Built in; recent versions skip colours when the stream is not a TTY or `NO_COLOR` is set. |

### 2.8 Essential CLI Flags and Environment Variables

| Flag / variable | Effect | Typical use |
| :--- | :--- | :--- |
| `--inspect`, `--inspect-brk`, `--inspect-wait` | Start the V8 inspector on 127.0.0.1:9229 (`-brk` pauses on the first line) | Debugging with Chrome DevTools or VS Code |
| `--watch` | Restart on file changes | Development |
| `--env-file=<path>` | Load environment variables from a file | Local configuration |
| `--test` | Run the built-in test runner | `npm test` |
| `--run <script>` | Run a `package.json` script without npm | Faster scripts |
| `--import <module>` / `-r <module>` | Preload an ESM / CJS module before the entry point | OpenTelemetry instrumentation, polyfills |
| `-C <name>` / `--conditions` | Activate a custom `exports` condition | `development` builds of internal packages |
| `--max-old-space-size=<MB>` | Old-generation heap limit | Containers (Stage 1.7) |
| `--heapsnapshot-near-heap-limit=<n>` | Write up to `n` heap snapshots as the heap approaches the limit | Diagnosing an OOM after the fact |
| `--cpu-prof`, `--heap-prof` | Write a CPU / sampling heap profile on exit | Performance work (Stage 6) |
| `--enable-source-maps` | Map stack traces of compiled code back to the source | Production with `tsc` output (`sourceMap: true`) |
| `--trace-warnings`, `--trace-deprecation` | Print stack traces for warnings | Finding which dependency triggers a warning |
| `--throw-deprecation` | Turn deprecation warnings into exceptions | Upgrade testing |
| `--unhandled-rejections=strict` | Throw on every unhandled rejection, even with a handler | Stricter than the default `throw` |
| `--permission`, `--allow-fs-read=<path>` | Enable the permission model | Defence in depth |
| `--stack-trace-limit=<n>` | Frames captured per stack trace (default 10) | Debugging deep async chains |
| `NODE_OPTIONS` | Extra flags for every `node` process, including children | Set flags when you cannot change the command, e.g. in Docker |
| `NODE_ENV` | **A convention, not read by Node itself.** Express, React and others change behaviour on `production` | Always set it in production images |
| `NODE_DEBUG=http,net` | Debug logs from core modules | Seeing what `http` does under the hood |
| `NODE_COMPILE_CACHE=<dir>` | Persist V8 compiled code between runs (22.1+) | Faster CLI startup |
| `NODE_EXTRA_CA_CERTS=<file>` | Trust extra root certificates | Corporate proxies |
| `UV_THREADPOOL_SIZE` | libuv pool size (Stage 1.10) | I/O-heavy services |
| `TZ=UTC` | Process time zone | Deterministic dates in tests |

### 2.9 The `node:util` Toolbox

`node:util` collects helpers that Node itself uses and that replace small npm packages:

```ts
import { readFile } from 'node:fs';
import util from 'node:util';

// promisify: error-first callback API -> Promise API
const readFileAsync = util.promisify(readFile);
const text = await readFileAsync(import.meta.filename, 'utf8');
console.log(text.length > 0);

// callbackify: the reverse, for libraries that still take callbacks
const add = util.callbackify(async (a: number, b: number) => a + b);
add(2, 3, (err, sum) => console.log(err ?? sum)); // 5

// inspect: what console.log uses; control depth and colours
const nested = { a: { b: { c: { d: { e: 'deep' } } } } };
console.log(util.inspect(nested, { depth: null, colors: false, compact: true }));

// types: brand checks that work across realms (worker threads, vm contexts)
console.log(util.types.isPromise(Promise.resolve()), util.types.isDate(new Date()));

// isDeepStrictEqual: the comparison behind assert.deepStrictEqual, as a boolean
console.log(util.isDeepStrictEqual({ x: [1, 2] }, { x: [1, 2] })); // true

// deprecate: warn once when old API is used
const oldApi = util.deprecate(() => 'still works', 'oldApi() is deprecated, use newApi()', 'DEP_APP_001');
oldApi();

// debuglog: logs only when NODE_DEBUG includes the section name
const debug = util.debuglog('starter');
debug('visible with NODE_DEBUG=starter');

// getSystemErrorName: errno number -> name
console.log(util.getSystemErrorName(-2)); // ENOENT on Linux and macOS
```

| Helper | Replaces | Gotcha |
| :--- | :--- | :--- |
| `util.promisify` | `pify`, hand-written wrappers | Many core APIs already have promise versions: `fs/promises`, `timers/promises`, `stream/promises`, `dns/promises`. Prefer those. |
| `util.inspect` | `JSON.stringify` for debugging | Handles cycles, `Map`, `Set`, symbols; output is not stable for machine parsing. |
| `util.parseArgs` | `yargs`, `minimist` for simple CLIs | No sub-commands or help text generation. |
| `util.styleText` | `chalk` | A small set of named styles. |
| `util.types.*` | `instanceof` | `instanceof Promise` fails for objects from another realm; `util.types` does not. |

### 2.10 Pitfalls

| Pitfall | Symptom | Fix |
| --- | --- | --- |
| Missing extension in an ESM import | `Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/app/src/app' imported from /app/src/server.ts` | Write the real file name: `./app.ts` (or `./app.js` for JavaScript). |
| Type-only import without `type` | `SyntaxError: The requested module 'node:http' does not provide an export named 'IncomingHttpHeaders'` | `import type { IncomingHttpHeaders } from 'node:http'`; enable `verbatimModuleSyntax`. |
| `enum` in a file run by Node | `SyntaxError [ERR_UNSUPPORTED_TYPESCRIPT_SYNTAX]: TypeScript enum is not supported in strip-only mode` | Use a `const` object with `as const` and a union type; enable `erasableSyntaxOnly`. |
| `__dirname` in an ES module | `ReferenceError: __dirname is not defined in ES module scope` | `import.meta.dirname` or `fileURLToPath(new URL('.', import.meta.url))`. |
| `require()` of a module with top-level `await` | `Error [ERR_REQUIRE_ASYNC_MODULE]` | `await import()` from an async function, or remove the top-level `await`. |
| Deep import into a package with `exports` | `Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/x.js' is not defined by "exports"` | Use the public entry point or ask the maintainer to export the subpath. |
| `default` listed before other conditions | Every consumer gets the `default` file; `import`/`require` entries ignored | Put `types` first and `default` last. |
| No `"type"` field | `Warning: Module type of file ... is not specified and it doesn't parse as CommonJS` plus slower startup | Set `"type": "module"` (or `"commonjs"`). |
| `npm install` in CI | Builds pass locally but CI picks up a newer transitive version | Use `npm ci` everywhere except when changing dependencies. |
| `.env` committed to git | Secrets in history | Commit `.env.example`; ignore `.env` and `.env.*` (the starter's `.gitignore` does). |
| Relying on `NODE_ENV` inside Node core | Nothing changes when you set it | It is only a convention read by libraries and your own code. |

### 2.11 Best Practices

- **Set `"type": "module"` in every new package**: one module system, no detection cost, no warnings.
- **Always use the `node:` prefix for built-ins**: it is unambiguous and cannot be shadowed by a package.
- **Give every published package an `exports` map**: it defines the public API and lets you refactor internals freely.
- **Use `imports` (`#internal/*`) instead of deep relative paths**: moves and renames stay local to `package.json`.
- **Commit the lockfile and use `npm ci` in CI and Docker**: builds become reproducible and integrity-checked.
- **Run `tsc --noEmit` in CI even when Node runs `.ts` directly**: type stripping never type-checks.
- **Enable `erasableSyntaxOnly` and `verbatimModuleSyntax`**: they turn runtime type-stripping errors into compile errors.
- **Prefer built-ins (`--watch`, `--env-file`, `parseArgs`, `node:test`) over dev dependencies**: fewer packages, fewer supply-chain risks, faster installs.
- **Install with `--ignore-scripts` where possible**: dependency install scripts run with your user's permissions.

### 2.12 Hands-on Exercise

**Task.** Turn `examples/packages/greeter` into a package that serves both module systems from one ESM build and has a development-only entry.

1. Run `node examples/packages/greeter/check.mjs` and `node examples/packages/greeter/check.cjs`. Explain why the CommonJS file can load an ES module.
2. Add a `"./format"` export that exposes `shout` publicly, and import it from `check.mjs`.
3. Add top-level `await` to `src/index.js` (for example `await Promise.resolve()`). Run `check.cjs` again. Which error appears, and why does `check.mjs` still work?
4. Remove the top-level `await`. Run `check.mjs` with and without `-C development` and confirm the `mode` export switches.

**Expected output.** Step 1 prints `HELLO, ESM` / `HELLO, CJS`. Step 3 makes `check.cjs` fail with `ERR_REQUIRE_ASYNC_MODULE` while `check.mjs` keeps working, because `import` can wait for an async graph and `require()` cannot. Step 4 prints `mode: production`, then `mode: development`.

**Solution.** The starting package and the asserted outputs for steps 1 and 4 are in `examples/packages/greeter/` and the `packages:` test in `test/examples.test.ts`.

[Back to top ⤴️](#-table-of-contents)

---

## 3. Stage 3: Async Programming, Events, Buffers, Files & Streams

Everything slow in Node is asynchronous, so the quality of a Node program depends on how it sequences, parallelises, cancels and recovers from asynchronous work. This stage moves from callbacks to promises and `async`/`await`, then covers error handling, cancellation with `AbortSignal`, the `EventEmitter` pattern, binary data, the file system, and streams with backpressure. It builds on the event loop from Stage 1. Afterwards you can process a file of any size in constant memory, cancel any operation on a deadline, and keep one failing request from taking the process down.

```mermaid
flowchart LR
    CB["Error-first callbacks: fs.readFile(path, cb)"] --> P["Promises: fs/promises, util.promisify"]
    P --> AA["async / await: sequential-looking code"]
    AA --> Cancel["AbortSignal: timeouts and cancellation"]
    AA --> EE["EventEmitter: many values over time"]
    EE --> ST["Streams: many chunks with backpressure"]
    ST --> AI["for await: streams as async iterables"]
```

### 3.1 From Callbacks to Promises to `async`/`await`

Node's original convention is the **error-first callback**: the last argument is a function called with `(err, result)`. It works, but composing several steps nests deeply and every step must check `err` itself.

```ts
import { readFile, writeFile } from 'node:fs';

readFile('input.json', 'utf8', (err, text) => {
  if (err) return console.error('read failed', err);
  let data: unknown;
  try {
    data = JSON.parse(text);
  } catch (parseErr) {
    return console.error('bad JSON', parseErr); // a throw here would escape the callback
  }
  writeFile('output.json', JSON.stringify(data, null, 2), (writeErr) => {
    if (writeErr) return console.error('write failed', writeErr);
    console.log('done');
  });
});
```

A **promise** is an object that will settle once, either fulfilled with a value or rejected with an error. `async` functions return promises, and `await` pauses the function (not the thread) until a promise settles.

```mermaid
stateDiagram-v2
    [*] --> Pending
    Pending --> Fulfilled: resolve(value)
    Pending --> Rejected: reject(error) or throw
    Fulfilled --> [*]: .then / await gets the value
    Rejected --> [*]: .catch / try-catch gets the error
    note right of Pending: settles exactly once, later calls are ignored
```

The same program with `node:fs/promises`:

```ts
import { readFile, writeFile } from 'node:fs/promises';

try {
  const data = JSON.parse(await readFile('input.json', 'utf8'));
  await writeFile('output.json', JSON.stringify(data, null, 2));
  console.log('done');
} catch (err) {
  console.error('copy failed', err); // read, parse and write errors all land here
}
```

The biggest performance mistake with `await` is accidental serialisation. Each `await` in a loop waits for the previous one:

```ts
const ids = [1, 2, 3, 4, 5];
const fetchUser = async (id: number) => {
  await new Promise((r) => setTimeout(r, 100)); // stands in for a 100 ms network call
  return { id };
};

console.time('sequential');
for (const id of ids) await fetchUser(id);           // about 500 ms
console.timeEnd('sequential');

console.time('parallel');
await Promise.all(ids.map((id) => fetchUser(id)));   // about 100 ms
console.timeEnd('parallel');
```

Unbounded parallelism has its own failure mode: 10,000 simultaneous requests exhaust sockets, file descriptors or the remote service's rate limit. Limit concurrency with a small worker loop:

```ts
// Runs fn over items with at most `limit` calls in flight; keeps result order.
export async function mapLimit<T, R>(
  items: readonly T[],
  limit: number,
  fn: (item: T, index: number) => Promise<R>
): Promise<R[]> {
  const results = new Array<R>(items.length);
  let next = 0;
  async function worker(): Promise<void> {
    while (next < items.length) {
      const i = next++; // safe: JavaScript runs one worker at a time between awaits
      results[i] = await fn(items[i] as T, i);
    }
  }
  await Promise.all(Array.from({ length: Math.min(limit, items.length) }, worker));
  return results;
}

const users = await mapLimit([1, 2, 3, 4, 5, 6], 2, async (id) => ({ id }));
console.log(users.length); // 6
```

| Combinator | Settles when | Result | Use for |
| :--- | :--- | :--- | :--- |
| `Promise.all(ps)` | All fulfil, or the first rejects | Array of values | Independent steps that must all succeed |
| `Promise.allSettled(ps)` | All settle | `{ status, value \| reason }[]` | Batch jobs that report partial failure |
| `Promise.race(ps)` | The first settles | That value or error | Rarely: prefer `AbortSignal.timeout()` for deadlines |
| `Promise.any(ps)` | The first fulfils, or all reject | First value, or `AggregateError` | Redundant requests to replicas |
| `Promise.withResolvers()` | Manual | `{ promise, resolve, reject }` | Adapting event-based APIs without nesting |

`Promise.all` does not cancel the other operations when one rejects; they keep running and their results are discarded. Pair it with an `AbortController` (3.3) when the losers should stop.

### 3.2 Error Handling Across Async Boundaries

An error thrown inside a callback is not caught by a `try` around the code that scheduled the callback, because that code has already returned. Promises fix this for code you `await`. Everything else needs to be handled at the right boundary.

```mermaid
flowchart TD
    Err["An error is thrown or a promise rejects"] --> Await{"Inside an awaited promise chain?"}
    Await -->|"yes"| Try["try / catch or .catch() at the boundary that can decide"]
    Await -->|"no, a plain callback"| Throw["uncaughtException"]
    Await -->|"a promise nobody awaits"| UR["unhandledRejection"]
    Throw --> Crash["Default: print stack, exit code 1"]
    UR --> Crash
    Try --> Handle["Retry, fall back, or return a 4xx/5xx response"]
```

`examples/06-async-errors.ts` shows the tools you use most:

```ts
import { readFile } from 'node:fs/promises';

async function loadSettings(path: string): Promise<unknown> {
  try {
    return JSON.parse(await readFile(path, 'utf8'));
  } catch (err) {
    // Add context and keep the original error as `cause`.
    throw new Error(`cannot load settings from ${path}`, { cause: err });
  }
}

try {
  await loadSettings('./does-not-exist.json');
} catch (err) {
  const wrapped = err as Error & { cause: NodeJS.ErrnoException };
  console.log(wrapped.message);                    // cannot load settings from ./does-not-exist.json
  console.log(`cause code: ${wrapped.cause.code}`); // cause code: ENOENT
}

const outcomes = await Promise.allSettled([Promise.resolve('ok'), Promise.reject(new Error('boom'))]);
console.log(outcomes.map((o) => o.status).join(', ')); // fulfilled, rejected
```

| Line / API | Role | What happens at runtime |
| :--- | :--- | :--- |
| `new Error(msg, { cause })` | Error chaining (ES2022) | `util.inspect` and `console.error` print the cause chain, so logs show both "what we were doing" and "what failed". |
| `err.code` | Stable machine-readable error | Node's system errors carry `code` (`ENOENT`, `EACCES`, `ECONNRESET`) and its own errors use `ERR_*` codes. Branch on `code`, never on `message`. |
| `Promise.allSettled` | Never rejects | Lets a batch finish and report every failure. |

Node's process-level behaviour:

| Event | Default since | What Node does | What you should do |
| :--- | :--- | :--- | :--- |
| `unhandledRejection` | Node 15 (`--unhandled-rejections=throw`) | Treats it as an uncaught exception: prints it and exits with code 1 | Fix the missing `await`/`.catch`. Log in a handler only for telemetry. |
| `uncaughtException` | Always | Prints the stack, exits with code 1 | Log, flush, exit. State may be corrupt, so do not keep serving. |
| `uncaughtExceptionMonitor` | 13.7 | Observes without changing behaviour | Telemetry hooks that must not swallow the crash |
| `warning` | Always | Prints to stderr | Route to your logger; `--trace-warnings` for stacks |

```ts
// A last-resort handler: report, then exit, never resume.
process.on('uncaughtException', (err, origin) => {
  console.error({ msg: 'fatal error, exiting', origin, err });
  process.exitCode = 1;
  // Give the logger a moment to flush, but do not keep accepting work.
  setTimeout(() => process.exit(1), 500).unref();
});
```

In web servers the useful boundary is the request. Express 5 catches rejections from `async` handlers and passes them to the error middleware, which is why the starter's controllers have no `try`/`catch`:

```ts
// src/controllers/sandboxController.ts
export async function runStreamSimulation(req: Request, res: Response): Promise<void> {
  const totalChunks = readIntField(req.body, 'totalChunks', 50, 1, 10_000); // may throw HttpError(400)
  const metrics = await runStreamPipelineDemo(totalChunks, 16, 32);         // may reject
  res.json({ success: true, data: metrics });
}
```

A thrown `HttpError(400, ...)` becomes `{"success":false,"error":"\"totalChunks\" must be an integer between 1 and 10000"}` with status 400. An unexpected error becomes a logged stack trace and a generic 500. Express 4 did not do this: an async handler that rejected there left the request hanging until the client timed out.

### 3.3 Cancellation and Deadlines with `AbortController`

Every operation that can wait should be cancellable, or one slow dependency ties up memory and sockets for every request queued behind it. Node uses the web-standard `AbortController`/`AbortSignal` pair: the controller aborts, the signal is passed into APIs that should stop.

```mermaid
sequenceDiagram
    participant H as Request handler
    participant S as AbortSignal.any([timeout, clientGone])
    participant W as timers/promises setTimeout (the "work")
    H->>S: AbortSignal.timeout(100)
    H->>W: sleep(200, { signal })
    Note over S: 100 ms later the timeout fires
    S-->>W: abort(TimeoutError)
    W-->>H: rejects with AbortError (cause: TimeoutError)
    H->>H: return status aborted, reason TimeoutError
```

The starter's `src/services/deadlineService.ts`:

```ts
import { setTimeout as sleep } from 'node:timers/promises';
import type { DeadlineResult } from '../types.ts';

export async function runWithDeadline(
  workMs: number,
  timeoutMs: number,
  callerSignal?: AbortSignal
): Promise<DeadlineResult> {
  const started = performance.now();
  const deadline = AbortSignal.timeout(timeoutMs);
  const signal = callerSignal ? AbortSignal.any([deadline, callerSignal]) : deadline;
  const elapsed = () => Number((performance.now() - started).toFixed(2));

  try {
    await sleep(workMs, undefined, { signal });
    return { status: 'completed', workMs, timeoutMs, elapsedMs: elapsed() };
  } catch (err) {
    if (!signal.aborted) throw err;
    const reason = signal.reason instanceof Error ? signal.reason.name : String(signal.reason);
    return { status: 'aborted', reason, workMs, timeoutMs, elapsedMs: elapsed() };
  }
}
```

and the controller that ties it to the client connection:

```ts
// excerpt from runDeadlineSimulation() in src/controllers/sandboxController.ts
const clientGone = new AbortController();
res.on('close', () => {
  if (!res.writableEnded) clientGone.abort(new Error('client disconnected'));
});
const result = await runWithDeadline(workMs, timeoutMs, clientGone.signal);
```

| Line / API | Role | What happens at runtime |
| :--- | :--- | :--- |
| `AbortSignal.timeout(ms)` | A signal that aborts itself | Uses an unref'd timer, so it never keeps the process alive. Its `reason` is a `DOMException` named `TimeoutError`. |
| `AbortSignal.any([a, b])` | Combine signals (20.3+) | Aborts as soon as any input aborts, with that input's reason. |
| `sleep(ms, value, { signal })` | Cancellable wait | Rejects with an `AbortError` whose `cause` is the signal's reason. |
| `signal.aborted` check in `catch` | Tell cancellation from failure | Only cancellation is converted into a result; real errors still propagate. |
| `res.on('close')` with `!res.writableEnded` | Detect a client that left | `close` fires on every response; `writableEnded` is false only if we never finished writing. |

```bash
curl -s -XPOST localhost:8080/api/sandbox/deadline -H 'content-type: application/json' \
  -d '{"workMs":300,"timeoutMs":50}'
# {"success":true,"data":{"status":"aborted","reason":"TimeoutError","workMs":300,"timeoutMs":50,"elapsedMs":59.05}}
```

The same signal works across the platform: `fetch(url, { signal })`, `readFile(path, { signal })`, `events.once(emitter, 'x', { signal })`, `stream.pipeline(a, b, { signal })`, `child_process.spawn(cmd, { signal })`. Your own long-running functions should accept a signal too and check it between steps:

```ts
async function processBatch(items: string[], signal: AbortSignal): Promise<number> {
  let done = 0;
  for (const item of items) {
    signal.throwIfAborted(); // throws signal.reason if aborted
    await handle(item);
    done++;
  }
  return done;
}

async function handle(item: string): Promise<void> {
  await new Promise((r) => setTimeout(r, 1, item));
}

console.log(await processBatch(['a', 'b'], AbortSignal.timeout(1000))); // 2
```

### 3.4 Events and `EventEmitter`

An `EventEmitter` delivers **many values over time** to any number of listeners: a server emits `'connection'`, a stream emits `'data'`, a child process emits `'exit'`. Most of Node's core objects extend it.

```mermaid
sequenceDiagram
    participant App as Application code
    participant E as EventEmitter
    participant L1 as Listener 1 (mailer)
    participant L2 as Listener 2 (audit log)
    App->>E: on('orderPlaced', mailer)
    App->>E: on('orderPlaced', audit)
    App->>E: emit('orderPlaced', order)
    E->>L1: mailer(order)
    E->>L2: audit(order)
    E-->>App: emit() returns true, after both listeners ran
    Note over E: Listeners run synchronously, in registration order
```

```ts
import { EventEmitter, once } from 'node:events';

interface OrderEvents {
  placed: [order: { id: number; total: number }];
  failed: [error: Error];
}

// Typed emitter: EventEmitter<T> generics are supported by @types/node.
class Orders extends EventEmitter<OrderEvents> {
  constructor() {
    super({ captureRejections: true }); // async listener rejections become 'error' events
  }

  async place(id: number, total: number): Promise<void> {
    if (total <= 0) {
      this.emit('failed', new Error(`order ${id}: total must be positive`));
      return;
    }
    await new Promise((r) => setTimeout(r, 10)); // simulated payment call
    this.emit('placed', { id, total });
  }
}

const orders = new Orders();
orders.on('placed', (o) => console.log(`placed #${o.id} for ${o.total}`));
orders.once('placed', () => console.log('first order of the day'));
orders.on('failed', (err) => console.error(err.message));
orders.on('error', (err) => console.error('listener crashed', err)); // always have one

void orders.place(1, 25);
const [order] = await once(orders, 'placed'); // promise for the next event
console.log(`awaited #${order.id}`);
await orders.place(2, -1);
```

| API | Behaviour | Detail |
| :--- | :--- | :--- |
| `emit(name, ...args)` | Calls listeners synchronously | A slow listener delays the emitter's caller. Do heavy work asynchronously inside the listener. |
| `on` / `once` / `off` | Add, add for one call, remove | `once` removes itself before calling the listener. |
| `prependListener` | Add to the front | For instrumentation that must run first. |
| `'error'` event | Special | Emitting `'error'` with no listener **throws** it, which usually crashes the process. |
| `captureRejections: true` | Async listeners | A rejected promise returned by a listener is routed to `'error'` instead of becoming an unhandled rejection. |
| `events.once(emitter, name, { signal })` | Promise of the next event | Rejects if `'error'` fires first or the signal aborts. |
| `events.on(emitter, name, { signal })` | Async iterator over events | `for await (const [value] of on(e, 'data', { signal }))`; stop it with the signal. |
| `setMaxListeners(n)` | Leak warning threshold (default 10) | More than `n` listeners for one event prints `MaxListenersExceededWarning: Possible EventEmitter memory leak detected`. Usually it is a leak; fix the code rather than raising the limit. |

Node also implements the web `EventTarget` (used by `AbortSignal`, `MessagePort`, `BroadcastChannel`). Prefer `EventEmitter` in Node-only code and `EventTarget` in code that must run in browsers too.

### 3.5 Buffers and Binary Data

Files and sockets deliver bytes, not strings. `Buffer` is Node's byte array: a subclass of `Uint8Array` with encoding helpers. Its memory lives outside the V8 heap, which is why large buffers show up in `external`/`arrayBuffers` rather than `heapUsed`.

```mermaid
flowchart LR
    AB["ArrayBuffer: raw memory, no accessors"] --> U8["Uint8Array: view with numeric indexing"]
    U8 --> Buf["Buffer: Uint8Array + encodings, read/write helpers"]
    Buf -->|"toString('utf8' / 'hex' / 'base64' / 'base64url')"| Str["string"]
    Str -->|"Buffer.from(str, enc)"| Buf
    Buf -->|"subarray(): same memory"| View["Buffer view"]
    Buf -->|"Buffer.from(buf): new memory"| Copy["Buffer copy"]
```

`examples/05-buffers.ts`:

```ts
import { StringDecoder } from 'node:string_decoder';

const text = 'caf\u00e9 \u{1F680}'; // "café" and a rocket emoji
const bytes = Buffer.from(text, 'utf8');
console.log(`string length ${text.length}, byte length ${bytes.length}`); // 7, 10
console.log(`hex ${bytes.toString('hex')}`);                              // 636166c3a920f09f9a80
console.log(`base64 ${bytes.toString('base64')}`);                        // Y2Fmw6kg8J+agA==

const view = bytes.subarray(0, 1);
view[0] = 0x43; // 'C': the original buffer changes too

const copy = Buffer.from(bytes.subarray(0, 1));
copy[0] = 0x58; // 'X': the original buffer does not change

const head = bytes.subarray(0, 4); // "Caf" plus the first byte of the two-byte "é"
const tail = bytes.subarray(4);
console.log((head.toString() + tail.toString()).includes('\uFFFD')); // true: corrupted

const decoder = new StringDecoder('utf8');
console.log(decoder.write(head) + decoder.write(tail) + decoder.end()); // "Café 🚀", intact
```

| Line / API | Role | What happens at runtime |
| :--- | :--- | :--- |
| `text.length` vs `bytes.length` | Characters vs bytes | JavaScript strings count UTF-16 code units (the emoji is 2); UTF-8 needs 1-4 bytes per character. Use `Buffer.byteLength(str)` for `Content-Length`. |
| `subarray(start, end)` | A view | No copy; writes are visible in both. Replaces `buf.slice()`, which is deprecated for buffers because it behaves differently from `Uint8Array.prototype.slice`. |
| `Buffer.from(buffer)` | Copy | Allocates new memory. Use when you keep a small piece of a large buffer, otherwise the view keeps the whole large allocation alive. |
| `StringDecoder` | Safe decoding across chunk boundaries | Holds back an incomplete multi-byte sequence until the next chunk. Streams do this for you when you call `setEncoding('utf8')`. |

| Allocation | Memory | Speed | Use when |
| :--- | :--- | :--- | :--- |
| `Buffer.alloc(n)` | Zero-filled | Slower for large `n` | Default choice |
| `Buffer.allocUnsafe(n)` | Uninitialised; may come from a shared 8 KiB pool for small sizes | Fast | You overwrite every byte immediately |
| `Buffer.from(string \| array \| buffer)` | Copy of the input | | Converting data |
| `new Buffer(n)` | Deprecated (DEP0005) | | Never |

Reading and writing binary formats uses the typed helpers:

```ts
// A 7-byte message header: version (u8), type (u16 BE), length (u32 BE)
const header = Buffer.alloc(7);
header.writeUInt8(1, 0);
header.writeUInt16BE(0x0102, 1);
header.writeUInt32BE(65_536, 3);
console.log(header.toString('hex'));                                                 // 01010200010000
console.log(header.readUInt8(0), header.readUInt16BE(1), header.readUInt32BE(3));   // 1 258 65536

// TextEncoder/TextDecoder are the web equivalents and work on any Uint8Array.
console.log(new TextDecoder().decode(new TextEncoder().encode('ok'))); // ok
```

### 3.6 The File System: `node:fs` and `node:fs/promises`

`node:fs` offers the same operations in three styles:

| Style | Example | Blocks the event loop | Use for |
| :--- | :--- | :--- | :--- |
| Synchronous | `readFileSync(path)` | **Yes** | Startup code and CLIs only |
| Callback | `readFile(path, cb)` | No | Legacy code |
| Promise | `await readFile(path)` from `node:fs/promises` | No | Everything new |
| Stream | `createReadStream(path)` | No | Large files, piping to sockets |

```mermaid
flowchart TD
    Q["Need to read a file"] --> Size{"Size known and small (< a few MB)?"}
    Size -->|"yes"| Whole["await readFile(path, 'utf8')"]
    Size -->|"no or unbounded"| Shape{"Line-oriented?"}
    Shape -->|"yes"| Lines["readline over createReadStream, for await"]
    Shape -->|"no"| Stream["pipeline(createReadStream(path), transform, destination)"]
    Q --> Random{"Need specific byte ranges?"}
    Random -->|"yes"| FH["open() a FileHandle, read(buffer, offset, length, position)"]
```

Everyday operations, all with promises:

```ts
import { mkdir, readdir, readFile, rename, rm, stat, writeFile } from 'node:fs/promises';
import { join } from 'node:path';
import { tmpdir } from 'node:os';

const dir = join(tmpdir(), `fs-demo-${process.pid}`);
await mkdir(join(dir, 'logs', '2026'), { recursive: true });        // like mkdir -p

await writeFile(join(dir, 'logs', '2026', 'app.log'), 'line 1\n');
await writeFile(join(dir, 'logs', '2026', 'app.log'), 'line 2\n', { flag: 'a' }); // append

const entries = await readdir(dir, { recursive: true, withFileTypes: true });
for (const e of entries) console.log(`${e.isDirectory() ? 'dir ' : 'file'} ${e.name}`);

const info = await stat(join(dir, 'logs', '2026', 'app.log'));
console.log(`${info.size} bytes, modified ${info.mtime.toISOString()}`);

// Atomic replace: write a temp file, then rename over the target.
// rename() is atomic on the same file system, so readers see the old or the new file, never half.
const target = join(dir, 'config.json');
await writeFile(`${target}.tmp`, JSON.stringify({ port: 8080 }));
await rename(`${target}.tmp`, target);
console.log(JSON.parse(await readFile(target, 'utf8')).port);

await rm(dir, { recursive: true, force: true });                     // like rm -rf
```

Random access with a `FileHandle`, without loading the file:

```ts
import { open } from 'node:fs/promises';

const handle = await open(import.meta.filename, 'r');
try {
  const { size } = await handle.stat();
  const buf = Buffer.alloc(16);
  const { bytesRead } = await handle.read(buf, 0, buf.length, 0); // 16 bytes from offset 0
  console.log(size, bytesRead, JSON.stringify(buf.toString('utf8')));
} finally {
  await handle.close(); // or `await using handle = await open(...)` on Node 24
}
```

Rules that avoid the classic file system bugs:

- **Do not check, then act.** `if (await exists(p)) await readFile(p)` races with other processes. Call `readFile` and handle `ENOENT`.
- **Close what you open.** Leaked `FileHandle`s eventually fail with `EMFILE: too many open files`. Node closes a garbage-collected handle and warns, but that may be too late.
- **Never pass user input straight into a path.** See 3.7.
- **`fs.watch` is platform-dependent.** Events are coalesced differently on Linux, macOS and Windows and some network file systems report nothing. Use it for developer tooling, not for correctness.

### 3.7 Paths, URLs and the Operating System

`node:path` manipulates path strings. It never touches the disk.

```ts
// Outputs shown are for Linux and macOS; Windows uses backslashes and drive letters.
import path from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';

console.log(path.join('/srv/app', 'public', '..', 'config', 'app.json')); // /srv/app/config/app.json
console.log(path.resolve('src', 'server.ts'));      // absolute: cwd + src/server.ts
console.log(path.relative('/srv/app/src/routes', '/srv/app/src/lib/db.ts')); // ../lib/db.ts
console.log(path.parse('/srv/app/server.ts'));
// { root: '/', dir: '/srv/app', base: 'server.ts', ext: '.ts', name: 'server' }
console.log(path.format({ dir: '/srv/app', name: 'server', ext: '.js' })); // /srv/app/server.js

// path uses the host OS rules; force one style with path.posix / path.win32.
console.log(path.win32.join('C:\\app', 'logs'), path.posix.join('/app', 'logs'));

// ESM gives you URLs; convert at the boundary.
const here = fileURLToPath(import.meta.url);           // same as import.meta.filename
const publicDir = fileURLToPath(new URL('../public/', import.meta.url));
console.log(here.endsWith('.ts'), pathToFileURL(publicDir).href.startsWith('file://'));
```

Serving files from a directory needs a containment check, because `path.join` happily resolves `..`:

```ts
import path from 'node:path';

// Returns an absolute path inside `root`, or null for anything that would escape it.
export function safeJoin(root: string, requestPath: string): string | null {
  let decoded: string;
  try {
    decoded = decodeURIComponent(requestPath);
  } catch {
    return null; // malformed percent-encoding such as "%E0%A4%A"
  }
  if (decoded.includes('\0')) return null;
  const full = path.resolve(root, '.' + path.sep + decoded);
  return full === root || full.startsWith(root + path.sep) ? full : null;
}

const root = path.resolve('public');
console.log(safeJoin(root, 'css/site.css'));            // <cwd>/public/css/site.css
console.log(safeJoin(root, '../../etc/passwd'));        // null
console.log(safeJoin(root, '..%2f..%2fetc%2fpasswd'));  // null: decoding happens before the check
```

`node:os` describes the machine:

```ts
import os from 'node:os';

console.log(os.platform(), os.arch(), os.release());
console.log(`parallelism ${os.availableParallelism()}, logical CPUs ${os.cpus().length}`);
console.log(`memory ${(os.freemem() / 2 ** 30).toFixed(1)} / ${(os.totalmem() / 2 ** 30).toFixed(1)} GiB`);
console.log(`load average ${os.loadavg().map((n) => n.toFixed(2)).join(' ')}`); // zeros on Windows
console.log(os.hostname(), os.homedir(), os.tmpdir(), JSON.stringify(os.EOL));
```

Use `os.availableParallelism()` rather than `os.cpus().length` to size worker pools: it respects the process's CPU affinity. In a container with a CPU quota, both can still report the host's cores, so make the worker count configurable.

### 3.8 Streams and Backpressure

A stream processes data **piece by piece**. Memory stays proportional to the chunk size, not to the total size, so a 10 GB log can be gzipped by a process with 50 MB of RAM.

```mermaid
classDiagram
    class Readable {
        +read(size)
        +push(chunk)
        +pipe(destination)
        +on(event, listener)
        +asyncIterator()
    }
    class Writable {
        +write(chunk) boolean
        +end()
        +on(event, listener)
    }
    class Duplex {
        +readableSide
        +writableSide
    }
    class Transform {
        +_transform(chunk, enc, callback)
        +_flush(callback)
    }
    Readable <|-- Duplex
    Writable <|-- Duplex
    Duplex <|-- Transform
```

| Type | Examples | You implement |
| :--- | :--- | :--- |
| Readable | `fs.createReadStream`, `http.IncomingMessage` (request on the server), `process.stdin` | `read()` |
| Writable | `fs.createWriteStream`, `http.ServerResponse`, `process.stdout` | `write()` (and optionally `final()`) |
| Duplex | `net.Socket`, TLS sockets | both |
| Transform | `zlib.createGzip()`, `crypto.createHash()`, your own parsers | `transform()` (and optionally `flush()`) |

**Backpressure** is how a fast producer learns that a slow consumer is full. Every writable has a buffer of `highWaterMark` bytes (64 KiB by default since Node 22; 16 items in object mode). `write()` returns `false` once the buffer reaches that level, and the writable emits `'drain'` when it has room again.

```mermaid
sequenceDiagram
    participant P as Producer
    participant W as Writable (highWaterMark 3, object mode)
    P->>W: write(1) returns true
    P->>W: write(2) returns true
    P->>W: write(3) returns false
    Note over P: stop producing
    W->>W: consumer processes 1, 2, 3
    W-->>P: 'drain'
    P->>W: write(4) ... continues
```

`examples/04-backpressure.ts` is that diagram as code:

```ts
import { Writable } from 'node:stream';

const sink = new Writable({
  objectMode: true,
  highWaterMark: 3,
  write(_item, _encoding, done) {
    setImmediate(done); // a consumer slower than the producer
  }
});

let next = 0;
function produce(): void {
  while (next < 10) {
    next++;
    if (!sink.write(next)) {
      console.log(`write #${next} returned false (buffered: ${sink.writableLength})`);
      sink.once('drain', produce);
      return;
    }
  }
  sink.end(() => console.log('all 10 items flushed'));
}
produce();
```

```text
write #3 returned false (buffered: 3)
write #6 returned false (buffered: 3)
write #9 returned false (buffered: 3)
all 10 items flushed
```

Ignoring the return value of `write()` does not lose data; the writable keeps buffering. That is the danger: memory grows without bound until the process is killed. In practice you rarely write the loop above yourself. `pipeline()` from `node:stream/promises` connects streams, applies backpressure, forwards the first error, and destroys every stream when one fails:

```ts
import { createReadStream, createWriteStream } from 'node:fs';
import { writeFile, rm } from 'node:fs/promises';
import { pipeline } from 'node:stream/promises';
import { createGzip } from 'node:zlib';

await writeFile('access.log', 'GET /health 200\nGET /api 500\nGET /health 200\n'.repeat(10_000));

await pipeline(
  createReadStream('access.log'),
  // An async generator is a valid transform: it receives the source as an async iterable.
  async function* onlyErrors(source: AsyncIterable<Buffer>) {
    let carry = '';
    for await (const chunk of source) {
      const lines = (carry + chunk.toString('utf8')).split('\n');
      carry = lines.pop() ?? ''; // keep the partial last line for the next chunk
      for (const line of lines) if (/ 5\d\d$/.test(line)) yield line + '\n';
    }
    if (/ 5\d\d$/.test(carry)) yield carry + '\n';
  },
  createGzip(),
  createWriteStream('errors.log.gz')
);
console.log('wrote errors.log.gz');
await rm('access.log');
await rm('errors.log.gz');
```

| Line / API | Role | What happens at runtime |
| :--- | :--- | :--- |
| `pipeline(a, b, c, d)` | Connect and supervise | Pulls from `a` only as fast as `d` accepts; returns a promise that rejects with the first error. |
| `async function*` stage | Transform without a class | `for await` reads chunks; each `yield` is written downstream and waits for room (backpressure). |
| `carry` | Line splitting across chunks | A 64 KiB chunk boundary can cut a line in half; keep the remainder. |
| `createGzip()` | Built-in Transform | Compression runs on the thread pool. |
| `createWriteStream` | Destination | `pipeline` waits for `'finish'` and closes the file. |

The starter's `src/services/streamService.ts` builds the same Readable -> Transform -> Writable chain with a deliberately slow sink and counts how many times the sink had to `'drain'`. `test/app.test.ts` asserts that the count is greater than zero, which proves backpressure engaged.

For line-oriented input, `node:readline` plus `for await` is the shortest correct code:

```ts
import { createReadStream } from 'node:fs';
import { createInterface } from 'node:readline';

async function countMatching(path: string, pattern: RegExp): Promise<number> {
  const lines = createInterface({ input: createReadStream(path), crlfDelay: Infinity });
  let count = 0;
  for await (const line of lines) if (pattern.test(line)) count++;
  return count;
}

console.log(await countMatching(import.meta.filename, /^import /)); // 2: the two import lines
```

| Approach | Error handling | Backpressure | Cleanup on failure | Verdict |
| :--- | :--- | :--- | :--- | :--- |
| `a.pipe(b).pipe(c)` | Errors are **not** forwarded; each stream needs its own handler | Yes | No: a failing `c` leaves `a` open (file descriptor leak) | Avoid in new code |
| `pipeline(a, b, c)` (`stream/promises`) | First error rejects the promise | Yes | Destroys all streams | Default choice |
| `for await (const chunk of readable)` | `try`/`catch` | Yes, reading pauses while you `await` | Breaking out of the loop destroys the stream | Consuming a stream in your own code |
| `Readable.from(iterable)` | Errors from the iterator destroy the stream | Yes | Yes | Turning arrays or generators into streams |
| `Readable.toWeb()` / `Readable.fromWeb()` | Web stream semantics | Yes | Yes | Interop with `fetch` bodies and web APIs |

### 3.9 Compression with `node:zlib`

`node:zlib` implements gzip/deflate, Brotli and, from 22.15 and 23.8 (experimental), Zstandard. Use the streaming classes inside `pipeline()` for anything large; use the promisified one-shot functions for small payloads.

```ts
import { promisify } from 'node:util';
import zlib from 'node:zlib';

const gzip = promisify(zlib.gzip);
const gunzip = promisify(zlib.gunzip);
const brotli = promisify(zlib.brotliCompress);
const unbrotli = promisify(zlib.brotliDecompress);

const payload = Buffer.from(JSON.stringify(Array.from({ length: 2000 }, (_, i) => ({ id: i, ok: true }))));

const gz = await gzip(payload, { level: 6 });
const br = await brotli(payload, {
  params: {
    [zlib.constants.BROTLI_PARAM_QUALITY]: 5, // 0-11; 11 is slow, fine for build-time static assets
    [zlib.constants.BROTLI_PARAM_SIZE_HINT]: payload.length
  }
});

console.log(`original ${payload.length} B, gzip ${gz.length} B, brotli ${br.length} B`);
console.log((await gunzip(gz)).equals(payload), (await unbrotli(br)).equals(payload)); // true true
console.log(`crc32 ${zlib.crc32(payload).toString(16)}`); // checksum helper (22.2+)
```

| Codec | Ratio | Speed | Use for |
| :--- | :--- | :--- | :--- |
| gzip / deflate | Good | Fast | HTTP responses (universal client support), log archives |
| Brotli | Better on text | Slow at high quality | Pre-compressed static assets; dynamic responses at quality 4-5 |
| Zstandard | Similar to Brotli | Very fast | Internal services and storage where both ends are under your control |

Compressing a dynamic HTTP response costs CPU on every request. Above a few thousand requests per second, let a reverse proxy or CDN compress, or pre-compress static files at build time.

### 3.10 Pitfalls

| Pitfall | Symptom | Fix |
| --- | --- | --- |
| Forgetting `await` | `unhandledRejection` crash with exit code 1, or a `Promise { <pending> }` in a log line | Enable `@typescript-eslint/no-floating-promises`; `await` or explicitly `void` the promise. |
| `await` inside a `for` loop over independent work | Latency equals the sum of all calls | `Promise.all` or `mapLimit` with a concurrency cap. |
| Throwing inside a plain callback | `uncaughtException`; the surrounding `try` never sees it | Convert to promises (`fs/promises`, `util.promisify`), or pass the error to the callback. |
| `'error'` emitted with no listener | Process crashes with `Error [ERR_UNHANDLED_ERROR]: Unhandled error.` or the raw error | Always attach an `'error'` listener to emitters and streams you create. |
| Adding a listener per request to a long-lived emitter | `MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 ... listeners added` | Remove listeners (`off`, `once`, `{ signal }`), or use one shared listener. |
| `.pipe()` chains | A destination error leaves the source open; `EMFILE` after a while | `pipeline()` from `node:stream/promises`. |
| Ignoring `write()`'s return value | RSS climbs until `OOMKilled` under load | Wait for `'drain'`, or let `pipeline()` handle it. |
| `chunk.toString()` on each chunk of multi-byte text | Occasional `U+FFFD` characters in output | `stream.setEncoding('utf8')` or a `StringDecoder`. |
| `readFile` on user-supplied or unbounded files | `RangeError [ERR_FS_FILE_TOO_LARGE]` or memory spikes | Stream it. |
| Path built from user input with `path.join` | Path traversal: `/../../etc/passwd` served | Resolve and check containment (`safeJoin` in 3.7). |
| `Promise.race` used as a timeout | The slow operation keeps running and holding resources | `AbortSignal.timeout()` passed into the operation. |
| Keeping a small `subarray` of a large buffer | Memory stays high after the big buffer "should" be freed | `Buffer.from(view)` to copy the part you keep. |

### 3.11 Best Practices

- **Use promise APIs (`node:fs/promises`, `node:timers/promises`, `node:stream/promises`)**: errors flow to one `try`/`catch` instead of every callback.
- **Run independent work concurrently, with a limit**: `Promise.all` for a handful, `mapLimit` for hundreds.
- **Accept an `AbortSignal` in every function that waits**: callers can then apply deadlines and cancel abandoned work.
- **Wrap errors with `{ cause }` and branch on `err.code`**: logs keep the full story and code stays stable across message changes.
- **Treat `uncaughtException` as fatal**: log and exit; let the orchestrator restart a clean process.
- **Stream anything whose size you do not control**: memory then depends on chunk size, not input size.
- **Use `pipeline()` or `for await`, not `.pipe()`**: you get error propagation and cleanup for free.
- **Attach `'error'` listeners to every emitter and stream you own**: an unhandled `'error'` event crashes the process.
- **Copy (`Buffer.from`) slices you keep long-term**: a view pins the entire parent buffer in memory.
- **Write files atomically with temp file plus `rename()`**: readers never see a half-written file.

### 3.12 Hands-on Exercise

**Task.** Add a `POST /api/sandbox/line-count` endpoint to the starter that counts lines containing a search term in a generated file, without ever holding the whole file in memory, and that stops when the client disconnects or a deadline passes.

1. In a new `src/services/lineCountService.ts`, generate a temporary file of N lines (`N` from the body, 1 to 1,000,000) with `pipeline(Readable.from(generator()), createWriteStream(tmp))`.
2. Count matching lines with `readline` and `for await`, checking `signal.throwIfAborted()` every 10,000 lines.
3. Combine `AbortSignal.timeout(timeoutMs)` with a client-disconnect signal using `AbortSignal.any()`, exactly like `runDeadlineSimulation`.
4. Delete the temporary file in a `finally` block.
5. Return `{ lines, matches, elapsedMs, status }` and add two tests to `test/app.test.ts`: one that completes, and one with `timeoutMs: 1` and `lines: 1000000` that returns `status: 'aborted'`.

**Expected output.**

```bash
curl -s -XPOST localhost:8080/api/sandbox/line-count -H 'content-type: application/json' \
  -d '{"lines":200000,"term":"ERROR","timeoutMs":5000}'
# {"success":true,"data":{"lines":200000,"matches":20000,"elapsedMs":...,"status":"completed"}}
```

(The generator writes `ERROR` on every tenth line.) `process.memoryUsage().rss` should stay flat as you raise `lines` from 10,000 to 1,000,000.

**Solution.** The building blocks are already in the repository: `src/services/deadlineService.ts` (signals and deadlines), `src/services/streamService.ts` (pipeline and backpressure), `src/controllers/sandboxController.ts` (`runDeadlineSimulation`, client disconnect) and `test/app.test.ts` (how to test an endpoint with `fetch`).

[Back to top ⤴️](#-table-of-contents)

---

## 4. Stage 4: Networking, HTTP & Web Frameworks

Stage 3 gave you streams and cancellation; this stage puts them on the network. It covers the built-in HTTP server, raw TCP and UDP sockets, and how the main web frameworks build on them. Afterwards you can write an HTTP service without a framework and choose one with clear trade-offs in mind.

### 4.1 HTTP Servers with `node:http`

Node.js includes built-in HTTP and HTTPS client and server modules that operate directly on streaming TCP sockets.

```javascript
import http from 'node:http';

const server = http.createServer((req, res) => {
  const { method, url, headers } = req;

  // Route matching
  if (method === 'GET' && url === '/health') {
    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify({ status: 'UP', timestamp: new Date().toISOString() }));
    return;
  }

  if (method === 'POST' && url === '/api/data') {
    let body = '';
    
    // Read request body as chunks arrive
    req.on('data', (chunk) => {
      body += chunk.toString();
      // Guard against payload flooding attack (>1MB)
      if (body.length > 1e6) {
        res.writeHead(413, { 'Content-Type': 'text/plain' });
        res.end('Payload Too Large');
        req.destroy();
      }
    });

    req.on('end', () => {
      try {
        const parsed = JSON.parse(body);
        res.writeHead(201, { 'Content-Type': 'application/json' });
        res.end(JSON.stringify({ received: parsed, status: 'CREATED' }));
      } catch {
        res.writeHead(400, { 'Content-Type': 'text/plain' });
        res.end('Invalid JSON payload');
      }
    });
    return;
  }

  res.writeHead(404, { 'Content-Type': 'text/plain' });
  res.end('Not Found');
});

server.listen(3000, () => {
  console.log('HTTP Server listening on http://localhost:3000');
});
```

### 4.2 Raw Networking: TCP and UDP (`node:net`, `node:dgram`)

The `node:net` module provides an asynchronous network API for creating stream-based TCP or IPC servers and clients. The `node:dgram` module provides an implementation of UDP datagram sockets.

```mermaid
flowchart TD
    subgraph TCP["TCP Socket (node:net) - Connection-Oriented, Guaranteed Delivery"]
        S1["net.createServer()"] <-->|"3-Way Handshake & Persistent Stream"| C1["net.createConnection()"]
    end
    subgraph UDP["UDP Socket (node:dgram) - Connectionless, Low-Latency Datagrams"]
        S2["dgram.createSocket('udp4')"] -.->|"Unreliable Packet Stream"| C2["socket.send(buffer, port, host)"]
    end
```

#### 1. Production TCP Server & Client with Backpressure

```javascript
import net from 'node:net';

// High-performance TCP Echo Server
const tcpServer = net.createServer((socket) => {
  console.log(`TCP Client connected from ${socket.remoteAddress}:${socket.remotePort}`);

  socket.on('data', (data) => {
    console.log(`Received ${data.length} bytes: ${data.toString('utf8')}`);
    // Echo data back with backpressure protection
    const canContinue = socket.write(`ECHO: ${data}`);
    if (!canContinue) {
      console.warn('Socket buffer full. Pausing incoming reads...');
      socket.pause();
    }
  });

  socket.on('drain', () => {
    console.log('Socket buffer drained. Resuming reads.');
    socket.resume();
  });

  socket.on('end', () => {
    console.log('Client disconnected gracefully');
  });

  socket.on('error', (err) => {
    console.error('Socket error:', err.message);
  });
});

tcpServer.listen(9000, () => {
  console.log('TCP Server listening on port 9000');
});
```

#### 2. High-Throughput UDP Datagram Socket

```javascript
import dgram from 'node:dgram';

// UDP Receiver
const udpServer = dgram.createSocket('udp4');

udpServer.on('message', (msg, rinfo) => {
  console.log(`UDP packet from ${rinfo.address}:${rinfo.port}: ${msg.toString()}`);
});

udpServer.on('listening', () => {
  const addr = udpServer.address();
  console.log(`UDP Server listening on ${addr.address}:${addr.port}`);
});

udpServer.bind(41234);

// UDP Sender
const client = dgram.createSocket('udp4');
const message = Buffer.from('Heartbeat metric payload');
client.send(message, 41234, 'localhost', (err) => {
  if (err) console.error(err);
  client.close();
});
```

### 4.3 Framework Comparison: Express 5 vs Fastify vs NestJS

The framework decides routing speed, validation, serialisation and how code is organised. The request rates below are rough single-process hello-world figures that show relative overhead; measure your own workload before choosing on throughput alone.

```mermaid
flowchart TD
    Req["Incoming HTTP Request"] --> Router
    Router --> Fastify["Fastify (Find-My-Way Radix Tree + Ajv Schema Compilation)<br/>~75,000 req/sec"]
    Router --> Express["Express 5 (RegExp linear matching + async error handling)<br/>~15,000 req/sec"]
    Router --> Nest["NestJS (Angular-style Dependency Injection & TS Decorators)<br/>~25,000 req/sec (Fastify adapter)"]
```

| Feature | Express 5 | Fastify | NestJS |
| :--- | :--- | :--- | :--- |
| **Throughput (req/sec)** | Moderate (~15k) | Extremely High (~75k) | Depends on engine (Express or Fastify) |
| **JSON Serialization** | Standard `JSON.stringify` | Fast JSON Stringify (Schema JIT) | Configurable |
| **Validation** | Third-party (Zod, Joi) | Built-in Ajv JSON Schema | Class-validator decorators |
| **Architecture** | Unopinionated middleware | Plugin architecture with encapsulation | Enterprise Modular DI (Controllers/Services) |

#### Fastify High-Throughput Implementation with Schema Validation

```javascript
import Fastify from 'fastify';

const fastify = Fastify({ logger: true });

// Schema compiles to high-performance JIT serialization code
const userSchema = {
  body: {
    type: 'object',
    required: ['email', 'password'],
    properties: {
      email: { type: 'string', format: 'email' },
      password: { type: 'string', minLength: 8 }
    }
  },
  response: {
    201: {
      type: 'object',
      properties: {
        id: { type: 'string' },
        email: { type: 'string' }
      }
    }
  }
};

fastify.post('/api/users', { schema: userSchema }, async (request, reply) => {
  const { email } = request.body;
  reply.code(201).send({ id: 'usr_101', email });
});

await fastify.listen({ port: 3000, host: '0.0.0.0' });
```

[Back to top ⤴️](#-table-of-contents)

---

## 5. Stage 5: Concurrency, Request Context & Testing

The event loop gives you concurrency for I/O; this stage adds parallelism for CPU work, request-scoped context that follows `await`, and the built-in test runner. Afterwards you can keep the main thread responsive under CPU load, trace one request through every log line, and test it all without a framework.

### 5.1 Worker Threads (`node:worker_threads`)

The `node:worker_threads` module enables the use of threads that execute JavaScript in parallel. Unlike child processes, worker threads share memory space efficiently.

```mermaid
flowchart TD
    MainThread["Main Thread (Event Loop, HTTP Server)"] -->|"workerData / postMessage"| W1["Worker Thread 1 (Isolate V8, CPU Hashing)"]
    MainThread -->|"workerData / postMessage"| W2["Worker Thread 2 (Isolate V8, Image Resize)"]
    W1 -->|"postMessage(result)"| MainThread
    W2 -->|"postMessage(result)"| MainThread
    MainThread <-->|"Zero-Copy Shared Memory"| SharedMem[("SharedArrayBuffer & Atomics")]
    W1 <--> SharedMem
```

#### Code Example: CPU Heavy Fibonacci in Worker Thread

```javascript
// worker-demo.mjs
import { Worker, isMainThread, parentPort, workerData } from 'node:worker_threads';
import { fileURLToPath } from 'node:url';

if (isMainThread) {
  console.log('Main Thread: Spawning worker for heavy computation...');

  const worker = new Worker(fileURLToPath(import.meta.url), {
    workerData: { num: 42 }
  });

  worker.on('message', (result) => {
    console.log(`Main Thread received result from worker: ${result}`);
  });

  worker.on('error', console.error);
  worker.on('exit', (code) => {
    if (code !== 0) console.error(`Worker stopped with exit code ${code}`);
  });

  console.log('Main Thread remains non-blocking and responsive!');
} else {
  // Heavy CPU blocking computation inside isolated worker thread
  function fibonacci(n) {
    if (n <= 1) return n;
    return fibonacci(n - 1) + fibonacci(n - 2);
  }

  const result = fibonacci(workerData.num);
  parentPort.postMessage(result);
}
```

### 5.2 Multi-Process Clustering (`node:cluster`)

A single instance of Node.js runs in a single thread. To take advantage of multi-core systems, the `node:cluster` module allows you to easily create child processes that all share server ports.

```mermaid
flowchart TD
    Primary["Primary Process (Master Cluster Manager)"] -->|"fork() & Port 8000 Binding"| W1["Worker 1 (CPU Core 0)"]
    Primary -->|"fork() & Port 8000 Binding"| W2["Worker 2 (CPU Core 1)"]
    Primary -->|"fork() & Port 8000 Binding"| W3["Worker 3 (CPU Core 2)"]
    Primary -->|"fork() & Port 8000 Binding"| W4["Worker 4 (CPU Core 3)"]
    OSNet["Incoming Client TCP Connections"] -->|"Round-Robin Distribution"| Primary
```

#### Production Cluster with Zero-Downtime Rolling Restarts

```javascript
import cluster from 'node:cluster';
import http from 'node:http';
import os from 'node:os';

if (cluster.isPrimary) {
  const numCPUs = os.cpus().length;
  console.log(`Primary master ${process.pid} is running. Forking ${numCPUs} workers...`);

  for (let i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  cluster.on('exit', (worker, code, signal) => {
    console.warn(`Worker ${worker.process.pid} died (code: ${code}, signal: ${signal}). Spawning replacement...`);
    cluster.fork(); // Automatic self-healing resurrection
  });
} else {
  // Workers share the TCP connection on port 8000
  http.createServer((req, res) => {
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    res.end(`Handled by worker process PID: ${process.pid}\n`);
  }).listen(8000);

  console.log(`Worker ${process.pid} started`);
}
```

### 5.3 Request Context with `AsyncLocalStorage`

In distributed enterprise architectures, tracking a `correlationId` or `traceId` through deeply nested async call chains normally requires passing arguments to every function ("parameter drilling"). `AsyncLocalStorage` provides thread-local storage semantics across asynchronous boundaries:

```mermaid
flowchart LR
    Req["HTTP Request (X-Request-Id: 'req_abc123')"] --> ALSRun["asyncLocalStorage.run({ requestId }, handler)"]
    ALSRun --> AsyncDB["await db.findUser()"]
    AsyncDB --> AsyncLog["logger.info() (Retrieves requestId automatically)"]
    AsyncLog --> AsyncThirdParty["await fetchPaymentGateway()"]
    AsyncThirdParty --> Resp["Response (Consistent Context Maintained)"]
```

```javascript
import { AsyncLocalStorage } from 'node:async_hooks';
import http from 'node:http';
import crypto from 'node:crypto';

const asyncLocalStorage = new AsyncLocalStorage();

function log(message) {
  const store = asyncLocalStorage.getStore();
  const reqId = store ? store.get('requestId') : 'SYSTEM';
  console.log(`[Request-ID: ${reqId}] ${message}`);
}

async function performDatabaseQuery() {
  await new Promise(r => setTimeout(r, 50));
  log('Database query executed successfully');
}

http.createServer((req, res) => {
  const requestId = req.headers['x-request-id'] || crypto.randomUUID();
  const contextMap = new Map([['requestId', requestId]]);

  // Wrap request lifecycle inside AsyncLocalStorage
  asyncLocalStorage.run(contextMap, async () => {
    log('Incoming HTTP request received');
    await performDatabaseQuery();
    log('Sending response to client');
    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify({ status: 'OK', requestId }));
  });
}).listen(4000);
```

### 5.4 The Built-in Test Runner (`node:test`, `node:assert`)

Starting in Node.js 18 and stable in 20+, Node provides a native, zero-dependency test runner that executes tests in milliseconds:

```javascript
// tests/sample.test.mjs
import { describe, it, before, after, mock } from 'node:test';
import assert from 'node:assert/strict';

describe('Calculator Service Test Suite', () => {
  before(() => {
    console.log('Suite setup...');
  });

  after(() => {
    console.log('Suite teardown...');
  });

  it('performs strict equality and object assertions', () => {
    assert.equal(1 + 1, 2);
    assert.deepEqual({ a: 1, b: [2, 3] }, { a: 1, b: [2, 3] });
  });

  it('verifies async promise rejection', async () => {
    await assert.rejects(
      async () => { throw new Error('Unauthorized'); },
      { name: 'Error', message: 'Unauthorized' }
    );
  });

  it('mocks timer functions deterministically', () => {
    mock.timers.enable({ apis: ['setTimeout'] });
    let fired = false;

    setTimeout(() => { fired = true; }, 5000);
    assert.equal(fired, false);

    mock.timers.tick(5000); // Advance virtual time instantly!
    assert.equal(fired, true);

    mock.timers.reset();
  });
});
```

[Back to top ⤴️](#-table-of-contents)

---

## 6. Stage 6: Performance, Security & Operations

This stage is about running Node in production: measuring where time and memory go, finding leaks, using cryptography correctly, and shutting down without dropping requests. Afterwards you can profile a slow service, explain its memory graph, and deploy it with zero-downtime restarts.

### 6.1 Performance Hooks and High-Resolution Timing (`node:perf_hooks`)

The `node:perf_hooks` module provides high-resolution time measurements based on the W3C Web Performance API.

```javascript
import { performance, PerformanceObserver } from 'node:perf_hooks';

// 1. Observe performance measures and garbage-collection pauses
const obs = new PerformanceObserver((list) => {
  for (const entry of list.getEntries()) {
    console.log(`[Performance: ${entry.name}] duration=${entry.duration.toFixed(2)}ms`);
  }
});

obs.observe({ entryTypes: ['measure', 'gc'], buffered: false });

// 2. Micro-benchmarking with performance.mark and performance.measure
performance.mark('hash-start');

// Simulate work
for (let i = 0; i < 1e6; i++) {
  Math.sqrt(i);
}

performance.mark('hash-end');
performance.measure('Sqrt-Loop-Performance', 'hash-start', 'hash-end');
```

### 6.2 Heap Statistics and Snapshots (`node:v8`)

```javascript
import v8 from 'node:v8';

console.log('--- V8 Heap Statistics ---');
const heapStats = v8.getHeapStatistics();
console.log('Total Heap Size:', (heapStats.total_heap_size / (1024 * 1024)).toFixed(2), 'MB');
console.log('Used Heap Size:', (heapStats.used_heap_size / (1024 * 1024)).toFixed(2), 'MB');
console.log('Heap Size Limit:', (heapStats.heap_size_limit / (1024 * 1024)).toFixed(2), 'MB');

// Programmatically generate a V8 Heap Snapshot on high memory threshold
export function takeHeapSnapshotIfExceeded(thresholdMB) {
  const currentMB = v8.getHeapStatistics().used_heap_size / (1024 * 1024);
  if (currentMB > thresholdMB) {
    const filename = `heap-${Date.now()}.heapsnapshot`;
    v8.writeHeapSnapshot(filename);
    console.warn(`Heap snapshot written to ${filename} due to memory threshold breach`);
  }
}
```

### 6.3 Memory Leak Profiling with Heap Snapshots

Common Node.js memory leaks include:
1. **Accidental Global Variables**: Storing items in global arrays or Maps without eviction.
2. **Dangling Event Listeners**: Adding listeners in request handlers without calling `removeListener()`.
3. **Closure Scope Retention**: Closures holding references to large buffers or request objects.

```mermaid
sequenceDiagram
    autonumber
    actor Engineer as SRE Engineer
    participant Node as Node.js Process (Port 9229)
    participant Chrome as Chrome DevTools (chrome://inspect)

    Engineer->>Node: node --inspect server.js
    Engineer->>Chrome: Connect to Remote Target
    Engineer->>Chrome: Take Heap Snapshot 1 (Baseline)
    Engineer->>Node: Execute 1,000 synthetic HTTP requests (Apache Bench)
    Engineer->>Chrome: Take Heap Snapshot 2 (Under Load)
    Chrome->>Chrome: Comparison View (Identify objects allocated between Snapshot 1 and 2)
    Note over Chrome: Spot detached closures & runaway Map sizes!
```

### 6.4 Cryptography (`node:crypto`)

The `node:crypto` module provides cryptographic functionality that includes a set of wrappers for OpenSSL's hash, HMAC, cipher, decipher, sign, and verify functions.

```mermaid
flowchart TD
    Crypto["node:crypto"] --> Hash["Hashing: SHA-256 / SHA-512 (One-Way)"]
    Crypto --> HMAC["HMAC: Hash with Secret Key (Integrity & Authenticity)"]
    Crypto --> Symmetric["Symmetric Encryption: AES-256-GCM (Encrypted + Auth Tag)"]
    Crypto --> Asymmetric["Asymmetric: RSA / ECDSA (Public/Private Key Pairs)"]
    Crypto --> KDF["Key Derivation: scrypt / Argon2 / pbkdf2"]
```

#### Complete Cryptography Suite: AES-256-GCM Authenticated Encryption

```javascript
import crypto from 'node:crypto';

const ALGORITHM = 'aes-256-gcm';
const IV_LENGTH = 12; // 96-bit recommended for GCM

export function encryptPayload(plaintext, secretKey32Bytes) {
  const iv = crypto.randomBytes(IV_LENGTH);
  const cipher = crypto.createCipheriv(ALGORITHM, secretKey32Bytes, iv);

  let encrypted = cipher.update(plaintext, 'utf8', 'hex');
  encrypted += cipher.final('hex');

  // GCM provides an authentication tag that prevents tampering
  const authTag = cipher.getAuthTag();

  return {
    iv: iv.toString('hex'),
    authTag: authTag.toString('hex'),
    ciphertext: encrypted
  };
}

export function decryptPayload(encryptedObj, secretKey32Bytes) {
  const iv = Buffer.from(encryptedObj.iv, 'hex');
  const authTag = Buffer.from(encryptedObj.authTag, 'hex');
  const decipher = crypto.createDecipheriv(ALGORITHM, secretKey32Bytes, iv);

  decipher.setAuthTag(authTag); // Verifies data integrity

  let decrypted = decipher.update(encryptedObj.ciphertext, 'hex', 'utf8');
  decrypted += decipher.final('utf8'); // Throws if ciphertext was modified!

  return decrypted;
}

// Verification Test
const key = crypto.randomBytes(32); // 256 bits
const secretMessage = "Confidential financial transaction payload";
const encrypted = encryptPayload(secretMessage, key);
console.log('Encrypted:', encrypted);

const decrypted = decryptPayload(encrypted, key);
console.log('Decrypted successfully:', decrypted);

// Constant-time string comparison to prevent Timing Attacks
const tokenA = Buffer.from('secret-auth-token-123');
const tokenB = Buffer.from('secret-auth-token-123');
const isMatch = crypto.timingSafeEqual(tokenA, tokenB);
console.log('Constant-Time Match:', isMatch);
```

### 6.5 Graceful Shutdown and Zero-Downtime Signal Handling

When deploying on Kubernetes or Docker, processes receive `SIGTERM` signals before being terminated. A robust graceful shutdown drains in-flight requests and closes database connections cleanly:

```javascript
import http from 'node:http';

const server = http.createServer((req, res) => {
  setTimeout(() => {
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    res.end('Processed transaction');
  }, 2000);
});

server.listen(3000);

let isShuttingDown = false;

function gracefulShutdown(signal) {
  console.log(`Received ${signal}. Initiating graceful shutdown...`);
  isShuttingDown = true;

  // 1. Stop accepting new connections
  server.close(async () => {
    console.log('HTTP server closed. In-flight requests drained.');

    try {
      // 2. Disconnect database connection pools & Redis clients
      // await db.disconnect();
      // await redis.quit();
      console.log('Database connections closed cleanly.');
      process.exit(0);
    } catch (err) {
      console.error('Error during teardown:', err);
      process.exit(1);
    }
  });

  // Force shutdown if connections do not drain within 10 seconds
  setTimeout(() => {
    console.error('Graceful shutdown timeout exceeded. Forcing exit (SIGKILL)...');
    process.exit(1);
  }, 10000).unref(); // unref prevents timeout from keeping process alive
}

process.on('SIGTERM', () => gracefulShutdown('SIGTERM'));
process.on('SIGINT', () => gracefulShutdown('SIGINT'));
```

[Back to top ⤴️](#-table-of-contents)

---

## M1. Masterclass: Streaming File Server with Range Requests

### M1.1 Reference Implementation

A streaming file server that supports HTTP `Range` requests (so media players can seek) and gzip, with a containment check that blocks `../` path traversal. Every response body goes through `pipeline()`, so a slow client applies backpressure all the way to the disk read.

```javascript
import http from 'node:http';
import { createReadStream } from 'node:fs';
import { stat } from 'node:fs/promises';
import path from 'node:path';
import zlib from 'node:zlib';
import { pipeline } from 'node:stream/promises';

const PUBLIC_DIR = path.resolve('public');

// Resolve inside PUBLIC_DIR or return null. path.join alone would follow "../".
function resolveSafe(urlPath) {
  let decoded;
  try {
    decoded = decodeURIComponent(new URL(urlPath, 'http://x').pathname);
  } catch {
    return null;
  }
  const full = path.resolve(PUBLIC_DIR, '.' + path.sep + decoded);
  return full.startsWith(PUBLIC_DIR + path.sep) ? full : null;
}

const server = http.createServer(async (req, res) => {
  const filePath = resolveSafe(req.url ?? '/');
  const info = filePath ? await stat(filePath).catch(() => null) : null;
  if (!filePath || !info || !info.isFile()) {
    res.writeHead(404, { 'Content-Type': 'text/plain' }).end('File Not Found');
    return;
  }

  try {
    // Range requests: "bytes=start-end" lets video and audio players seek.
    const range = /^bytes=(\d*)-(\d*)$/.exec(req.headers.range ?? '');
    if (range) {
      const start = range[1] ? Number(range[1]) : 0;
      const end = range[2] ? Math.min(Number(range[2]), info.size - 1) : info.size - 1;
      if (start > end || start >= info.size) {
        res.writeHead(416, { 'Content-Range': `bytes */${info.size}` }).end();
        return;
      }
      res.writeHead(206, {
        'Content-Range': `bytes ${start}-${end}/${info.size}`,
        'Accept-Ranges': 'bytes',
        'Content-Length': end - start + 1,
        'Content-Type': 'application/octet-stream'
      });
      await pipeline(createReadStream(filePath, { start, end }), res);
      return;
    }

    if ((req.headers['accept-encoding'] ?? '').includes('gzip')) {
      res.writeHead(200, { 'Content-Encoding': 'gzip', 'Content-Type': 'application/octet-stream' });
      await pipeline(createReadStream(filePath), zlib.createGzip(), res);
    } else {
      res.writeHead(200, { 'Content-Length': info.size, 'Content-Type': 'application/octet-stream' });
      await pipeline(createReadStream(filePath), res);
    }
  } catch (err) {
    // A client that disconnects mid-download rejects the pipeline; nothing to send back.
    if (err.code !== 'ERR_STREAM_PREMATURE_CLOSE') console.error('streaming error', err);
  }
});

server.listen(8080, '0.0.0.0', () => console.log('Streaming server running on :8080'));
```

[Back to top ⤴️](#-table-of-contents)

---

## M2. Masterclass: Queue-Backed Order Gateway

### M2.1 Architecture and Implementation

This gateway accepts orders over HTTP and processes them asynchronously. It demonstrates:
- Fastify server with compiled schema validation
- Distributed BullMQ background queue with Redis
- Correlation tracking across async boundaries using `AsyncLocalStorage`
- Graceful shutdown and health check endpoints

```mermaid
flowchart LR
    Client["Client POST /orders"] --> Fastify["Fastify Ingestion Gateway"]
    Fastify --> ALS["AsyncLocalStorage (Assigns reqId: uuid)"]
    ALS --> Queue["BullMQ Producer (order-processing-queue)"]
    Queue --> Redis[("Redis Broker / In-Memory Store")]
    Redis --> Worker["BullMQ Worker Process"]
    Worker --> Log["Context Logger [reqId: uuid] Task Completed"]
```

```javascript
// order-gateway.mjs
import Fastify from 'fastify';
import { Queue, Worker } from 'bullmq';
import { AsyncLocalStorage } from 'node:async_hooks';
import crypto from 'node:crypto';

const als = new AsyncLocalStorage();
const redisConnection = { host: 'localhost', port: 6379 };

// 1. Initialize BullMQ Queue & Worker
const orderQueue = new Queue('order-processing-queue', { connection: redisConnection });

const orderWorker = new Worker('order-processing-queue', async (job) => {
  const { orderId, amount, requestId } = job.data;
  
  // Rehydrate execution context in worker thread
  await als.run({ requestId }, async () => {
    console.log(`[Worker - ${requestId}] Processing payment for Order #${orderId} ($${amount})...`);
    await new Promise(r => setTimeout(r, 200)); // Simulate payment gateway call
    console.log(`[Worker - ${requestId}] Order #${orderId} settled successfully.`);
  });
}, { connection: redisConnection });

// 2. Initialize Fastify Server
const app = Fastify({ logger: false });

app.addHook('onRequest', (req, reply, done) => {
  const requestId = req.headers['x-request-id'] || crypto.randomUUID();
  als.run({ requestId }, done);
});

app.post('/api/orders', {
  schema: {
    body: {
      type: 'object',
      required: ['orderId', 'amount'],
      properties: {
        orderId: { type: 'string' },
        amount: { type: 'number', minimum: 1 }
      }
    }
  }
}, async (request, reply) => {
  const store = als.getStore();
  const requestId = store.requestId;

  const { orderId, amount } = request.body;
  
  // Enqueue background job with correlation ID
  await orderQueue.add('charge-card', { orderId, amount, requestId });

  reply.status(202).send({
    status: 'ACCEPTED',
    message: 'Order enqueued for asynchronous processing',
    orderId,
    requestId
  });
});

await app.listen({ port: 5000, host: '0.0.0.0' });
console.log('Order Microservice Gateway listening on port 5000');
```

[Back to top ⤴️](#-table-of-contents)

---

## M3. Masterclass: Real-Time WebSocket Gateway

### M3.1 Architecture and Implementation

A WebSocket gateway built on the `ws` package: clients join a room from the query string, and every message is broadcast to the other members of that room. The diagram also shows where a worker thread would take over CPU-heavy per-message work such as encryption; the code below keeps that step out to stay short.

```mermaid
flowchart TD
    BrowserA["Client Alice"] <-->|"ws://localhost:8080"| WSGateway["WebSocket Server (ws)"]
    BrowserB["Client Bob"] <-->|"ws://localhost:8080"| WSGateway
    WSGateway -->|"Offload Broadcast Encryption"| Worker["Worker Thread (Crypto Engine)"]
    Worker -->|"Encrypted Frame"| WSGateway
```

```javascript
// realtime-server.mjs
import { WebSocketServer, WebSocket } from 'ws';
import { createServer } from 'node:http';

const server = createServer();
const wss = new WebSocketServer({ server });

// Map of roomId -> Set of WebSockets
const rooms = new Map();

wss.on('connection', (ws, req) => {
  const url = new URL(req.url, 'http://localhost:8080');
  const roomId = url.searchParams.get('room') || 'general';

  if (!rooms.has(roomId)) rooms.set(roomId, new Set());
  rooms.get(roomId).add(ws);
  console.log(`Client joined room: ${roomId} (Total in room: ${rooms.get(roomId).size})`);

  ws.on('message', (data) => {
    // Broadcast to all other peers in the room
    const clients = rooms.get(roomId);
    if (clients) {
      for (const client of clients) {
        if (client !== ws && client.readyState === WebSocket.OPEN) {
          client.send(data);
        }
      }
    }
  });

  ws.on('close', () => {
    rooms.get(roomId)?.delete(ws);
    if (rooms.get(roomId)?.size === 0) rooms.delete(roomId);
    console.log(`Client left room: ${roomId}`);
  });
});

server.listen(8080, () => console.log('Real-Time Gateway listening on ws://localhost:8080'));
```

[Back to top ⤴️](#-table-of-contents)

---

## 7. Stage 7: Staff Node.js Engineer Interview Handbook & Cheatsheet

These questions cover what interviewers probe for senior and staff Node.js roles: runtime internals, failure modes, trade-offs and production debugging. Answers link back to the stage that explains the topic in depth.

### 7.1 Core Concepts (Q1-Q20)

#### Q1. What is the exact execution order of the Libuv Event Loop phases?

1. Timers (`setTimeout`, `setInterval`)
2. Pending Callbacks (I/O callbacks deferred from previous loop iteration)
3. Idle, Prepare (Internal Libuv housekeeping)
4. Poll (Calculates timeout, waits for OS I/O events, executes I/O callbacks)
5. Check (`setImmediate`)
6. Close Callbacks (`socket.on('close')`)

*Between each phase and callback*, Node drains the Microtask Queue (`process.nextTick` followed by Promise jobs).

The order between the two microtask queues depends on context: in CommonJS top-level code and in plain callbacks the nextTick queue drains first, but code that is already running inside a Promise job (an ES module body, anything after an `await`) drains Promise jobs first. Section [1.9](#19-microtasks-the-processnexttick-and-promise-queues) shows both cases with tested output.

#### Q2. What is the difference between `process.nextTick()` and `setImmediate()`?

- `process.nextTick()` does NOT run in the Event Loop; it executes immediately at the end of the current synchronous tick before the Event Loop advances to any other phase or callback. Recursive `process.nextTick` can starve the Event Loop of all I/O!
- `setImmediate()` queues a callback to run in the **Check Phase** of the Event Loop, after the Poll phase processes I/O events.

#### Q3. Why does `setTimeout(fn, 0)` not always execute before `setImmediate(fn)`?

In the root scope, the ordering depends on the performance of the process:

- If Node.js enters the event loop and the OS timer has elapsed (1 ms or more), the Timers phase executes `setTimeout` first.
- If the timer has not elapsed yet, Node skips Timers and reaches the Check phase, executing `setImmediate` first.

However, **inside any I/O callback** (e.g., `fs.readFile`), `setImmediate` is **guaranteed** to execute before `setTimeout(fn, 0)` because the Check phase immediately follows the Poll phase.

#### Q4. What is the difference between `setImmediate` and `setTimeout(fn, 1)`?

`setTimeout(fn, 1)` registers a timer with the OS. Due to timer quantization and clock drift, it executes after at least 1 ms in the Timers phase. `setImmediate()` queues directly into the Check phase and executes as soon as the current Poll phase finishes, with zero OS timer overhead.

#### Q5. What is the difference between `Buffer.alloc()` and `Buffer.allocUnsafe()`?

- `Buffer.alloc(size)`: Allocates memory and initializes every byte to `0`. Safe, but incurs minor initialization latency.
- `Buffer.allocUnsafe(size)`: Allocates raw memory without clearing previous contents. Significantly faster, but contains uninitialized memory that can leak sensitive data (passwords, TLS keys) if sent over network sockets without being overwritten.

#### Q6. What is the difference between `ArrayBuffer`, `TypedArray`, and `Buffer`?

- `ArrayBuffer`: A fixed-length raw binary data buffer in ECMAScript. Cannot be accessed or modified directly.
- `TypedArray` (`Uint8Array`, `Float64Array`): A view over an `ArrayBuffer` providing typed numeric indexing.
- `Buffer`: Node.js subclass of `Uint8Array` with additional binary encoding, slicing, and native I/O capabilities.

#### Q7. How does backpressure work in Node.js streams?

When a writable stream cannot keep up with a readable stream, its internal buffer fills up to `highWaterMark`. When full, `writable.write(chunk)` returns `false`. Compliant readable streams pause reading (`readable.pause()`) until the writable stream flushes its buffer to the OS and emits the `'drain'` event, signaling the reader to resume.

#### Q8. What is the difference between `fs.readFile()` and `fs.createReadStream()`?

- `fs.readFile()`: Loads the entire file into V8 memory at once. If a file is 2GB, the process will consume 2GB of RAM and likely crash with an OOM error.
- `fs.createReadStream()`: Streams the file in small chunks (default 64KB `highWaterMark`), keeping RAM usage constant (a few MB) regardless of file size.

#### Q9. What is the difference between `Stream.Readable.from()` and `fs.createReadStream()`?

`Readable.from(iterable)` converts any JavaScript iterable or AsyncIterable (generators, arrays, sets) into a standard Node.js Readable stream, complete with backpressure handling.

#### Q10. What is the purpose of `stream.finished()`?

`stream.finished(stream, callback)` provides a reliable notification when a stream has completely finished, closed, or encountered an error. It handles both modern and legacy streams and guarantees cleanup without missing premature socket termination events.

#### Q11. What is the `captureRejections` option in `EventEmitter`?

Introduced in Node 13.4, setting `{ captureRejections: true }` on an EventEmitter automatically catches unhandled Promise rejections inside `async` event listener functions and routes them to the emitter's `'error'` event rather than triggering an unhandled promise rejection.

#### Q12. What is `unref()` and `ref()` on timers and network sockets?

- `timer.unref()`: Tells the Event Loop that this active timer should NOT prevent the Node.js process from exiting if it is the only task remaining.
- `timer.ref()`: Restores default behavior, ensuring the Event Loop stays alive until the timer expires.

#### Q13. How does Node.js implement the Module Resolution Algorithm in CommonJS?

1. Core modules: Loaded instantaneously if name matches (`fs`, `path`).
2. File modules (`./`, `../`, `/`): Checks path, appends extensions (`.js`, `.json`, `.node`), checks directory `index.js` or `package.json` `main`.
3. Package modules (`lodash`): Searches `./node_modules`, `../node_modules`, traversing upward to root filesystem until found or throws `MODULE_NOT_FOUND`.

#### Q14. How does Node.js resolve circular dependencies in CommonJS?

When Module A requires Module B, and Module B requires Module A, Module B receives an **incomplete (partial) copy of Module A's `exports` object** evaluated up to that point. In ES Modules, circular dependencies use live bindings, but accessing uninitialized variables triggers a `ReferenceError (TDZ - Temporal Dead Zone)`.

#### Q15. What is the purpose of the `exports` field in `package.json`?

The `exports` field provides strict encapsulation for modern npm packages. It specifies exact entrypoints for different environments (`import`, `require`, `types`, `default`) and completely hides private internal files from consumer `require()` calls.

#### Q16. What is the difference between `fs.stat()`, `fs.lstat()`, and `fs.fstat()`?

- `fs.stat(path)`: Retrieves metadata of the file or directory. If path is a symbolic link, it follows the link and returns the target file's metadata.
- `fs.lstat(path)`: If path is a symbolic link, returns the metadata of the symbolic link itself.
- `fs.fstat(fd)`: Retrieves metadata directly from an open integer file descriptor.

#### Q17. What is the difference between `fs.watch` and `fs.watchFile`?

- `fs.watch`: Uses native OS notification APIs (Linux `inotify`, macOS `FSEvents`, Windows `ReadDirectoryChangesW`). Fast, lightweight, event-driven, but platform-inconsistent.
- `fs.watchFile`: Polls the file system periodically (checking `stat.mtime`). Reliable across all network filesystems (NFS), but incurs substantial CPU and disk polling overhead.

#### Q18. What is the difference between `String.prototype.normalize()` and path normalization?

`path.normalize()` resolves `.` and `..` path segments and duplicate slashes. `String.prototype.normalize()` performs Unicode canonical equivalence normalization (NFC, NFD, NFKC, NFKD), critical when handling international filenames across macOS and Linux.

#### Q19. What is the difference between `crypto.randomBytes()` and `Math.random()`?

- `Math.random()`: Pseudo-random number generator (PRNG). Completely deterministic and predictable; **never use for security tokens, passwords, or session IDs!**
- `crypto.randomBytes()`: Cryptographically Secure Pseudo-Random Number Generator (CSPRNG) seeded directly from operating system entropy (`/dev/urandom` on Unix, CryptGenRandom on Windows).

#### Q20. What is the difference between `process.exit(0)` and `process.exit(1)`?

- `0`: Success status code. Informs the host OS, Docker, or Kubernetes that the process completed its mission normally.
- `1` (or non-zero): Error status code. Triggers container restart policies (e.g., Kubernetes `restartPolicy: OnFailure`) and alerts monitoring pipelines.

### 7.2 Internals & Performance (Q21-Q32)

#### Q21. How do you prevent thread pool starvation in Node.js?

Heavy use of `crypto.pbkdf2`, `fs.readFile`, or `dns.lookup` can saturate the default 4-thread Libuv pool. To mitigate:

1. Increase `UV_THREADPOOL_SIZE` (the libuv maximum is 1024), for example `UV_THREADPOOL_SIZE=64 node server.js`. This helps jobs that wait (disk, `getaddrinfo`), not jobs that compute on few cores.
2. Offload CPU-heavy tasks to `worker_threads` instead of relying on the libuv thread pool.
3. Use non-blocking alternatives (e.g., `dns.resolve()` instead of `dns.lookup()`).

#### Q22. What is the difference between `dns.lookup()` and `dns.resolve()`?

- `dns.lookup()`: Uses the synchronous `getaddrinfo(3)` OS system call, which executes in the **libuv thread pool**. High volumes of `dns.lookup()` calls can saturate the 4-thread pool, starving file I/O and crypto operations.
- `dns.resolve()`: Uses the C-Ares asynchronous library, making direct asynchronous network queries over non-blocking UDP sockets without consuming any worker threads.

#### Q23. What is V8 Code Caching and how can it accelerate Node.js cold startup?

Each start, V8 parses and compiles the JavaScript it loads. A code cache stores the compiled output so later runs can skip that work. Since 22.1, `module.enableCompileCache()` or the `NODE_COMPILE_CACHE=<dir>` environment variable persists V8's code cache for loaded modules on disk; CLIs and serverless functions with many dependencies benefit the most. Lower-level options are `vm.Script` with `cachedData` for custom loaders and the code cache embedded in single executable applications. Measure the gain with `hyperfine` or repeated `time node app.js`: it varies with how much code runs at startup.

#### Q24. How does Node.js manage memory for Buffers vs standard V8 objects?

Buffer bytes are stored outside the V8 heap, in `ArrayBuffer` backing stores, and are counted in `process.memoryUsage().external` and `.arrayBuffers` rather than `heapUsed`. The small JavaScript `Buffer` object on the heap points at that memory, and the memory is freed when the object is garbage-collected.

For small sizes (under half of `Buffer.poolSize`, so below 4 KiB by default), `Buffer.allocUnsafe()` and `Buffer.from()` slice the memory from a shared pre-allocated 8 KiB pool to avoid many tiny allocations; `Buffer.alloc()` never uses the pool. Consequences: a tiny `subarray` can keep a large parent allocation alive, and `--max-old-space-size` does not limit Buffer memory, so a process can be OOM-killed with a modest heap.

#### Q25. What is the purpose of `process.memoryUsage()` fields?

- `rss` (Resident Set Size): Total physical RAM occupied by the process.
- `heapTotal`: Total memory allocated by V8 for the heap.
- `heapUsed`: Actual memory currently utilized by JavaScript objects.
- `external`: Memory bound to C++ objects managed by V8 (Buffers).
- `arrayBuffers`: Memory allocated for ArrayBuffers and SharedArrayBuffers.

#### Q26. What is a WeakRef and FinalizationRegistry, and when should they be used in Node.js?

- `WeakRef`: Holds a weak reference to an object without preventing it from being garbage collected.
- `FinalizationRegistry`: Registers a cleanup callback that executes when a tracked object is collected by V8 GC.

Used in advanced caching layers where cached objects can be evicted automatically by V8 under memory pressure.

#### Q27. How do you measure Event Loop Lag / Delay in production?

Use `perf_hooks.monitorEventLoopDelay({ resolution: 20 })`:
```javascript
import { monitorEventLoopDelay } from 'node:perf_hooks';
const histogram = monitorEventLoopDelay({ resolution: 20 });
histogram.enable();
// In periodic metrics reporting:
console.log('Mean delay:', histogram.mean / 1e6, 'ms');
console.log('99th percentile delay:', histogram.percentile(99) / 1e6, 'ms');
```
If p99 delay exceeds 50ms, synchronous CPU work is blocking the single-threaded Event Loop.

#### Q28. What is the difference between `http.Agent({ keepAlive: true })` and default HTTP requests?

Without keep-alive every request opens a new TCP connection (and a TLS handshake for HTTPS), which adds round trips and exhausts ephemeral ports under load. An agent with `keepAlive: true` pools idle sockets and reuses them for the next request to the same origin. Since Node 19 the global `http`/`https` agent enables keep-alive by default, and `fetch` (undici) always pools connections. Tune `maxSockets` and `maxFreeSockets` per upstream, and set the client's idle timeout below the server's keep-alive timeout so the client never reuses a socket the server is closing (the source of sporadic `ECONNRESET`).

#### Q29. What is the C++ N-API (Node-API)?

Node-API is an ABI-stable C/C++ API for building native binary addons in Node.js. Unlike legacy V8 C++ bindings, addons compiled against Node-API do not need to be recompiled when upgrading Node.js major versions.

#### Q30. What is the purpose of `node:diagnostics_channel`?

`diagnostics_channel` is an internal publish-subscribe channel mechanism designed for APM (Application Performance Monitoring) tools to publish telemetry events synchronously with near-zero overhead without monkey-patching core modules.

#### Q31. How does `crypto.timingSafeEqual()` prevent timing attacks?

Standard string comparison (`===`) terminates early as soon as the first mismatched character is detected. An attacker can measure request latency down to microseconds to guess secrets character by character. `crypto.timingSafeEqual(bufA, bufB)` executes in constant time, taking the exact same number of CPU cycles regardless of where or whether characters match.

#### Q32. How do you achieve true CPU parallelism in Node.js?

By using `node:worker_threads` (for shared-memory multi-threading) or `node:cluster` / `node:child_process` (for multi-process execution). Pure async functions (`async/await`) only interleave I/O latency; they do not run CPU calculations in parallel.

### 7.3 Architecture & Design (Q33-Q43)

#### Q33. What is the difference between `cluster` and `worker_threads`?

- `node:cluster`: Multi-process architecture. Each worker is an independent OS process with its own V8 instance, memory heap, and Event Loop. Communication occurs via IPC. Shares TCP server ports via OS socket passing.
- `node:worker_threads`: Multi-threaded architecture within a single OS process. Each worker runs in an isolated V8 Isolate, but threads can share memory directly via `SharedArrayBuffer` with zero IPC serialization overhead.

#### Q34. What is `AsyncLocalStorage` and why is it superior to manual context passing?

`AsyncLocalStorage` creates asynchronous state stores that automatically propagate across asynchronous boundaries (promises, callbacks, timers). It eliminates the anti-pattern of passing request IDs or user tokens down through dozens of function parameters ("parameter drilling"), allowing deep database clients and loggers to access request context transparently.

#### Q35. What is the difference between `spawn()`, `exec()`, and `execFile()` in `child_process`?

- `spawn()`: Streams stdout/stderr via streams. Ideal for long-running processes or large data transfers without memory buffering.
- `exec()`: Spawns a subshell (`/bin/sh` or `cmd.exe`) and buffers the entire output in memory before calling a callback. Vulnerable to shell command injection!
- `execFile()`: Invokes an executable binary directly without spawning a subshell, making it immune to shell injection vulnerabilities.

#### Q36. What is the difference between `child_process.fork()` and `cluster.fork()`?

- `child_process.fork('path.js')`: Spawns an arbitrary new Node.js process with an IPC communication channel.
- `cluster.fork()`: Specifically designed for server scaling; forks a copy of the *current script* and automatically configures OS-level TCP socket sharing across worker processes.

#### Q37. How do you handle database connection pooling gracefully in Node.js?

Instantiate a single shared `pg.Pool` or database connection manager at application startup. Set `max` connection limits based on available PostgreSQL slots. Always release clients back to the pool in a `finally` block:
```javascript
const client = await pool.connect();
try {
  await client.query('...');
} finally {
  client.release();
}
```

#### Q38. How do you implement distributed locking in Node.js?

For a single Redis instance, acquire with `SET lock:<resource> <random-token> NX PX 30000`, do the work, and release with a Lua script that deletes the key only if it still holds your token, so you never release a lock that expired and was taken by another worker. Redlock extends this to several independent Redis nodes and requires a majority; its safety under clock drift and long GC pauses is debated.

Whatever the lock, a paused process can resume after its lease expired. For correctness-critical work, pass a **fencing token** (a number that increases with every acquisition) to the storage layer and have it reject writes with an older token. Often a database constraint or a transactional `SELECT ... FOR UPDATE` removes the need for a distributed lock altogether.

#### Q39. How do you trace asynchronous operations with OpenTelemetry in Node.js?

Initialize the OpenTelemetry NodeSDK at the very first line of your entrypoint before any other imports. OpenTelemetry uses `AsyncLocalStorage` and monkey-patches `http`, `pg`, and `redis` to generate spans and propagate W3C Trace Context headers (`traceparent`) automatically across microservices. For ES module applications, load the setup file with `node --import ./instrumentation.mjs app.js` so the loader hooks that patch modules are registered before the application's own imports run.

#### Q40. How do you handle file uploads in Node.js without storing entire files in memory?

Use streaming multipart parsers like **`busboy`** or **`formidable`**. As binary chunks stream from the incoming HTTP socket, pipe them directly into a file write stream or S3 multipart upload stream, keeping server RAM below 5MB.

#### Q41. How do you architect an enterprise Node.js microservice for 99.999% availability?

1. Stateless process model: Externalize all session and cache state to distributed Redis/PostgreSQL clusters.
2. Containerized orchestration: Run multiple replicas across distinct Availability Zones in Kubernetes.
3. Zero-downtime rolling updates with readiness/liveness probes and graceful shutdown hooks (`SIGTERM`).
4. Distributed tracing and APM using OpenTelemetry and Prometheus metrics for Event Loop lag and heap utilization.
5. Asynchronous job offloading via distributed message brokers (Kafka/BullMQ) to prevent blocking the Event Loop.

#### Q42. How do you gracefully handle SIGUSR2 for zero-downtime cluster restarts?

When the primary cluster process receives `SIGUSR2`, it iterates over its worker collection sequentially:

1. Spawns a new replacement worker (`cluster.fork()`).
2. Waits for the replacement worker to emit `'listening'`.
3. Disconnects and kills the old worker (`oldWorker.disconnect()`).
4. Repeats for the next worker, ensuring there is never a millisecond without healthy workers serving traffic.

#### Q43. What is prototype pollution, and how do you prevent it when parsing JSON or query strings?

Prototype pollution happens when attacker-controlled keys such as `__proto__`, `constructor` or `prototype` reach a recursive merge, `set(obj, path, value)` helper or query-string parser, and the code writes to `Object.prototype` instead of to the target object. Every object in the process then inherits the injected property, which can flip authorisation checks (`if (user.isAdmin)`) or crash code paths.

1. Validate input against a schema that rejects unknown keys before merging it into anything.
2. Reject or skip the keys `__proto__`, `constructor` and `prototype` in merge and path-setting helpers.
3. Use `Object.create(null)` or `Map` for dictionaries built from user input, so there is no prototype chain to reach.
4. Run with `--disable-proto=delete` to remove the `__proto__` accessor, or freeze `Object.prototype` in security-critical processes so tampering throws.
5. Keep parsers and merge utilities up to date; many past CVEs in this class were in small utility packages.

### 7.4 Scenario & Debugging Questions (Q44-Q49)

#### Q44. A service crashes with "JavaScript heap out of memory". How do you stabilise, diagnose and prevent it?

V8 aborts the process when the old generation reaches its limit. That limit is derived from the machine's memory and capped by V8 (commonly 2-4 GB on 64-bit systems); read it with `v8.getHeapStatistics().heap_size_limit`.

1. **Stabilise.** Set `--max-old-space-size` to about 75% of the container limit so V8, not the kernel, fails first, and add `--heapsnapshot-near-heap-limit=2` so the next crash leaves evidence.
2. **Diagnose.** Take heap snapshots (`v8.writeHeapSnapshot()` or the flag above) at intervals under load and open them in Chrome DevTools' Comparison view. Sort by retained size and follow the retainer chain from the growing objects to the root: usually an unbounded `Map` cache, listeners added per request, or closures that capture large request objects.
3. **Prevent.** Bound every cache (size and TTL), remove listeners with `{ signal }` or `once`, alert on `heapUsed` growth across GC cycles rather than on absolute RSS, and keep a load test that runs long enough to show slow leaks.

#### Q45. A request with a crafted input freezes the server for seconds. How do you detect and prevent ReDoS?

ReDoS (regular expression denial of service) happens when a pattern with nested or overlapping quantifiers, such as `/(a+)+$/` or `/^(\w+\s?)*$/`, meets an input that forces catastrophic backtracking. V8's backtracking engine can then take exponential time, and because the match runs on the main thread, every request stalls.

1. **Detect.** The symptom is one CPU at 100% with event-loop delay in seconds; a CPU profile (`--cpu-prof`, or the inspector) shows the time inside `RegExp` execution. Lint patterns with a ReDoS checker in CI.
2. **Fix.** Rewrite the pattern without nested quantifiers, cap input length before matching (`if (s.length > 256)`), or use a linear-time engine such as RE2 (the `re2` package) for patterns that handle untrusted input.
3. **Contain.** JavaScript regex matching cannot be interrupted, so if you must run user-supplied patterns, run them in a worker thread and terminate it after a deadline.

#### Q46. The process exits with code 1 after an unhandled rejection. How does Node treat uncaught errors, and what should your handlers do?

- `uncaughtException`: Indicates an unhandled synchronous error. The application state is corrupted and unpredictable; log the error and invoke `process.exit(1)`. Never attempt to resume normal execution!
- `unhandledRejection`: Emitted when a Promise rejects without a `.catch()` handler. Since Node 15, unhandled promise rejections terminate the process with exit code 1 by default (`--unhandled-rejections=throw`).

#### Q47. You need a last-chance crash report before exit. What does `process.setUncaughtExceptionCaptureCallback()` do?

It installs a single global error interceptor that supersedes `uncaughtException` listeners. When registered, any uncaught error is routed to this function and the standard Node.js default behavior (printing stack trace and aborting) is suppressed, allowing a clean custom emergency telemetry dump before exiting.

#### Q48. A production process is misbehaving and you cannot restart it. How do you attach a debugger?

Send the `SIGUSR1` signal to the process PID:
```bash
kill -SIGUSR1 <PID>
```
Node.js activates the V8 Inspector dynamically on port 9229 without interrupting running transactions, allowing Chrome DevTools connection.

`SIGUSR1` is not available on Windows; there, start with `--inspect` or use `process._debugProcess(pid)` from another Node process. The inspector binds to 127.0.0.1:9229; reach it through an SSH tunnel or `kubectl port-forward` and never expose it on a public interface, because it allows arbitrary code execution.

#### Q49. The server is hit by slow-header (Slowloris) attacks. Which `http.Server` settings protect it?

- `maxHeadersCount`: Caps incoming HTTP headers (default 2,000) to prevent hash collision DoS attacks.
- `headersTimeout`: Enforces a maximum duration for receiving complete HTTP request headers (default 60s), protecting the server against Slowloris attacks.

### 7.5 Staff Cheatsheet

| Need | Node.js answer | Gotcha |
| --- | --- | --- |
| Yield to I/O between CPU batches | `await new Promise((r) => setImmediate(r))` | `process.nextTick` and `queueMicrotask` do not yield; they run before I/O. |
| Run CPU-heavy work without blocking | `new Worker(url, { workerData })` or a pool such as Piscina | Each worker is a new V8 isolate: tens of MB and tens of ms to start. Pool them. |
| Use every core for an HTTP server | One process per core behind a load balancer, or `node:cluster` | Cluster workers share nothing; keep state in Redis or a database. |
| Deadline for any async operation | `AbortSignal.timeout(ms)` passed as `{ signal }` | `Promise.race` does not stop the losing operation. |
| Combine cancellation sources | `AbortSignal.any([timeout, userAbort])` | Needs 20.3+. |
| Request-scoped context (trace id, user) | `AsyncLocalStorage.run(store, fn)` + `getStore()` | Context is lost through callback APIs that pool work (some DB drivers) unless they use `AsyncResource`. |
| Copy a large file through a transform | `await pipeline(createReadStream(a), transform, createWriteStream(b))` | `.pipe()` neither forwards errors nor closes the source on failure. |
| Read a file line by line | `for await (const line of readline.createInterface({ input, crlfDelay: Infinity }))` | Without `crlfDelay: Infinity`, `\r\n` files can produce empty lines. |
| Bytes of a string | `Buffer.byteLength(str, 'utf8')` | `str.length` counts UTF-16 code units, not bytes. |
| Decode text split across chunks | `stream.setEncoding('utf8')` or `StringDecoder` | `chunk.toString()` per chunk corrupts multi-byte characters at edges. |
| `__dirname` in ESM | `import.meta.dirname` (20.11+) | `process.cwd()` is the shell's directory, not the module's. |
| Load CommonJS from ESM | `import pkg from 'pkg'`, or `createRequire(import.meta.url)` | Named imports only work for exports `cjs-module-lexer` can see. |
| Load ESM from CommonJS | `require('./esm.mjs')` (20.19+, 22.12+) | Throws `ERR_REQUIRE_ASYNC_MODULE` if the graph uses top-level `await`. |
| Restrict a package's public files | `"exports"` in `package.json` | Adding it later breaks deep imports: release as a major. |
| Run TypeScript without a build | `node file.ts` (22.18+, 24) | No type checking, no `enum`, no `paths`; imports need `.ts`. |
| Reproducible installs | `npm ci` (`--omit=dev` in production) | `npm install` may rewrite the lockfile. |
| Environment files | `node --env-file=.env app.js` | Real environment variables win over the file. |
| Heap limit in a container | `NODE_OPTIONS=--max-old-space-size=<75% of limit in MB>` | Without it the kernel OOM-kills you (exit 137) with no stack trace. |
| Measure event-loop lag | `perf_hooks.monitorEventLoopDelay()`; alert on p99 | Mean hides spikes; report percentiles. |
| See what keeps the process alive | `process.getActiveResourcesInfo()` | `_getActiveHandles()` is private and undocumented. |
| Capture a heap snapshot | `v8.writeHeapSnapshot()` or `--heapsnapshot-near-heap-limit=2` | Writing a snapshot pauses the process and needs about as much memory again. |
| CPU profile in production | `node --cpu-prof` or the inspector via `SIGUSR1` | `--inspect` on `0.0.0.0` exposes remote code execution. Bind to localhost and tunnel. |
| Compare secrets | `crypto.timingSafeEqual(a, b)` | Both buffers must have equal length; compare hashes (HMACs) of the values. |
| Hash passwords | `crypto.scrypt` (async) or Argon2 via a vetted package | Never a fast hash (`sha256`) for passwords. |
| Graceful shutdown | `SIGTERM` -> `server.close()` + `closeIdleConnections()` + unref'd hard deadline | Docker sends SIGTERM to PID 1; use exec-form `CMD` so Node is PID 1. |
| Fail fast on bad config | Validate env at startup, throw before `listen()` | A process that listens with bad config passes the health check and fails real traffic. |
| Test without a framework | `node --test`, `node:assert/strict`, `mock.timers` | `node --test` runs files in parallel processes by default; use `--test-concurrency=1` for shared resources. |
| Upgrade safely | CI matrix on current and next LTS, `--throw-deprecation` in tests | Native addons need `npm rebuild` unless they use Node-API. |

[Back to top ⤴️](#-table-of-contents)

---

## 🧪 Example Projects

All code examples in this guide and the starter project use **TypeScript written with erasable syntax only**. Node 22.18+ and 24 run those files directly (`node file.ts`), and deleting the annotations leaves plain JavaScript, so the examples double as JavaScript references. TypeScript is the default for new Node.js services, the starter is a typed Express server, and `tsc --noEmit` in CI catches the mistakes that type stripping alone would let through. Module-system examples use `.mjs` and `.cjs` files on purpose, because the file extension is part of what they demonstrate.

| Project | Builds on | What you practise | Stretch goal |
| --- | --- | --- | --- |
| [Starter project](./src) (in this repo: `src/`, `test/`, `examples/`) | [Stage 1](#1-stage-1-runtime-architecture-toolchain--the-event-loop), [Stage 2](#2-stage-2-modules-packages--tooling), [Stage 3](#3-stage-3-async-programming-events-buffers-files--streams) | Express 5 app factory, graceful shutdown, event-loop, stream, worker and deadline sandboxes, `node:test` with `fetch` | Add the line-count endpoint from [3.12](#312-hands-on-exercise) |
| Log analyser CLI | [2.7](#27-built-in-dev-tooling---watch---env-file-parseargs-and-styletext), [3.8](#38-streams-and-backpressure) | `parseArgs`, `readline` over a stream, `for await`, exit codes | Accept `.gz` input by inserting `createGunzip()` when the file name ends in `.gz` |
| Dual-consumer package | [2.3](#23-interop-requireesm-import-of-commonjs-and-createrequire), [2.4](#24-packagejson-type-exports-imports-and-conditions) | One ESM build consumed by `import` and `require(esm)`, `exports`, `imports`, conditions | Publish to a local registry (Verdaccio) and consume it from a CommonJS project |
| Resilient HTTP client | [3.1](#31-from-callbacks-to-promises-to-asyncawait), [3.3](#33-cancellation-and-deadlines-with-abortcontroller) | `fetch` with `AbortSignal.timeout`, retries with jitter, `mapLimit` concurrency | Add a circuit breaker that opens after N consecutive failures |
| Streaming CSV to NDJSON converter | [3.5](#35-buffers-and-binary-data), [3.8](#38-streams-and-backpressure) | Async generator transforms, chunk-boundary handling, backpressure | Handle quoted fields that contain newlines |
| Static file server | [3.7](#37-paths-urls-and-the-operating-system), [M1](#m1-masterclass-streaming-file-server-with-range-requests) | `safeJoin`, `Range` requests, `pipeline` to `res`, caching headers | `ETag` and `If-None-Match` support with `304` responses |
| Job queue worker | [M2](#m2-masterclass-queue-backed-order-gateway) | BullMQ, `AsyncLocalStorage`, graceful shutdown of workers | Dead-letter queue and a retry dashboard |
| Real-time chat gateway | [M3](#m3-masterclass-real-time-websocket-gateway) | `ws` rooms, broadcast, connection cleanup | Horizontal scale-out through Redis pub/sub |

## 📝 Cheat Sheets

### CLI / Commands

```bash
# Versions and runtime info
node --version && npm --version
node -p "process.versions"                       # V8, libuv, OpenSSL, undici ...
node -p "os.availableParallelism()"

# Running code
node app.ts                                      # TypeScript via type stripping (22.18+, 24)
node --watch src/server.ts                       # restart on change
node --env-file=.env app.js                      # load environment variables
node --run test                                  # package.json script without npm
node -e "console.log(1)"  /  node -p "1 + 1"     # evaluate / evaluate and print
node --input-type=module -e "console.log(import.meta.resolve('express'))"

# Testing
node --test                                      # find and run test files
node --test "test/**/*.test.ts"                  # explicit glob (quote it)
node --test --watch                              # re-run on change
node --test --experimental-test-coverage         # coverage report

# Debugging and profiling
node --inspect src/server.ts                     # DevTools on 127.0.0.1:9229
node --inspect-brk app.js                        # pause on first line
node --cpu-prof app.js                           # writes *.cpuprofile on exit
node --heap-prof app.js                          # sampling heap profile
node --heapsnapshot-near-heap-limit=2 app.js     # snapshots before an OOM
node --trace-warnings --trace-deprecation app.js
NODE_DEBUG=http,net node app.js                  # core module debug logs

# npm
npm ci                                           # exact lockfile install
npm ci --omit=dev --ignore-scripts               # production, no install scripts
npm install express@^5 && npm install -D typescript
npm outdated && npm ls <pkg> && npm explain <pkg>
npm audit --omit=dev
npm pkg set engines.node=">=22.18.0"

# Tuning
UV_THREADPOOL_SIZE=16 node app.js
NODE_OPTIONS="--max-old-space-size=1536" node app.js
```

### Core API Quick Reference

| Task | Code |
| --- | --- |
| Read a small file | `await readFile(path, 'utf8')` from `node:fs/promises` |
| Write atomically | `await writeFile(tmp, data); await rename(tmp, path)` |
| List a tree | `await readdir(dir, { recursive: true, withFileTypes: true })` |
| Path relative to this module | `new URL('./data.json', import.meta.url)` |
| Sleep | `await setTimeout(ms)` from `node:timers/promises` |
| Timeout | `AbortSignal.timeout(ms)` |
| Wait for an event | `const [value] = await once(emitter, 'ready')` |
| Stream pipeline | `await pipeline(src, transform, dest)` from `node:stream/promises` |
| Iterable to stream | `Readable.from(generator())` |
| Gzip a stream | `pipeline(src, createGzip(), dest)` |
| Hash | `createHash('sha256').update(data).digest('hex')` |
| Random id | `crypto.randomUUID()` |
| HTTP request | `const res = await fetch(url, { signal }); await res.json()` |
| Minimal HTTP server | `createServer((req, res) => res.end('ok')).listen(8080, '0.0.0.0')` |
| Parse CLI flags | `parseArgs({ options: { port: { type: 'string' } } })` from `node:util` |
| Deep equality | `util.isDeepStrictEqual(a, b)` |
| Child process | `await promisify(execFile)('git', ['status'])` |
| Worker thread | `new Worker(new URL('./worker.js', import.meta.url), { workerData })` |
| Request context | `als.run({ id }, handler)`; `als.getStore()?.id` |
| Test | `test('name', () => assert.equal(a, b))` from `node:test` and `node:assert/strict` |

## 📚 Resources

- [Node.js documentation (API reference)](https://nodejs.org/api/)
- [Node.js Learn section](https://nodejs.org/en/learn)
- [Node.js release schedule and previous releases](https://nodejs.org/en/about/previous-releases)
- [Node.js blog (release announcements)](https://nodejs.org/en/blog)
- [Node.js changelogs](https://github.com/nodejs/node/tree/main/doc/changelogs)
- [Node.js source repository](https://github.com/nodejs/node)
- [Deprecations reference](https://nodejs.org/api/deprecations.html)
- [Modules: packages, `exports` and conditions](https://nodejs.org/api/packages.html)
- [Running TypeScript natively](https://nodejs.org/api/typescript.html)
- [libuv documentation](https://docs.libuv.org/)
- [V8 blog](https://v8.dev/blog)
- [npm documentation](https://docs.npmjs.com/)
- [TypeScript handbook](https://www.typescriptlang.org/docs/)
- [Express 5 documentation](https://expressjs.com/)

## 🤝 Contributing

Contributions are welcome. Read the [contributing guide](https://github.com/TheLearningHubOrg/.github/blob/main/CONTRIBUTING.md) and the [code of conduct](https://github.com/TheLearningHubOrg/.github/blob/main/CODE_OF_CONDUCT.md) before you open a pull request. Commit messages follow [Conventional Commits](https://www.conventionalcommits.org/), which drive the automated changelog.

Before you push, run the same checks as CI:

```bash
npm ci
npm run lint        # tsc --noEmit over src, test and examples
npm test            # node:test, including every script in examples/
npm run build       # tsc -p tsconfig.build.json
docker build -t learn-nodejs . && docker run --rm -p 8080:8080 learn-nodejs
```

## 📄 License

This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.

## 🙏 Acknowledgments

- [Node.js](https://nodejs.org/) and its maintainers
- [V8](https://v8.dev/) and [libuv](https://libuv.org/), the engine and I/O library underneath
- [Express](https://expressjs.com/), the starter project's web framework
- [TypeScript](https://www.typescriptlang.org/), used for type-checking the starter and examples

## 🌐 Connect with me

- [Twitter](https://twitter.com/manthan_ank)
- [LinkedIn](https://www.linkedin.com/in/manthanank)
- [Facebook](https://www.facebook.com/manthanank/)
- [Instagram](https://www.instagram.com/manthan_ank/)
- [YouTube](https://www.youtube.com/@manthanank)
- [GitHub](https://github.com/manthanank)

## ☕ Support

If you like this learning repository and find it useful, consider buying me a coffee or sponsoring me through GitHub Sponsors. Your support helps me keep these guides up to date. Thank you!

[![Buy Me A Coffee](https://img.shields.io/badge/Buy%20Me%20a%20Coffee-donate-FFDD00?logo=buymeacoffee&logoColor=000)](https://www.buymeacoffee.com/manthanank)

[![Sponsor Me](https://img.shields.io/badge/Sponsor-GitHub-green?logo=github)](https://github.com/sponsors/manthanank)

---

Show your support by 🌟 the repository.