MediumConcept

What are streams in Node.js and why should you use them?

Technology:
Node.js
Experience:
Mid-level,
Senior

Quick answer

Streams process data piece by piece as it arrives instead of loading it all into memory, which lets you handle large files and network data efficiently; they come in Readable, Writable, Duplex and Transform types.

Detailed explanation

Reading a 2 GB file with fs.readFile loads all 2 GB into memory and can crash the process. A readable stream emits small chunks, so memory use stays flat. Streams can be chained: read, compress, then write.

Backpressure is the key concept: when the destination is slower than the source, the stream signals the source to pause. Use stream.pipeline() (or pipeline from stream/promises) instead of manual .pipe() chains, because it handles errors and cleans up all streams properly.

Example

javascript
import { pipeline } from "node:stream/promises";
import { createReadStream, createWriteStream } from "node:fs";
import { createGzip } from "node:zlib";

await pipeline(
  createReadStream("access.log"),
  createGzip(),
  createWriteStream("access.log.gz")
);
  • Node.jsEasy

    What is the difference between require and import in Node.js?

    require is the CommonJS module system, which loads modules synchronously at runtime, while import is the ES module standard, which is statically analysed, asynchronous and supports top-level await and tree shaking.

    Junior 路 Mid-level 路 Programming Fundamentals
  • Node.jsEasy

    What is middleware in Express and how does it work?

    Middleware are functions that receive the request, response and a next callback; they run in the order they are registered and can modify the request, end the response or pass control on with next().

    Junior 路 Mid-level 路 REST API
  • Node.jsMedium

    What is the difference between JWT and session-based authentication?

    Session authentication stores the login state on the server and gives the browser a session ID cookie, while JWT authentication gives the client a signed token that contains the user's claims so the server can verify it without storing state.

    Mid-level 路 Senior 路 Authentication
  • Node.jsHard

    How do you secure a Node.js Express API?

    Validate all input, use parameterised queries, add security headers with helmet, rate-limit requests, restrict CORS, hash passwords, keep secrets in environment variables, and keep dependencies updated and audited.

    Mid-level 路 Senior 路 Security