What are streams in Node.js and why should you use them?
- Technology:
- Node.js
Quick answer
Streams process data piece by piece as it arrives instead of loading it all into memory, which lets you handle large files and network data efficiently; they come in Readable, Writable, Duplex and Transform types.
Detailed explanation
Reading a 2 GB file with fs.readFile loads all 2 GB into memory and can crash the process. A readable stream emits small chunks, so memory use stays flat. Streams can be chained: read, compress, then write.
Backpressure is the key concept: when the destination is slower than the source, the stream signals the source to pause. Use stream.pipeline() (or pipeline from stream/promises) instead of manual .pipe() chains, because it handles errors and cleans up all streams properly.
Example
import { pipeline } from "node:stream/promises";
import { createReadStream, createWriteStream } from "node:fs";
import { createGzip } from "node:zlib";
await pipeline(
createReadStream("access.log"),
createGzip(),
createWriteStream("access.log.gz")
);