How do I screenshot every page of a website?

Q: How do I screenshot every page of a website?

Get the page list from the site's XML sitemaps first, filter it to the pages you want, then send those URLs to a screenshot API. On nodejs.org the list was 1,733 URLs in 3 seconds for $0.87, and capturing five of them took 7.7 seconds.

In two steps, because they are two different jobs. First get the list of pages — from the site's sitemaps, which is seconds of work and costs almost nothing. Then screenshot the list, which is browser time and is the part you pay for.

Doing it in one step is what makes people crawl a whole site to take pictures of it. Splitting it lets you look at the list first, filter it down to the pages you actually want, and only then spend anything.

Step one: the list

Input and result — 1,733 URLs from nodejs.org in 3.0 seconds

{
  "startUrls": ["https://nodejs.org"],
  "maxUrls": 2000,
  "flatten": true
}

→ 1733 URL(s) from 1 entry, 1733 charged.

Step two: the captures

Take the url field of those rows and pass them straight in — the output shape of the first tool is already the input shape of the second. Here are the first five:

Input

{
  "urls": [
    "https://nodejs.org/learn/command-line/accept-input-from-the-command-line-in-nodejs",
    "https://nodejs.org/learn/command-line/how-to-read-environment-variables-from-nodejs",
    "https://nodejs.org/learn/command-line/how-to-use-the-nodejs-repl",
    "https://nodejs.org/learn/command-line/output-to-the-command-line-using-nodejs",
    "https://nodejs.org/learn/command-line/run-nodejs-scripts-from-the-command-line"
  ],
  "device": "desktop",
  "fullPage": false,
  "format": "jpeg",
  "quality": 85,
  "concurrency": 4,
  "dismissCookieBanners": "hide",
  "blockAds": true
}

Output — five rows in 7.7 seconds, in the order they came back. The URLs share the prefix https://nodejs.org/learn/command-line/ and are shortened to it here

{ "url": ".../how-to-read-environment-variables-from-nodejs", "ok": true, "statusCode": 200,
  "title": "How to read environment variables from Node.js | Node.js Learn",
  "width": 1920, "height": 1080, "bytes": 191347, "durationMs": 1945 }

{ "url": ".../accept-input-from-the-command-line-in-nodejs", "ok": true, "statusCode": 200,
  "title": "Accept input from the command line in Node.js | Node.js Learn",
  "width": 1920, "height": 1080, "bytes": 171408, "durationMs": 1986 }

{ "url": ".../output-to-the-command-line-using-nodejs", "ok": true, "statusCode": 200,
  "title": "Output to the command line using Node.js | Node.js Learn",
  "width": 1920, "height": 1080, "bytes": 169346, "durationMs": 1943 }

{ "url": ".../how-to-use-the-nodejs-repl", "ok": true, "statusCode": 200,
  "title": "How to use the Node.js REPL | Node.js Learn",
  "width": 1920, "height": 1080, "bytes": 187091, "durationMs": 2070 }

{ "url": ".../run-nodejs-scripts-from-the-command-line", "ok": true, "statusCode": 200,
  "title": "Run Node.js scripts from the command line | Node.js Learn",
  "width": 1920, "height": 1080, "bytes": 209988, "durationMs": 1466 }

What the whole site would cost

Worth doing the arithmetic before you start the run, not after. For nodejs.org:

	Rows	Price each	Total
The URL list	1,733	$0.0005	$0.87
A screenshot of every one	1,733	$0.01	$17.33
Only the pages under `/learn/`	87	$0.01	$0.87

The list is cheap, the pictures are not — which is the whole argument for filtering between the two steps. Adding "includeRegex": "/learn/" to the first run returns only those pages, and the ones it filters out are never charged either. If you only want to know how big the job is, a report-only run counts the pages for free.

Two practical notes

Viewport instead of full page, for a lot of URLs. fullPage: true scrolls the whole page first, which is slower per capture; for thumbnails or a visual index of a site, fullPage: false with JPEG is several times cheaper in time.
A URL that fails does not sink the run. It comes back as its own row with ok: false and the reason, is not charged, and the other pages carry on.

The tool that does it

Website Screenshot & PDF Generator API on Apify Store.

$0.01 per successful capture. No start fee, no subscription. A URL that fails is not charged.