PDF snapshots

Snapshot.VerifyPdfAsync is the core of PDiffy: one call prints the PDF's snapshot and compares it with the approved .verified.pdfua.txt file.

await Snapshot.VerifyPdfAsync(pdf);

The first argument is the PDF: a stream (read from its start and left open), a byte[], or a PdfUaDocument already loaded. Everything else is an option, set in configure:

Option	Use it to	Default
`Name(...)`	Name the snapshot, when one test makes several	`{TestFile}.{TestMethod}`
`Variant(...)`	Add to the name, so each row of a theory gets its own snapshot	None
`Directory(...)`	Keep this snapshot in another folder	`Snapshot.Directory`, else the test file's folder
`Ignore`, `ScrubDates` and the rest	Leave out or scrub what the test doesn't care about	Everything is compared

Snapshot.VerifyPdf does the same synchronously, with the same options, for a test that isn't async. VerifyPdfAsync does its work before it returns, so a failure throws at the call even if a void test forgets the await.

A stream is read from its start wherever the generator left it, and its position is put back afterward, so a test can load it, assert on it and snapshot it. A stream that isn't already a MemoryStream, such as a file or a network response, is read into memory first, so a stream that fails partway throws its own exception rather than looking like a damaged document. Pass the loaded document instead to read the PDF only once:

var document = PdfUaDocument.Load(pdf);
Assert.Equal("Trail Guide", document.Metadata.Title);
await Snapshot.VerifyPdfAsync(document);

An empty stream, a Word document passed as a PDF, or a web page (a generator's error page, say) fails with an InvalidDataException that says so.

What a snapshot looks like

A snapshot is plain text: one line per element, in a fixed order, the same on every OS. Here is a shortened one:

# pdfua-snapshot v1
document
  marked=yes lang=en-US display-doc-title=yes pages=3
structure
  Document lang=en-US
    H1 p1 "Trail Table"
    Figure p1 alt="Park logo" images=[3f9c0a1b2d4e5f60]
    Table p2-3 summary="Trail details"
      TR
        TH1 p2 scope=Column "Trail"
        TH2 p2 scope=Column "Length"
      TR
        TD p2 headers=[TH1] "Ridgeline"
        TD p2-3 headers=[TH2] "Cedars"
untagged
  p1 "Stray text"
fonts
  "Arial" TrueType embedded=yes to-unicode=yes glyphs=3f9c0a1b2d4e5f60
layout
  p1 text tagged at=72,720 width=85 font="Arial" size=18 color=gray:0 "Trail Table"
  p1 image tagged at=72,600 size=120x40 pixels=600x200 hash=3f9c0a1b2d4e5f60

The PDF snapshot format reference describes every section and key.

Reading a failure

When the snapshot differs, the test throws a SnapshotException whose message explains the change:

The PDF's snapshot differs from the verified one.

3 change(s): 1 moved, 1 attribute, 1 text
by section: structure 3
2 can affect accessibility, listed first
↕ moved structure > Document > Figure (p1) [accessibility]
    reading position 3 -> 10
~ attribute structure > Document > Table > TR[1] > TH2 (p2) [accessibility]
    scope: Column -> (missing)
~ text structure > Document > Table > TR[2] > TD[2] (p2)
    "Go [-with-] {+without+} a guide"

verified: /home/me/trails/tests/Snapshots/TrailGuideTests.TrailGuide_ShouldMatchApprovedSnapshot_WhenGenerated.verified.pdfua.txt
received: /home/me/trails/tests/Snapshots/TrailGuideTests.TrailGuide_ShouldMatchApprovedSnapshot_WhenGenerated.received.pdfua.txt
report: /home/me/trails/tests/Snapshots/TrailGuideTests.TrailGuide_ShouldMatchApprovedSnapshot_WhenGenerated.received.diff.html
To accept, run:
  dotnet pdiffy accept "/home/me/trails/tests/Snapshots/TrailGuideTests.TrailGuide_ShouldMatchApprovedSnapshot_WhenGenerated.received.pdfua.txt"
(pdiffy accept with a global install), or set PDIFFY_ACCEPT=1 and run the test again, or copy the received file over the verified one.

The counts come first: by kind, then by section, so a reflow that moves 200 layout lines doesn't hide one change to the structure. The message lists up to 20 changes; the report lists them all.
Accessibility changes lead the list, tagged [accessibility]. See Changes that affect accessibility for the rules.
Each change has a symbol, a kind, a path to the element and its page, then the detail.

Symbol	Kind	Example detail
`+`	Added	The new line
`-`	Removed	The old line
`↕`	Moved	`reading position 3 -> 10`, and `parent: Document > P -> Document` when it moved to another parent
`~`	Attribute	`scope: Column -> (missing)`
`~`	Text	`"Go [-with-] {+without+} a guide"`: removed words in `[- -]`, added in `{+ +}`
`~`	Changed	`type: H2 -> H3`, or a changed page range

Elements are matched one to one, so:

An edited paragraph is one change, not a removed line and an added one.
An element that moved in reading order, or to another parent, is reported as moved, along with anything that changed on it.
A heading that became another level or a paragraph is one change: type: H2 -> P.
A one-word value replaced on an otherwise unchanged line, such as a number in a table cell, is reported as changed, not as removed and added. A paragraph replaced by entirely different words is reported as removed and added.
Inserted or removed pages are one change, such as p2 inserted; p2-10 -> p3-11. Pages are lined up by their text, and each element is compared with the page it lines up with, so a new cover page doesn't report every later element.
Within a page, three or more layout lines one after another that all moved the same way, such as everything below an inserted paragraph, are listed as one shift: ~ shifted layout > p1[4] to p1[31] (p1) and 28 lines down 16 pt. Only a move is grouped, so a line whose color, font or text also changed still lists that change. The summary groups them; Changes, the JSON and the report keep every line.

The HTML report

A failing test also writes {name}.received.diff.html next to the received file. It's one self-contained page that opens straight from a build artifact. Open the sample report to see one, built from two versions of a small trail guide. It has:

A checks panel, when the checks section changed: new problems first, then fixed ones, each with its rule, page and path, a link to its line and a link to how to fix it.
The list of changes, accessibility changes first, filterable by kind or to only those that affect accessibility.
Links to each page that changed.
Each section side by side, the verified snapshot on the left and the received one on the right, with edited lines paired and the changed words highlighted. Long runs of unchanged lines fold away.
A map of each page with a layout change, drawn from the layout lines: unchanged things in gray, a dashed outline where something was and a filled box where it is now. Hover over a box to see its line. It's a sketch of positions, not a picture of the page, so it needs no renderer and works from the snapshots alone.
The reading order on those maps, when the report has the PDF as well as its snapshot: a failing VerifyPdf test, DocumentDiff.Compare on two documents, or pdiffy before.pdf after.pdf. A numbered line runs through where each tagged element's text or image starts, in the order of the structure tree, with the previous order in red beside it when both documents are there. Pages whose structure changed get a map too.
The screen reader transcript, when the report has the documents: before and after side by side with the changed words marked, or just the received one for a failing test.

DocumentDiff.ToHtml gives the same page for any comparison.

Naming and folders

A test's snapshot is named after its test file and method. When a test makes several, as a theory does, give each a variant, which is appended to that name (TrailMapTests.TrailMap_ShouldMatchApprovedSnapshot_WhenGeneratedForATrail.ridgeline):

[Theory]
[InlineData("ridgeline")]
[InlineData("lakeshore")]
public async Task TrailMap_ShouldMatchApprovedSnapshot_WhenGeneratedForATrail(string trail)
{
    using Stream pdf = await ReportGenerator.CreateTrailMapAsync(trail);

    await Snapshot.VerifyPdfAsync(pdf, o => o.Variant(trail));
}

Or give a whole name with Name(...). If two calls in one run give the same snapshot different content, the second one fails with a message that the name is taken, instead of overwriting the first one's file. The same content twice is fine, and so is a call after one that failed, so a retried test can pass.

A helper shared by many tests should take the caller's file path and member name and pass them on, so each test keeps its own snapshot name instead of all of them sharing the helper's:

public static Task VerifyTrailMapAsync(string trail, [CallerFilePath] string testFile = "",
    [CallerMemberName] string testName = "") =>
    Snapshot.VerifyPdfAsync(TrailMaps.Create(trail), o => o.Variant(trail), testFile, testName);

Each snapshot has up to three files:

{name}.verified.pdfua.txt is the approved snapshot, written when you accept one. Commit it.
{name}.received.pdfua.txt is written when the test fails. Don't commit it.
{name}.received.diff.html is the report, written when the test fails and a verified file exists. Don't commit it.

To keep every snapshot in one folder, set Snapshot.Directory once (see Getting started). To place one snapshot elsewhere, use Directory(...). For a rule across the whole project, such as a second folder for one group of tests or a naming rule of your own, set Snapshot.Location. It gets each call with its default location and returns the folder and name to use:

Snapshot.Location = call => call.TestFile.Contains("Production")
    ? call.Default with { Folder = Snapshot.ProjectFolder("Snapshots", "Production") }
    : call.Default;

Accepting changes

After reviewing a failure, accept the new snapshot:

pdiffy accept --all replaces every verified file under the current folder with its received one (command line).
PDIFFY_ACCEPT=1 dotnet test makes every failing snapshot pass and overwrite its verified file. It's ignored when CI, TF_BUILD, JENKINS_URL or TEAMCITY_VERSION is set.
Or copy the received file over the verified one.

Damaged files

A page that can't be loaded (for example, one with a damaged font) comes back marked unreadable, and its snapshot line says so, instead of the whole read failing. The same goes for an attachment whose contents can't be decoded, and for XMP metadata that isn't well-formed XML or has a DTD (xmp=unreadable). An unreadable page's line also prints a hash of the page as the file stores it, so a change to the page still fails the test. A file that can't be opened at all throws InvalidDataException.

PDiffy also reads structure elements and bookmarks only 256 levels deep, so a file built to overflow the stack can't crash the test run. An element whose children are deeper gets unread-children=N, and their content counts as untagged. Bookmarks nested deeper, or that loop back on themselves, print unreadable. A Flate-compressed stream that would decompress to more than 256 MB isn't decompressed. If it holds a page's content, the page comes back unreadable; if it's an image, the image is hashed as stored; if it's XMP metadata, the snapshot prints xmp=unreadable. See Limitations for what PDiffy can't guard against.

Table of Contents