Table of Contents

Compare two documents

CompareTo gives the same element-by-element comparison as a failing snapshot test, for any two documents and without snapshot files. It suits one-off checks such as before and after a library upgrade, or a customer's file against yours.

PdfUaDocument before = PdfUaDocument.Load("guide-v1.pdf");
PdfUaDocument after = PdfUaDocument.Load("guide-v2.pdf");

// The same options as a snapshot: here, only the tag structure and text matter.
DocumentDiff diff = before.CompareTo(after, options => options.Ignore(PdfSection.Layout, PdfSection.Metadata));

DocumentDiff.Compare(before, after) does the same with the earlier document first, and works for Word documents too, as does DocxDocument.CompareTo. DocumentDiff.CompareSnapshots compares two snapshot texts, such as the contents of two .verified.pdfua.txt files, and DocumentDiff.CompareFiles compares two files the way pdiffy before after does, reading a PDF or Word document, whatever its extension, and taking any other text as a snapshot.

Assert on the changes

A DocumentDiff lists its Changes. Each DocumentChange has:

Property Holds Example
Kind A ChangeKind: Added, Removed, Moved, AttributeChanged, TextChanged or Changed TextChanged
SectionName The section as the snapshot prints it, as in the JSON output; null for text without sections "structure"
PdfSection The PdfSection it's in; null outside PDF snapshots, and for the snapshot's first lines and whitespace-only changes PdfSection.Structure
DocxSection The DocxSection it's in; null outside Word snapshots, and for the snapshot's first lines and whitespace-only changes DocxSection.Body
Path Where it is structure > Document > Table > TR[2] > TD[1]
Page The page, when it's on one 2
Detail What changed "Go [-with-] {+without+} a guide"
AffectsAccessibility Whether it can change what a screen reader gets; null for text, which PDiffy doesn't assess true 
// No element changed place in reading order.
Assert.DoesNotContain(diff.Changes, change => change.Kind == ChangeKind.Moved);

// Any change is fine except one that can affect what a screen reader gets.
Assert.DoesNotContain(diff.Changes, change => change.AffectsAccessibility == true);

// Nothing changed at all.
Assert.True(diff.IsEmpty);
Tip

To fail a test only on accessibility changes, compare with CompareTo and assert on AffectsAccessibility. A snapshot test always fails on any change; the flag only orders and marks the explanation.

New and fixed problems

When both snapshots have a checks section, its changes read as progress or regression rather than as lines added and removed. A finding that only the later snapshot has is a new problem: an Added change in the checks section whose path is the rule and whose detail is the finding's path. A finding that only the earlier snapshot has is fixed: a Removed change. A new problem from an accessibility rule (rather than a house rule) can affect accessibility, so it's listed first:

2 change(s): 1 new problem, 1 fixed
by section: checks 2
1 can affect accessibility, listed first
+ new problem FigureAltText (p4) [accessibility]
    structure > Document > Figure[3]
- fixed TableHeaders (p3)
    structure > Document > Table > TR > TH[2] > scope

A fixed problem still fails the test until the snapshot is accepted, so it can't come back unnoticed. When an upgrade adds rules to the set, the summary says how many, so new problems can be told apart from changes to the document. If you leave the section out with Ignore(PdfSection.Checks), every finding the earlier snapshot recorded is reported as fixed, next to the header change that says the section is now ignored.

Method Gives
Describe(maxChanges) The summary a failing test shows: counts by kind and section, then up to maxChanges changes, accessibility ones first, with layout lines that moved together shown as one shift
ToHtml() The side-by-side HTML report
ToJson() The changes as JSON, for scripts and tools outside .NET 
Console.WriteLine(diff.Describe());
File.WriteAllText("report.html", diff.ToHtml());
File.WriteAllText("changes.json", diff.ToJson());

JSON output

ToJson(), or pdiffy before.pdf after.pdf --json on the command line, writes a small, versioned shape:

{
  "version": 1,
  "changes": [
    {
      "kind": "Changed",
      "section": "structure",
      "path": "structure > Document > P[1]",
      "page": 1,
      "detail": "type: H1 -> P",
      "affectsAccessibility": true,
      "wcagCriteria": ["1.3.1"],
      "needsReview": false
    }
  ]
}
Field Holds
version Goes up only when a field is renamed, removed or changes meaning. New fields can be added without changing it.
kind A ChangeKind name
section The section as the snapshot prints it, such as structure or body; header for the snapshot's first lines, whitespace when only blank lines or line endings differ, and null for text without sections, such as JSON
path Where the change is
page The page, or null when the change isn't on a page
detail What changed
affectsAccessibility Whether it can change what a screen reader gets, or null for text snapshots
needsReview Whether a specialist should read it although it doesn't affect accessibility, such as reworded alt text (which ones), or null for text snapshots
wcagCriteria The WCAG 2.2 success criteria a flagged change could affect, such as 1.1.1. Empty when the change isn't flagged or no criterion clearly covers it (which ones)

Compare a PDF with its remediated version

Remediating a PDF should change only its accessibility layer: tags, alt text, artifacts, tab order and accessible names. RemediationDiff.Compare lists the changes to the content, which a remediation isn't expected to make, separately from the full diff:

PdfUaDocument original = PdfUaDocument.Load("brochure.pdf");
PdfUaDocument remediated = PdfUaDocument.Load("brochure-remediated.pdf");

RemediationDiff diff = RemediationDiff.Compare(original, remediated, options => options.Check(PdfCheckSet.Latest));

// Text, images, paths, page sizes, link targets, comments and form values the remediation changed.
Assert.True(diff.ContentChanges.IsEmpty, diff.ContentChanges.Describe());

// Every change, with the problems the remediation fixed and any it introduced in the checks section.
Console.WriteLine(diff.AllChanges.Describe());
  • Tagging text splits its layout runs, and some tools reorder the content stream to match the reading order, so ContentChanges joins text across tags and compares content in page order.
  • ContentChanges leaves out metadata, bookmarks, fonts and settings, which a remediation often sets. AllChanges has them.
  • An empty ContentChanges only means the content didn't change: it doesn't say the tags are right. The checks section and a specialist's review cover that.
  • From the command line, pdiffy original.pdf remediated.pdf --remediation prints the same two lists (Compare).

Compare a Word document with its PDF

Exporting a Word document to PDF can lose what a screen reader needs. DocumentDiff.CompareExport compares what a screen reader announces in each (its transcript): the title and language, the text in reading order, heading levels, alt text, tables with their header rows, lists and links.

DocxDocument word = DocxDocument.Load("guide.docx");
PdfUaDocument pdf = PdfUaDocument.Load("guide.pdf");

DocumentDiff diff = DocumentDiff.CompareExport(word, pdf);

// ~ text line 2
//     [-heading level 1:-] Trail Guide
Assert.True(diff.IsEmpty, diff.Describe());
  • Layout and fonts aren't compared, since the two formats record them differently.
  • The transcripts are compared line by line, so a change shows as the line it's on, and changes that can affect accessibility aren't flagged as they are when comparing snapshots.
  • From the command line, pdiffy guide.docx guide.pdf compares them the same way (Compare).

See also