Verify Snapshot Test

Overview

Code generated by source generators can span tens to hundreds of lines. Verifying this with individual assertions makes the test code more complex than the generated code itself. Snapshot testing saves the entire generated code to a .verified.txt file and automatically compares it with subsequent execution results. When changes occur, it shows the diff, allowing you to immediately determine whether the change was intentional or accidental. This testing strategy is particularly suited for cases like source generators where the output is deterministic and the overall shape matters.

Learning Objectives

Core Learning Objectives

1. Understanding the Verify library
   • How the snapshot testing framework integrated with xUnit works
2. Managing .verified.txt files
   • The approve workflow and source control strategy
3. Snapshot testing workflow
   • The complete process from first run to change detection and approval

---

What Is Snapshot Testing?

Snapshot testing saves the output result to a file and compares it in subsequent tests.

First Run
=========
Input -> Generate -> Save result (*.verified.txt)

Subsequent Runs
===============
Input -> Generate -> Compare with saved result
                  |
                  +- Match -> Test passes
                  |
                  +- Mismatch -> Test fails
                       |
                       +- *.received.txt generated (for comparison)

---

Verify Library

Installation

<PackageReference Include="Verify.Xunit" Version="28.9.2" />

Basic Usage

using VerifyXunit;

[UsesVerify]
public class MyTests
{
    [Fact]
    public Task Should_Generate_Expected_Output()
    {
        string result = GenerateSomething();

        return Verify(result);
    }
}

Usage in Source Generator Tests

[Fact]
public Task Should_Generate_ObservableClass()
{
    string input = """
        [GenerateObservablePort]
        public class TestAdapter : IObservablePort
        {
            public virtual FinT<IO, int> GetValue() => FinT<IO, int>.Succ(42);
        }
        """;

    string? actual = _sut.Generate(input);

    // Compare generated code with snapshot
    return Verify(actual);
}

---

.verified.txt Files

File Naming Convention

{TestClassName}.{TestMethodName}.verified.txt

Example:
ObservablePortGeneratorTests.ObservablePortGenerator_ShouldGenerate_ObservableClass_WithSingleMethod.verified.txt

File Content

// ObservablePortGeneratorTests.*.verified.txt
//------------------------------------------------------------------------------
// <auto-generated>
//     This code was generated by source generator
//
//     Changes to this file may cause incorrect behavior and will be lost if the code is regenerated.
// </auto-generated>
//------------------------------------------------------------------------------

#nullable enable

using System.Diagnostics;
using System.Diagnostics.Metrics;
using Functorium.Adapters.Observabilities;
using Functorium.Adapters.Observabilities.Naming;
using Functorium.Abstractions.Observabilities;

using LanguageExt;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;

namespace TestNamespace;

public class TestAdapterObservable : TestAdapter
{
    private readonly ActivitySource _activitySource;
    private readonly ILogger<TestAdapterObservable> _logger;
    // ... entire generated code
}

---

Test Workflow

1. Write New Test

[Fact]
public Task Should_Generate_NewFeature()
{
    string input = """
        // New scenario input
        """;

    string? actual = _sut.Generate(input);

    return Verify(actual);
}

2. First Run - Fails

[xUnit.net 00:00:01.23]     Should_Generate_NewFeature [FAIL]

  Verify returned:
  - Received: ...NewFeature.received.txt
  - Verified: ...NewFeature.verified.txt (not found)

3. Review received.txt

# Check generated file
cat Tests/.../Should_Generate_NewFeature.received.txt

4. Approve

# Rename received.txt to verified.txt
mv *.received.txt *.verified.txt

# Or use a tool
dotnet tool run verify.tool accept

5. Subsequent Runs - Pass

[xUnit.net 00:00:01.23]     Should_Generate_NewFeature [PASS]

---

Change Detection

Intentional Changes

Modifying the source generator changes the output.

// Before change
sb.AppendLine("    private readonly ActivitySource _activitySource;");

// After change
sb.AppendLine("    private readonly ActivitySource _newActivitySource;");

Test Result

[xUnit.net 00:00:01.23]     Should_Generate_ObservableClass [FAIL]

  Verify mismatch:
  - Received: _activityContext
  - Verified: _parentContext

Compare with Diff Tool

# Use diff tool
diff *.verified.txt *.received.txt

# Visual Studio Code
code --diff *.verified.txt *.received.txt

Approve Changes

# Approve new version if change is intentional
mv *.received.txt *.verified.txt

---

Source Control Integration

Commit .verified.txt

# verified.txt files must be committed
git add *.verified.txt
git commit -m "test: update verified snapshots"

Ignore .received.txt

.gitignore

*.received.txt
*.received/

Verification in CI

# CI pipeline
steps:
  - run: dotnet test
    # CI fails on snapshot mismatch

---

Practical Test Examples

Single Method Test

[Fact]
public Task ObservablePortGenerator_ShouldGenerate_ObservableClass_WithSingleMethod()
{
    string input = """
        using Functorium.Adapters.SourceGenerators;
        using Functorium.Abstractions.Observabilities;
        using LanguageExt;

        namespace TestNamespace;

        public interface ITestAdapter : IObservablePort
        {
            FinT<IO, int> GetValue();
        }

        [GenerateObservablePort]
        public class TestAdapter : ITestAdapter
        {
            public string RequestCategory => "Test";
            public virtual FinT<IO, int> GetValue() => FinT<IO, int>.Succ(42);
        }
        """;

    string? actual = _sut.Generate(input);

    return Verify(actual);
}

Corresponding .verified.txt File

TestNamespace/TestAdapterObservable.g.cs

// Filename: TestAdapter.Observable.g.cs

public class TestAdapterObservable : TestAdapter
{
    private readonly ActivitySource _activitySource;
    private readonly ILogger<TestAdapterObservable> _logger;
    private readonly Counter<long> _requestCounter;
    private readonly Counter<long> _responseCounter;
    private readonly Histogram<double> _durationHistogram;

    // ... entire generated code
}

---

Test File Structure

Tests/Functorium.Tests.Unit/
+-- AdaptersTests/
    +-- SourceGenerators/
        +-- ObservablePortGeneratorTests.cs
        |
        |   // .verified.txt files (31)
        +-- ObservablePortGeneratorTests.*.WithSingleMethod.verified.txt
        +-- ObservablePortGeneratorTests.*.WithMultipleMethods.verified.txt
        +-- ObservablePortGeneratorTests.*.WithPrimaryConstructor.verified.txt

        +-- ...

---

Advantages and Considerations

Advantages

Advantage	Description
Full output verification	Verifies entire generated code
Change tracking	Manages change history with Git
Refactoring safety	Immediately detects unintended changes
Documentation	verified.txt serves as expected output documentation

Considerations

Consideration	Solution
File count increase	One per test needed
Non-deterministic output	Must ensure deterministic output
Review complexity	Need to review changed files in PRs

---

Best Practices

1. Clear Test Names

// ✅ Good example
public Task Should_Generate_ObservableClass_WithPrimaryConstructor()

// ❌ Bad example
public Task Test1()

2. Test Only One Scenario

// ✅ Good example - test one thing
[Fact]
public Task Should_Generate_Count_ForCollectionParameter() { }

[Fact]
public Task Should_Generate_Length_ForArrayParameter() { }

// ❌ Bad example - mixing multiple things
[Fact]
public Task Should_Generate_CountAndLength() { }

3. Minimize Input

// ✅ Good example - minimal code needed for the test
string input = """
    [GenerateObservablePort]
    public class TestAdapter : IObservablePort
    {
        public virtual FinT<IO, List<int>> GetItems() => ...;
    }
    """;

// ❌ Bad example - includes unnecessary code
string input = """
    // Unnecessary comments
    public interface IOtherInterface { }  // Unrelated to test

    [GenerateObservablePort]
    public class TestAdapter : IObservablePort
    {
        private readonly string _unused;  // Unrelated to test
        public virtual FinT<IO, List<int>> GetItems() => ...;
    }
    """;

---

Summary at a Glance

The Verify snapshot testing workflow consists of three stages: “run, compare, approve.” .verified.txt is the approved expected output committed to Git, and .received.txt is the latest execution result registered in .gitignore. When a source generator is modified, the test fails due to snapshot mismatch, so you review the diff and approve received as verified if the change was intentional.

---

FAQ

Q1: Won’t management become difficult as .verified.txt files grow to dozens?

A: While it is true that file count increases, each file contains the “answer” for a single scenario, serving as documentation. You only need to review changed .verified.txt files in Git, and unintended changes are automatically detected in CI. When test names are written clearly, the scenario can be understood from the filename alone.

Q2: What is the fastest way to check the difference between .received.txt and .verified.txt?

A: The Verify library can be configured to automatically launch a diff tool on test failure. In Visual Studio, the built-in diff viewer opens, and on the CLI, you can compare with diff *.verified.txt *.received.txt or code --diff commands. In Functorium, the Build-VerifyAccept.ps1 script batch-approves all pending snapshots.

Q3: Won’t snapshot tests fail every time if the source generator’s output is non-deterministic?

A: Correct. That is why snapshot testing assumes deterministic output. If non-deterministic elements such as timestamps, GUIDs, or environment variables are included in the generated code, .received.txt differs on every run, causing tests to continuously fail. This is why the deterministic output principle covered in Chapter 12 of Part 2 is an essential prerequisite for snapshot testing.

---

With testing tools in place, we will examine ObservablePortGenerator’s 31 test scenarios by category.

-> 07. Test Scenarios