microsoft/typescript/compiler-and-fourslash-tests

Writing, Running, and Debugging Compiler Tests and Fourslash Tests

This guide covers the complete testing workflow for the TypeScript repository, including compiler tests (type-checking, emit, diagnostics) and fourslash tests (language server features like completions, hover, go-to-definition).

1. Compiler Tests

Compiler tests validate the TypeScript compiler's behavior: diagnostics, JavaScript emit, source maps, type/symbol baselines, and more. Each test is a .ts or .tsx file that the test runner compiles, then compares output against stored baselines.

1.1 Where Test Files Live

1.2 Writing a New Compiler Test

A compiler test is just a .ts or .tsx file — no Go code needed. Place it in tsc/testdata/tests/cases/compiler/ for regression tests or tsc/testdata/tests/cases/conformance/<subdir>/ for conformance tests.

Simple single-file test

// tsc/testdata/tests/cases/compiler/myNewTest.ts
const x: number = "hello"; // expect type error

Using compiler option directives

Set compiler options with // @option: value comment directives at the top of the file:

// @target: es2020
// @strict: true
// @declaration: true
// @jsx: react
// @noEmit: true
const x: number = 42;

Multi-file test

Use // @filename: directives to define multiple files in one test:

// @target: es2015
// @module: commonjs
// @filename: /src/utils.ts
export function greet(name: string): string {
    return `Hello, ${name}`;
}
// @filename: /src/main.ts
import { greet } from "./utils";
const msg: number = greet("world"); // type error

For readability, make sure to add an extra line between the contents of a file and the next // @filename directive (as we did above).

Generating test variations

Options can specify multiple comma-separated values to generate separate sub-test configurations:

// @target: es2015, esnext
// @module: commonjs, esnext
// @strict: true, false
export const x = 1;

This generates a sub-test for each combination, with names like myTest.ts (target=es2015,module=commonjs,strict=true).

Note: // @lib: is not variant — commas add additional lib files rather than creating separate test configurations:

// @lib: es2020,dom

Symlink tests

Use // @symlink: to create symlinks in the virtual filesystem:

// @symlink: /src -> /node_modules/mylib

Other directives

• // @currentDirectory: /custom/path — Set the working directory
• // @noImplicitReferences — Don't auto-include referenced files

1.3 Running Compiler Tests

Via hereby (recommended)

Always use npx hereby test to run tests. It ensures a clean state by clearing stale baselines before running, so results are always trustworthy. Trust the results — if hereby test passes, the tests pass.

It's generally best to run all tests — the full suite is very quick and will find issues you didn't realize you were introducing:

npx hereby test    # Run ALL tests — recommended, fast, and catches unexpected breakage

If a test fails, the output will include the full test name and package, which you can use to re-run it directly with go test for debugging (see below).

Via Go directly (for print-debugging a single test)

Use go test directly only when you need verbose output for a specific test to debug with print statements. The test output from hereby test will tell you the exact package and test name to use:

go -C ./tsc test ./internal/testrunner/ -run 'TestLocal/myNewTest' -v

The test entry points are:

• TestLocal — runs tests from testdata/tests/cases/ (both compiler/ and conformance/)

What happens during a test run

For each test file, the runner:

1. Parses directives (// @option:, // @filename:, etc.)
2. Generates configurations for each option variation
3. For each configuration, runs these parallel sub-tests:

• error — Verifies diagnostics against .errors.txt baseline
• output — Verifies JavaScript emit against .js baseline
• sourcemap — Verifies source map output
• sourcemap record — Verifies source map record
• union ordering — Validates AST union type ordering
• source file parent pointers — Validates AST structure integrity

1.4 Baseline System

Baselines are the expected output files that test results are compared against.

Baseline file types

Viewing baseline diffs

git diff --diff-filter=AM --no-index ./tsc/testdata/baselines/reference ./tsc/testdata/baselines/local

Accepting baselines

Important: Only accept baselines immediately after a successful npx hereby test run. The hereby test command clears stale baselines before running, so accepting after it guarantees you're only accepting baselines from the current test run. If you accept without running hereby test first, you risk accepting old/stale baselines from previous runs.

npx hereby test              # MUST run this first — clears stale state
npx hereby baseline-accept   # Then accept the baselines

The baseline-accept task:

1. Copies all files from local/ to reference/ (excluding .delete files)
2. Deletes reference files that have corresponding .delete markers in local/

2. Fourslash Tests

Fourslash tests validate language server (LSP) features: completions, hover/quick info, go-to-definition, find references, rename, code fixes, formatting, and more. They're Go test files that set up TypeScript source with position markers, then verify LSP responses.

2.1 Where Test Files Live

New fourslash tests should be hand-written. Do not create or regenerate an upstream-generated fourslash suite.

2.2 Writing a New Fourslash Test

Create a Go test file in tsc/internal/fourslash/tests/. The file uses the fourslash_test package.

Minimal template

package fourslash_test
 
import (
    "testing"
 
    "github.com/microsoft/TypeScript/tsc/internal/fourslash"
    "github.com/microsoft/TypeScript/tsc/internal/testutil"
)
 
func TestMyFeature(t *testing.T) {
    t.Parallel()
    defer testutil.RecoverAndFail(t, "Panic on fourslash test")
    const content = `
var x/*marker1*/ = 42;
`
    f, done := fourslash.NewFourslash(t, nil /*capabilities*/, content)
    defer done()
    f.VerifyQuickInfoAt(t, "marker1", "var x: number", "")
}

Real-world example: Quick Info

func TestBasicQuickInfo(t *testing.T) {
    t.Parallel()
    defer testutil.RecoverAndFail(t, "Panic on fourslash test")
    const content = `
/**
 * Some var
 */
var someVar/*1*/ = 123;
 
/**
 * Other var
 * See {@link someVar}
 */
var otherVar/*2*/ = someVar;
`
    f, done := fourslash.NewFourslash(t, nil /*capabilities*/, content)
    defer done()
    f.VerifyQuickInfoAt(t, "1", "var someVar: number", "Some var")
    f.VerifyQuickInfoAt(t, "2", "var otherVar: number",
        "Other var\nSee [someVar](file:///basicQuickInfo.ts#4,5-4,12)")
}

Real-world example: Editing and Completions

func TestBasicEdit(t *testing.T) {
    t.Parallel()
    defer testutil.RecoverAndFail(t, "Panic on fourslash test")
    const content = `export {};
interface Point {
    x: number;
    y: number;
}
declare const p: Point;
p/*a*/`
    f, done := fourslash.NewFourslash(t, nil /*capabilities*/, content)
    defer done()
    f.GoToMarker(t, "a")
    f.Insert(t, ".")
    f.GoToEOF(t)
    f.VerifyCompletions(t, nil, &fourslash.CompletionsExpectedList{
        IsIncomplete: false,
        ItemDefaults: &fourslash.CompletionsExpectedItemDefaults{
            CommitCharacters: &DefaultCommitCharacters,
        },
        Items: &fourslash.CompletionsExpectedItems{
            Exact: []fourslash.CompletionsExpectedItem{
                &lsproto.CompletionItem{
                    Label:    "x",
                    Kind:     new(lsproto.CompletionItemKindField),
                    SortText: new(string(ls.SortTextLocationPriority)),
                },
                "y",
            },
        },
    })
}

Marker syntax

Markers define cursor positions and text ranges in the test content:

Multi-file tests

Use // @Filename: (capital F) to define multiple files:

const content = `
// @Filename: /src/utils.ts
export function greet(name: string) { return name; }
 
// @Filename: /src/main.ts
import { greet } from "./utils";
greet(/*marker*/"world");
`

Setting compiler options

Embed a tsconfig.json file or use directive comments:

const content = `
// @Filename: /tsconfig.json
{ "compilerOptions": { "strict": true, "target": "es2020" } }
 
// @Filename: /src/test.ts
const x/*1*/ = 42;
`

Content mapper tests

When writing a test for a content-mapped file, include a comment that shows the virtual TS output of the test content mapper implementation, and a description or diagram of the mapping spans.

2.3 Verification Methods (Common API)

The fourslash.FourslashTest type (variable f) provides these verification methods:

Quick Info / Hover

f.VerifyQuickInfoAt(t, "marker", "var x: number", "documentation text")
f.VerifyBaselineHover(t)  // generates baseline file

Completions

f.VerifyCompletions(t, "marker", &fourslash.CompletionsExpectedList{
    IsIncomplete: false,
    ItemDefaults: &fourslash.CompletionsExpectedItemDefaults{
        CommitCharacters: &DefaultCommitCharacters,
        EditRange:        Ignored,
    },
    Items: &fourslash.CompletionsExpectedItems{
        Includes: []fourslash.CompletionsExpectedItem{
            &lsproto.CompletionItem{Label: "myVar"},
        },
        // Or use Exact for exact match:
        // Exact: []fourslash.CompletionsExpectedItem{"x", "y"},
    },
})

Import the test utilities for shared constants:

import . "github.com/microsoft/TypeScript/tsc/internal/fourslash/tests/util"
// Provides: DefaultCommitCharacters, Ignored, CompletionGlobalThisItem, etc.

Navigation

f.VerifyBaselineGoToDefinition(t)           // baseline-based
f.VerifyBaselineGoToTypeDefinition(t)
f.VerifyBaselineGoToImplementation(t)

References and Rename

f.VerifyBaselineFindAllReferences(t)
f.VerifyBaselineRename(t)

Diagnostics

f.VerifyNoErrors(t)
f.VerifyErrorExistsBetweenMarkers(t, "start", "end")
f.VerifyBaselineNonSuggestionDiagnostics(t)

Signature Help

f.VerifyBaselineSignatureHelp(t)
f.VerifyNoSignatureHelp(t)