jocyscom/focuslogger/qa-tester

QA Tester

Core philosophy: one test runner, minimum tools, executable code is the spec.

The fewer different tools a codebase has, the lower the cognitive load on every engineer and AI that touches it. Prefer in-box Microsoft APIs; every extra dependency is a cost, not a feature.

1. First principles

Each principle has a why so you can judge edge cases instead of pattern-matching on the rule.

1. Minimum tool surface. Two tools that do the job beat five that do it elegantly. Why: every extra framework is cognitive debt owed by the next person who opens the repo.
2. In-box first. Prefer APIs that ship with .NET or Windows. Why: a test that runs with dotnet test on a clean checkout is the test that actually gets run.
3. Code-first. A test is the spec. Do not duplicate intent into Gherkin + bindings + helpers. Why: every layer of indirection is another place the next AI has to look.
4. Readable beats clever. A new engineer should understand a test in under 30 seconds.
5. Lightest abstraction that still reads well. Inline → helper → screen object → fixture. Why: page objects are a symptom of test duplication, not a solution.
6. Stable selectors. Roles, accessible names, AutomationId. Never CSS tied to layout internals. Why: a fragile selector is a test that will break for reasons unrelated to what it's checking. If a stable selector doesn't exist, fix the product first.
7. Deterministic waits. Use framework auto-waits or in-box event subscriptions. Why: Thread.Sleep hides real race conditions and taxes everyone else's suite time.
8. Test the lowest layer that gives confidence. Unit → API/integration → UI. Why: UI tests are slow, flaky, and tell you less than the integration test that already covers the same rule.
9. Isolation. No shared mutable state. Each test sets up and tears down its own world. Why: ordered tests and shared fixtures produce heisenbugs whose failure mode depends on run order — impossible to reproduce locally and demoralising for everyone on call.
10. Minimal change. Edit small. Don't rewrite suites unless asked. Why: a "while I'm in here" refactor turns a 5-line fix into a 500-line review and blocks the actual bug shipping.
11. Test project must be simpler than the app. One {Project}.Tests project by default. Mirror the product folder layout 1:1. Split into more projects only when a real constraint (headless CI, massive scale, different TFM) forces it. Why: a test project fractured into more pieces than the app it tests is absurd — every extra csproj duplicates infrastructure, slows the loop, and makes newcomers wonder which project to open.
12. Performance data is a byproduct of functional tests. If a code path already runs in CI under a unit / integration / UI test, attaching a lightweight in-box collector gives you perf coverage equal to your functional coverage at near-zero marginal cost. Why: a parallel "perf suite" that re-implements the same sign-in / order / search scenarios is pure duplication, and it creates coverage gaps wherever the two suites drift. Dedicated load/stress/soak tests exist only for what functional tests genuinely cannot express: concurrency, saturation, sustained throughput, and memory growth over time. See §6a.

2. The two-tool stack

That's the whole stack. `System.Windows.Automation` is in-box (no NuGet), so it doesn't appear in the package list but is the default for any Windows desktop test.

Performance collection (§6a) uses only in-box instrumentation — EventPipe / dotnet-trace, dotnet-gcdump, dotnet-counters, SQL Server Extended Events, Playwright tracing, ETW via wpr.exe — plus one Microsoft-owned load generator (Microsoft.Crank). Collectors are runtime/OS facilities, not "tools", so the two-tool rule is unchanged.

Quick exclusion list (see `references/legacy-bdd.md` for inherited-project support):

Before adding any third tool, write one sentence justifying it. If the sentence is "it would be more convenient", reject it.

3. Layer → test type (the whole decision matrix)

Default rule of thumb: MSTest + System.Windows.Automation covers every .NET desktop scenario; MSTest + Playwright covers every web scenario. If you reach for a third tool, re-read §2.

4. Writing style — code-first, business-readable second

When the user says:

Given a valid user, when they sign in, then they should see the dashboard

Write this:

// @under-test: src/Foo.Web/Pages/SignIn.razor
// @area: auth   @layer: ui-web
[TestClass]
public class SignInTests : PageTest
{
    [TestMethod]
    [TestCategory("auth"), TestCategory("smoke")]
    [Description("Valid user signs in and lands on dashboard")]
    public async Task Valid_user_signs_in_and_lands_on_dashboard()
    {
        // Given: a valid user on the sign-in page
        await Page.GotoAsync("/sign-in");
 
        // When: they sign in with valid credentials
        await Page.GetByLabel("Email").FillAsync("valid.user@example.com");
        await Page.GetByLabel("Password").FillAsync(Environment.GetEnvironmentVariable("TEST_USER_PASSWORD")!);
        await Page.GetByRole(AriaRole.Button, new() { Name = "Sign in" }).ClickAsync();
 
        // Then: the dashboard is visible
        await Expect(Page).ToHaveURLAsync(new Regex("/dashboard$"));
        await Expect(Page.GetByRole(AriaRole.Heading, new() { Name = "Dashboard" })).ToBeVisibleAsync();
    }
}

The Given/When/Then phrasing lives in test name + comments + `[TestCategory]` tags + `[Description]`. No .feature file, no bindings, no regex maintenance, no indirection. A reporter that lifts [Description] + [TestCategory] into Markdown covers every "but stakeholders need to read it" objection without introducing BDD tooling.

5. Discoverability — finding tests from a code path (and vice versa)

For Code ↔ UI mapping (web routes, desktop menu breadcrumbs), see the `solution-patterns` skill. This section covers Code ↔ Test only.

This section is load-bearing. Renaming any convention here breaks every future locate-or-create query.

5.1 Test project naming

Default: one test project per product project — {Project}.Tests, sibling to {Project}. Why: a test project more complex than the app it tests is absurd; the whole point of §1.1 (minimum tool surface) applies to the test tree too. One csproj, one namespace root, one dotnet test command.

Split into multiple projects only when one of these is actually true:

• Headless CI constraint. UI tests need a graphical session; unit and integration tests must pass on a headless build server. Split into {Project}.Tests (headless-safe) and {Project}.Tests.UI (requires a desktop session). Tag the UI csproj so CI can skip it in headless runs, e.g. <IsTestProject>true</IsTestProject> plus a [TestCategory("ui-interactive")] filter.
• Large codebase. Hundreds of product files and thousands of tests where a single csproj's compile + test-discovery time noticeably slows the loop. This is rare in practice.
• Fundamentally different runtime. e.g. product targets net8.0 but one test layer needs net48 for COM interop.
• Dedicated perf/load/soak suite. {Project}.Tests.Perf/ houses Microsoft.Crank scenarios and long-running soak tests. Its runtime constraints are genuinely different: long durations, non-parallel with the functional suite, its own CI job, statistical gating instead of pass/fail per run. See §6a.5. Do not put piggyback perf collectors here — those belong next to the functional tests whose coverage they inherit.

Do not split because "separation of concerns" or "unit vs integration is cleaner" — those are solved by folder structure and [TestCategory] tags, not by csproj boundaries. Extra csprojs bring: duplicated [AssemblyInitialize] code, duplicated TestSandbox/TestHelpers, multiple InternalsVisibleTo grants, separate Directory.Packages.props entries, and separate test hosts that all have to be kept in sync. Pay that cost only when a real constraint forces it.

Naming, whichever variant you pick: {Project}.Tests or {Project}.Tests.UI. One spelling forever. Never {Project}Tests, never Test.{Project}, never {Project}.UnitTests.

5.2 File-to-file mapping (deterministic)