ditrixnew/edt-mcp/e2e

EDT-MCP E2E Testing — the complete guide

Audience: an agent (or human) who has never seen this project and is asked to write or run an e2e test for one MCP tool. Read this fully before writing a single line.

1. What you are testing (and what you are NOT)

The EDT-MCP server is a Java plugin that runs inside a live 1C:EDT IDE. It speaks MCP (JSON-RPC 2.0 over Streamable HTTP) on http://127.0.0.1:8765/mcp and exposes ~61 tools that drive a real 1C project (read code, write code, create/delete/rename metadata, debug, profile, render forms, etc.).

You are writing black-box end-to-end tests: a Python test process that is a real MCP client. It sends real tools/call requests to the real, running server and asserts on the real responses and the real on-disk effect.

• You are NOT mocking anything. No fakes, no stubs. The server, the EDT model, and the project files are all real.
• This is a different layer from the Java/JUnit unit tests. JUnit runs headless (no live EDT) and covers pure logic / schema / argument-validation. It physically cannot test what happens against a real loaded project. That is exactly the gap these e2e tests fill.

If a tool "succeeded" in its response but the project did not actually change on disk, that is a bug — and a test that only checks the response would miss it. So we verify the on-disk truth, see §4.

2. Architecture (how the suite is laid out)

tests/e2e/
  SKILL.md            <- this file (the contract + the rules)
  harness.py          <- THE shared base. Import everything from here. Do NOT duplicate its logic.
  run_all.py          <- the orchestrator. Discovers and runs every test, SERIALLY, with reset between.
  tools/
    test_list_projects.py          <- one file per tool. You write/own exactly one of these.
    test_write_module_source.py
    test_create_metadata.py
    ... (one per tool)
    test_coverage_ratchet.py       <- meta: fails if a tools/list tool has no @e2e_test

• `harness.py` owns: the HTTP/JSON-RPC client (+ SSE framing), the git-fixture helpers, and all assertion helpers (including error-quality). You call its functions; you never re-implement them.
• `tools/test_<tool>.py` — one file per MCP tool. It contains the test functions for that one tool: the happy path(s) and the negative matrix. Because each agent owns one file, there are no merge conflicts when many agents work in parallel.
• `run_all.py` — discovers every @e2e_test function, runs them one at a time (see §3 — execution is serial), resets the fixture before each, and emits a --junit-xml report.

Golden rule of parallelism: writing the test files can be parallelized across agents. Running them cannot — every test mutates the same TestConfiguration project and the same git working tree, so the orchestrator runs them serially with a hard reset between each.

3. The git-fixture isolation protocol (THE core mechanism)

TestConfiguration is a git-tracked 1C project committed in this repo (TestConfiguration/). That is our test fixture and our source of on-disk truth. The protocol for every test:

1. Before the test — hard reset, never trust the previous test. The orchestrator calls reset_fixture() = git reset (unstage) + git checkout HEAD -- + git clean -fd on TestConfiguration/. ALL THREE are needed: metadata delete/rename persist to disk and can land STAGED in the index, which a plain git checkout -- (restores from the index) would NOT revert. checkout HEAD -- restores from the commit; reset unstages; clean -fd removes untracked files.
2. Run the tool (one or more call(...)).
3. Assert the effect — the assertion depends on the tool kind (see §4): on-disk git-diff (file-write), MODEL read-back (metadata-write), or empty-diff (read).
4. `kind="write-metadata"` tools get extra isolation (the orchestrator does this, not you): immediately after such a test it runs reset_fixture() then reset_model(). Why: these tools mutate EDT's in-memory BM model, EDT may async-autosave it to disk, and a git reset alone cannot undo an unsaved model change — reset_model() (calls clean_project, which refreshes the model from the just-reset clean disk) discards it before the next test.
5. After the whole run the final git status TestConfiguration/ MUST be empty. A run that leaves the project dirty is a failed run.

Metadata writes now persist to disk (since commit 71a48ba — fix-card metadata-writes-not-persisted-to-disk, see [[project_metadata_write_persistence]]). create_metadata / modify_metadata now forceExport the mutated top object(s) to .mdo after the write; delete_metadata/rename_metadata_object always did (via the refactoring service); write_module_source flushes .bsl synchronously. Since #406 the write tools that extend AbstractMetadataWriteTool also WAIT for that export QUEUE to drain before answering (rename_metadata_object and build_external_objects do not extend it and still lag). That wait is ordered with the export only when the SAME call submitted it. Since #408 the generic delete_metadata path submits too, but only for the CONTAINER of the deleted node (its Configuration.mdo / owner .mdo); the deleted object's OWN file cannot be resubmitted at all (EDT builds a save task by looking the FQN up in the transaction, and a deleted FQN yields none), so poll for that one. Where the call did submit, a drained queue means nothing of that call is still pending - it does NOT prove the bytes are right (EDT logs a per-file write failure and completes the computation anyway), and the wait is skipped where the export state cannot be observed. So still ASSERT the bytes; you just no longer need to wait for them. Consequence: verify a metadata-write BOTH ways — a MODEL READ-BACK (semantic: a read tool sees the change in the model) AND the on-disk structure. For a call that SUBMITTED its own export (create/modify, and the specialized delete branches) assert the disk directly (assert_disk_lacks / assert_disk_path_gone / read_disk) and do it BEFORE any other MCP round trip. Poll instead for a file whose export the call did NOT submit — for the generic delete_metadata path that is the deleted object's own file, while its container's file is asserted immediately (§ the comment in test_delete_metadata.py, and #408). rename_metadata_object / build_external_objects poll for the separate reason that they do not inherit the barrier at all. The orchestrator reverts the on-disk .mdo via reset_fixture (git) and re-syncs the model via reset_model().

4. How to verify "something really changed" (per tool type)

Ground truth depends on the tool: a file-write is proven by the git diff; a metadata-write by the model read-back (semantic) AND the on-disk structure (asserting the specific element - directly when the call submitted its own export, polled when it did not; see §3). Always assert WHAT changed (the structural element), never merely that a diff exists or that the bare name appears somewhere; and never use assert_no_diff() as the only assertion (it must accompany a real content/sentinel check).

5. Negative coverage is MANDATORY (not just happy paths)

For every tool, in addition to the happy path, cover the negative matrix:

• Invalid path / non-existent object — a modulePath / fqn / objectName that does not exist; a non-existent projectName.
• Non-existent lines / ranges — startLine/endLine out of range; an oldSource (searchReplace) that is not in the file.
• Invalid parameter combinations — every XOR branch (both / neither of modulePath vs objectName), every conditional-required (formName when moduleType=FormModule, commandName when moduleType=CommandModule, oldSource when mode=searchReplace), an invalid enum value.
• Missing required parameter.

If a tool has XOR / conditional / enum parameters and you only tested the happy path, the test is incomplete.

5.1 Assert the QUALITY of the error (not just "it failed")

A negative test must assert that the error is a good error: