facebook/pyre-check/pysa-integration-tests
Claude Code and agent skill by facebook/pyre-check
- community
Checking that the SKILL.md file is still in the repository.
Use when running, debugging, updating, or creating Pysa end-to-end integration tests. Use when taint analysis tests fail, when expected output files need updating, or when working with .models, .cg, .hofcg, .overrides files under `source/interprocedural_analyses/taint/test/integration`.
Install
npx vskill@latest install facebook/pyre-check/pysa-integration-testsSKILL.md
The same SKILL.md was published earlier as facebook/pyre-check/pysa-integration-tests.
Pysa Integration Tests
Overview
End-to-end integration tests for the Pysa taint analysis engine. Each test is a .py file under source/interprocedural_analyses/taint/test/integration/. Tests run the full taint analysis pipeline and compare output against expected files.
Running Tests
All commands must be run from the source/ directory.
With Pyrefly Frontend
After running ./facebook/scripts/setup.sh --local, Pyrefly is the default backend — tests use source/pyrefly.exe automatically. To override with a custom binary:
Test File Structure
Each test <name>.py may have these companion files:
| File | Required | Purpose | |
|---|---|---|---|
<name>.py | Yes | Python source code to analyze | |
<name>.py.pysa | No | Pysa model file: declares sources, sinks, TITO | |
<name>.py.config | No | Taint configuration: rules, sources, sinks, options | |
<name>.py.models | Yes | Expected output: taint models and issues (JSON) | |
<name>.py.cg | Yes | Expected output: call graph | |
<name>.py.hofcg | Yes | Expected output: higher-order call graph | |
<name>.py.overrides | Yes | Expected output: override graph | |
<name>.py.pyrefly.models | No | Expected Pyrefly output: models | |
<name>.py.pyrefly.cg | No | Expected Pyrefly output: call graph | |
<name>.py.pyrefly.hofcg | No | Expected Pyrefly output: higher-order call graph | |
<name>.py.pyrefly.overrides | No | Expected Pyrefly output: overrides |
Default models: When no .pysa and no .config file is present, the test runner automatically provides default test models (_test_sink, _test_source, etc.). When either file is present, the test must be self-contained.
Debugging Test Failures
When a test fails because expected output doesn't match:
- The test runner prints a diff (may be truncated for large diffs)
- It creates
.actualfiles for each mismatched output (e.g.,format.py.models.actual) - Use
diffto compare expected vs actual:
If the test fails with type errors or analysis errors (not output mismatches), the issue is in the Python source or model definitions.
Updating Expected Files
If the output changes are expected (e.g., you intentionally changed the analysis):
This replaces each <name>.py.<ext> with the corresponding <name>.py.<ext>.actual.
Always review the diff before updating — run sl diff after updating to verify changes are intentional.
Creating a New Test
- Create
<name>.pyundersource/interprocedural_analyses/taint/test/integration/ - Optionally create
<name>.py.pysa(model file) and/or<name>.py.config(taint config) - Run the test — the runner auto-creates missing expected output files on the first run:
- Review the generated
.models,.cg,.hofcg,.overridesfiles - Commit all files together
Common Mistakes
- Wrong env var name: It's
PYSA_INTEGRATION_TEST, notPYSA_TESTor similar - Forgetting the `.py` extension: Use
PYSA_INTEGRATION_TEST=format.py, notformat - Running from wrong directory:
dune execmust be run fromsource/ - Guessing OUnit flags: Don't use
-- formatto filter tests; use thePYSA_INTEGRATION_TESTenv var - Manually editing expected files: Use
pysa-update-expectedinstead; expected files are@generated - Forgetting OUNIT_SHARDS: Running all tests without
OUNIT_SHARDS=16is much slower
Security
- Security scanPassed, 95 of 100Pattern checks on the skill's files.
- Model reviewPassed
Quality evaluation
- Task evaluationNot runNo evaluation yet. Run one in Skill Studio.
Source
- SKILL.md in the repositoryChecking.llms/skills/pysa-integration-tests/SKILL.md
Popularity
- Repository stars
- 7,165
- Repository forks
- 452
- 7-day trend score
- 44
Stars and forks are for the whole GitHub repository, not this skill alone. Not refreshed yet.
Works with
Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.
- Claude Code
- Cursor
- GitHub Copilot
- Windsurf
- Codex
- Gemini CLI
- Cline
- Amp
- Roo Code
- Goose
- Continue
- Kilo Code
- Junie
- Trae
- OpenHands
- Qwen Code
- AI Agent
- Tabnine Chat
- Sourcegraph Cody
- Amazon Q
- Firebase Genkit
- Sweep
- Devin
- Phind
- Mentat
- Claude Desktop
- Aider
- GPT Pilot
- Plandex
- bolt.new
- v0
- Replit Agent
- Anse
- TypingMind
- BetterChatGPT
- LibreChat
- Warp AI
- Pieces
- Fig AI
Embed badge
[](https://verified-skill.com/skills/facebook/pyre-check/pysa-integration-tests)