facebook/pyre-check/pysa-integration-tests

Claude Code and agent skill by facebook/pyre-check

Trust tier 3
  • community
Source check pending

Checking that the SKILL.md file is still in the repository.

Use when running, debugging, updating, or creating Pysa end-to-end integration tests. Use when taint analysis tests fail, when expected output files need updating, or when working with .models, .cg, .hofcg, .overrides files under `source/interprocedural_analyses/taint/test/integration`.

Install

npx vskill@latest install facebook/pyre-check/pysa-integration-tests

SKILL.md

The same SKILL.md was published earlier as facebook/pyre-check/pysa-integration-tests.

Pysa Integration Tests

Overview

End-to-end integration tests for the Pysa taint analysis engine. Each test is a .py file under source/interprocedural_analyses/taint/test/integration/. Tests run the full taint analysis pipeline and compare output against expected files.

Running Tests

All commands must be run from the source/ directory.

bash
cd source
# Run ALL tests (parallelized with 16 shards)
OUNIT_SHARDS=16 dune exec interprocedural_analyses/taint/test/integrationTest.exe
# Run a SINGLE test (e.g., format.py)
PYSA_INTEGRATION_TEST=format.py dune exec interprocedural_analyses/taint/test/integrationTest.exe
With Pyrefly Frontend

After running ./facebook/scripts/setup.sh --local, Pyrefly is the default backend — tests use source/pyrefly.exe automatically. To override with a custom binary:

bash
PYREFLY_BINARY=<path-to-binary> PYSA_INTEGRATION_TEST=format.py dune exec interprocedural_analyses/taint/test/integrationTest.exe

Test File Structure

Each test <name>.py may have these companion files:

FileRequiredPurpose
<name>.pyYesPython source code to analyze
<name>.py.pysaNoPysa model file: declares sources, sinks, TITO
<name>.py.configNoTaint configuration: rules, sources, sinks, options
<name>.py.modelsYesExpected output: taint models and issues (JSON)
<name>.py.cgYesExpected output: call graph
<name>.py.hofcgYesExpected output: higher-order call graph
<name>.py.overridesYesExpected output: override graph
<name>.py.pyrefly.modelsNoExpected Pyrefly output: models
<name>.py.pyrefly.cgNoExpected Pyrefly output: call graph
<name>.py.pyrefly.hofcgNoExpected Pyrefly output: higher-order call graph
<name>.py.pyrefly.overridesNoExpected Pyrefly output: overrides

Default models: When no .pysa and no .config file is present, the test runner automatically provides default test models (_test_sink, _test_source, etc.). When either file is present, the test must be self-contained.

Debugging Test Failures

When a test fails because expected output doesn't match:

  1. The test runner prints a diff (may be truncated for large diffs)
  2. It creates .actual files for each mismatched output (e.g., format.py.models.actual)
  3. Use diff to compare expected vs actual:
bash
diff source/interprocedural_analyses/taint/test/integration/format.py.models \
source/interprocedural_analyses/taint/test/integration/format.py.models.actual

If the test fails with type errors or analysis errors (not output mismatches), the issue is in the Python source or model definitions.

Updating Expected Files

If the output changes are expected (e.g., you intentionally changed the analysis):

bash
# Automatically moves all .actual files to their expected counterparts
facebook/scripts/in_path/pysa-update-expected

This replaces each <name>.py.<ext> with the corresponding <name>.py.<ext>.actual.

Always review the diff before updating — run sl diff after updating to verify changes are intentional.

Creating a New Test

  1. Create <name>.py under source/interprocedural_analyses/taint/test/integration/
  2. Optionally create <name>.py.pysa (model file) and/or <name>.py.config (taint config)
  3. Run the test — the runner auto-creates missing expected output files on the first run:
bash
PYSA_INTEGRATION_TEST=<name>.py dune exec interprocedural_analyses/taint/test/integrationTest.exe
  1. Review the generated .models, .cg, .hofcg, .overrides files
  2. Commit all files together

Common Mistakes

  • Wrong env var name: It's PYSA_INTEGRATION_TEST, not PYSA_TEST or similar
  • Forgetting the `.py` extension: Use PYSA_INTEGRATION_TEST=format.py, not format
  • Running from wrong directory: dune exec must be run from source/
  • Guessing OUnit flags: Don't use -- format to filter tests; use the PYSA_INTEGRATION_TEST env var
  • Manually editing expected files: Use pysa-update-expected instead; expected files are @generated
  • Forgetting OUNIT_SHARDS: Running all tests without OUNIT_SHARDS=16 is much slower

Security

  • Security scanPassed, 95 of 100Pattern checks on the skill's files.
  • Model reviewPassed

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

Source

Popularity

Repository stars
7,165
Repository forks
452
7-day trend score
44

Stars and forks are for the whole GitHub repository, not this skill alone. Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/facebook/pyre-check/pysa-integration-tests/badge)](https://verified-skill.com/skills/facebook/pyre-check/pysa-integration-tests)

Something wrong with this skill or its page? Tell us.