junmo-kim/mac-local-vision/macvis

macvis

macvis reads and locates things in images, on-device. Put the binary on your PATH (repo README → Install) and call it. Output is YAML by default; add --format json to parse.

Which command

Rule of thumb: ocr to read everything, find to get one word's pixel to click/assert, document-ocr when the layout (table cells, list items) matters rather than flat lines, ask only when you need interpretation rather than raw text.

Examples

macvis ocr ./receipt.png                          # full text + per-line entries
macvis ocr ./receipt.png --words --format json    # per-word pixel boxes, JSON
macvis find ./screen.png --target "Submit"        # → x,y click center + bounding box
macvis find ./screen.png --target "결제하기"        # non-Latin works (locale-aware)
macvis ocr ./doc.pdf --page 2                      # PDF page (rasterized)
macvis barcode ./ticket.png                        # scan every QR/barcode symbology
macvis qr ./ticket.png                             # scan for a QR code only
macvis classify ./photo.jpg                        # tag against a 1,303-label taxonomy
macvis make-qr "https://example.com" --out ./qr.png  # write a scannable QR PNG
macvis document-bounds ./receipt.jpg                  # find a document's 4 corners
macvis rectify-document ./receipt.jpg --out ./flat.png # flatten a photographed document
macvis document-ocr ./invoice.png                    # title/paragraphs/tables/lists, structured
macvis segment ./photo.jpg --point 640,420 --out ./mask.png # top-left pixel seed → grayscale mask
macvis ask ./receipt.png --prompt "extract the fields" --schema ./receipt-schema.json  # structured JSON answer
macvis sort-faces ./photos --output-dir ./by-person  # cluster a folder of photos by person

Reading the output

• stdout = data, stderr = structured errors (error / reason / hint) — keep them apart.
• Exit codes: 0 ok · 1 ran but no result (e.g. find miss) · 64 bad args ·

70 permanently unavailable · 71 retry later.

• `find`: always check found — it's false (with exit 1) when the target isn't on

screen. x,y is the click center; approximate: true means the box is the whole text line, not word-tight.

• PDF input: output starts with page and page_count. A PDF without --page means page 1 only — read every page with --page 1 … --page <page_count>. An out-of-range page fails with page_out_of_range (exit 64).
• Recognition languages auto-detect from the system locale; override with --lang ko-KR,en-US.
• find filters at --min-confidence 0.3 by default (ocr keeps everything); lower it for blurry/headless renders.
• segment never downloads model assets implicitly. If it returns

segment_unavailable/assets_not_ready, use --download-assets only when the user explicitly wants to install the assets, then retry without changing the point/box seed.

• doctor.segment: "unknown: assets_not_ready" means Vision has not established readiness

in this process, not that the model is necessarily missing. Try a normal segment call first.

• segment reports native mask width/height separately from input image_width/image_height (upright pixels after PDF rasterization). Mask resolution varies by quality. Map input pixels with x * width / image_width, y * height / image_height, or resize the mask to the input raster before overlaying. Foreground is white, background black.

classify scores an image against Vision's 1,303-label taxonomy (identifiers are unlocalized technical names, e.g. outdoor/document/people — not meant for direct UI display). Vision scores every label for every image, so results are filtered by --min-confidence (default 0.1) and capped by --top (default 20) before they come back; label_count: 0 is a valid outcome (not an error), same as barcode's code_count: 0.

Full flags for any command live in macvis <command> --help (the canonical, code-generated reference). To drive it as a tool server instead of the CLI, run macvis mcp — same engine, exposes ocr / find / barcode / qr / classify / make-qr / document-bounds / rectify-document / document-ocr / segment / doctor / ask as MCP tools. On macOS 26, ask and segment return structured availability errors.