azukiazusa1/sapper-blog-app/blogpost

{"timestamp":"2026-06-21T02:04:08Z","kind":"mouse.click","app":{"name":"Google Chrome"},"window":{"title":"TimePort | 勤怠管理","url":"http://localhost:5173/"},"mouse":{"button":"left","target":{"role":"AXButton","title":"月次工数"}}}
{"timestamp":"2026-06-21T02:04:14Z","kind":"mouse.click","app":{"name":"Google Chrome"},"window":{"title":"TimePort | 勤怠管理","url":"http://localhost:5173/"},"mouse":{"button":"left","target":{"description":"6/15(月)の工数を入力","role":"AXCheckBox","subrole":"AXToggleButton"}}}
{"timestamp":"2026-06-21T02:04:15Z","kind":"selection.changed","app":{"name":"Google Chrome"},"window":{"title":"TimePort | 勤怠管理","url":"http://localhost:5173/"},"selection":{"selectedItems":[{"role":"AXMenuItem","title":"社内業務"}],"target":{"role":"AXMenu"}}}
{"timestamp":"2026-06-21T02:04:16Z","kind":"selection.changed","app":{"name":"Google Chrome"},"window":{"title":"TimePort | 勤怠管理","url":"http://localhost:5173/"},"selection":{"selectedItems":[{"role":"AXMenuItem","title":"Web刷新"}],"target":{"role":"AXMenu"}}}
{"timestamp":"2026-06-21T02:04:20Z","kind":"keyboard.text_input","app":{"name":"Google Chrome"},"window":{"title":"TimePort | 勤怠管理","url":"http://localhost:5173/"},"keyboard":{"target":{"role":"AXTextArea","title":"作業内容","value":"画面実装"}}}
{"timestamp":"2026-06-21T02:04:24Z","kind":"keyboard.text_input","app":{"name":"Google Chrome"},"window":{"title":"TimePort | 勤怠管理","url":"http://localhost:5173/"},"keyboard":{"target":{"role":"AXIncrementor","title":"時間（0.25時間単位）","value":"5"}}}
{"timestamp":"2026-06-21T02:04:24Z","kind":"mouse.click","app":{"name":"Google Chrome"},"window":{"title":"TimePort | 勤怠管理","url":"http://localhost:5173/"},"mouse":{"button":"left","target":{"role":"AXButton","title":"保存する"}}}
{"timestamp":"2026-06-21T02:04:27Z","kind":"selection.changed","app":{"name":"Google Chrome"},"window":{"title":"TimePort | 勤怠管理","url":"http://localhost:5173/"},"selection":{"selectedItems":[{"role":"AXMenuItem","title":"顧客サポート"}],"target":{"role":"AXMenu"}}}
{"timestamp":"2026-06-21T02:04:32Z","kind":"keyboard.text_input","app":{"name":"Google Chrome"},"window":{"title":"TimePort | 勤怠管理","url":"http://localhost:5173/"},"keyboard":{"target":{"role":"AXTextArea","title":"作業内容","value":"問い合わせ調査"}}}
{"timestamp":"2026-06-21T02:04:38Z","kind":"keyboard.text_input","app":{"name":"Google Chrome"},"window":{"title":"TimePort | 勤怠管理","url":"http://localhost:5173/"},"keyboard":{"target":{"role":"AXIncrementor","title":"時間（0.25時間単位）","value":"3"}}}
{"timestamp":"2026-06-21T02:04:38Z","kind":"mouse.click","app":{"name":"Google Chrome"},"window":{"title":"TimePort | 勤怠管理","url":"http://localhost:5173/"},"mouse":{"button":"left","target":{"role":"AXButton","title":"保存する"}}}

{
  "timestamp": "2026-06-21T02:04:14Z",
  "kind": "mouse.click",
  "app": {
    "name": "Google Chrome"
  },
  "window": {
    "title": "TimePort | 勤怠管理",
    "url": "http://localhost:5173/"
  },
  "mouse": {
    "button": "left",
    "target": {
      "description": "6/15(月)の工数を入力",
      "role": "AXCheckBox",
      "subrole": "AXToggleButton"
    }
  }
}

---
name: record-and-replay
description: Record the user's actions on their Mac with Record & Replay, and turn it into a reusable Codex skill from the captured event stream.
---
 
# Record & Replay
 
Record & Replay lets Codex learn a user-demonstrated macOS workflow and turn it into a reusable skill. Use it when the user asks you to watch them perform a task, record a workflow, or create or refine a skill from their demonstration.
 
## Recording Workflow
 
- Use `event_stream_start` only when the user is ready to begin recording.
- Starting asks the user to confirm before capture begins.
- After `event_stream_start` succeeds, do not sleep, poll, or wait in a loop for the user to finish. End your turn and ask the user to tell you when they are done recording and tell them what the time limit is on recording.
- Use `event_stream_status` only when the user asks for status or returns after recording; do not use it to poll while waiting.
- Use `event_stream_stop` when recording is complete.
- When the user says they are done recording, read the returned `metadataPath` and `eventsPath` from disk with normal filesystem tools and inspect the captured events before responding.
- When the user says they cancelled recording, do not call `event_stream_stop` again or attempt to use the event stream. You may read `session.json` if needed to confirm that its `endReason` is `recording_controls_cancelled`; acknowledge the cancellation without creating or updating a skill.
- If the recording contains enough information to identify a reusable workflow, create or refine a skill for that workflow. Do this by default even if the user did not explicitly ask for a skill, and do not stop after providing a summary, replay plan, runbook, or suggestion to create one.
- If the recording does not contain enough information to identify the reusable workflow or create a useful skill, do not guess. Explain what is unclear and ask the user for the missing information needed to create the skill.
- The MCP server does not expose event-stream contents directly.
 
## Concurrent Recording
 
Record & Replay supports one active recording at a time. If `event_stream_start` reports an active recording, do not restart it. Explain that another recording is already in progress and ask whether the user wants to use that active recording or wait until it is stopped.
 
## Interpreting Events
 
- Treat `events.jsonl` as the primary evidence. `session.json` gives paths and session timing only.
- Each event has app/window attribution when available. Use those fields to understand where the event happened; AX payloads may be full trees or diffs for the relevant window.
- AX diff payloads use compact render syntax with ~, +, and - representing changed, added, and removed elements, respectively.
- Pay special attention to selection events, selected text, focused elements, and mouse & keyboard targets. If the user asks a question or refers to the content they are looking at on-screen, selected/focused/targeted content is often the best clue, though visible surrounding UI can also matter.
- Do not include sensitive information from recorded events in summaries or generated skills. Treat passwords, OTPs, API keys, SSNs/passports, financial account/card numbers, and private personal, medical, legal, or HR details as sensitive; use placeholders or generic descriptions when the workflow shape needs to mention them.
 
## Creating Skills
 
Before creating or refining a skill, read and follow the `skill-creator` skill for guidance on structure, reusable resources, and validation.
 
Create or refine an actual discoverable skill, not only a standalone Markdown runbook or replay-plan draft. Complete the skill-creator workflow, including validation, before reporting that the skill was created.
 
When creating a skill from a recording, treat the recording as evidence of the user's intended outcome, not a requirement to reproduce every UI action. Check whether an available connector or dedicated tool supports the task; prefer it for stable semantic operations such as creating a Google Doc or calendar event. Use Computer Use for unsupported UI interactions, visually dependent verification, or when manipulating the interface is itself the task. A skill may combine connectors and Computer Use. When using Computer Use, name it explicitly, describe stable app/window/control targets and interactions, include verification steps, and avoid coordinate-only replay unless the event stream gives no better target.