volcengine/searchcli/vs-search-tuning-partial-case

Viking Search Partial Case Tuning

When to Use

Use this skill when the user provides a small number of concrete bad-case queries, usually 1-50, and asks to "optimize the search results for these queries."

This skill only performs local, highly deterministic, fine-grained operational fixes:

• It does not replace global relevance or recall-parameter tuning.
• It does not change online configuration; it only validates candidates through request-level payloads.
• The final deliverables are a fine-grained operations configuration sheet that can be copied into the console, validated payloads, and a replay script.

Version Check

Before starting this skill workflow, run vs version check --json. Continue only when status is up-to-date. If status is update-available, stop and tell the user to update the cloned vs repository, then run git pull --ff-only, bash ./scripts/install.sh, and bash ./scripts/install-skills.sh all --target auto --force (PowerShell: scripts/install.ps1 and scripts/install-skills.ps1). If the status is unknown, stop and report that the CLI version could not be verified.

Preconditions

• A Viking Search application id is available.
• A dataset id and baseline search scene id are either provided by the user or can be resolved from read-only CLI checks.
• The user has provided 1-50 concrete bad-case queries for the same application.
• vs auth status --json succeeds before any runtime search probe.
• The work can be completed with request-level candidate payloads and offline deliverables only.

Constraints

1. Do not change core recall parameters: user_defined_recall_mode, dense_weight, text_weight, query_keyword_match_percent, or max_retrieved_num.
2. The baseline scene is read-only; do not create, modify, derive, publish, or switch scenes.
3. Do not write app config, dataset config, dictionaries, recommend scenes; do not bind anything, and do not perform any persistent writes.
4. Validation may only take effect through a single-request payload passed to vs search run --data; no side effects may remain after the request finishes.
5. Do not rewrite the user's real query. Query-rewrite ideas may only be used as candidate evidence; they must not be written back to the original query or used as online trigger terms that replace the user's query.
6. Do not use conclusions from local cases to judge the overall quality of the recall strategy.
7. Do not force a search-mode change. Evaluate the currently locked baseline scene as-is; if the scene itself is not UserDefined, do not change it for this skill.
8. In the apply phase, only deliver the configuration sheet and evidence; actual launch must be performed manually in the Viking console based on the configuration sheet.

3. Naming Conventions

• File names and report references should use "query + strategy name"; do not use internal aliases such as C1, case-3, or cand-1.
• File-name slug: keep the first few words of the query, convert spaces to hyphens, and remove symbols that are unsuitable for file names.
• When referring to a plan in the report, use readable names such as filter item scope(category=Light Sports,product_type=SHOE).

Commands

• auth status: verify local Viking authentication before runtime probes.
• llm status: check whether optional LLM-assisted analysis is configured.
• app status / app get: inspect app readiness and metadata without modifying it.
• app online-config get / app dataset-config get: read current online and dataset-facing configuration.
• search scene list / search scene get: find and lock a read-only baseline scene.
• search run: run baseline and request-level candidate probes using full --data payloads.
• search tune run / search tune plan / search tune validate: inspect existing tuning command capability and validate query files when useful.

4. Known CLI Behaviors and Pitfalls

These execution constraints must be internalized. Read this section before writing scripts.

4.1 Command Names Must Follow the Actual CLI

Common read-only commands:

vs auth status --json
vs llm status --json
vs app status --application-id <app_id> --json
vs app get --id <app_id> --json
vs app online-config get --application-id <app_id> --json
vs app dataset-config get --application-id <app_id> --dataset-id <dataset_id> --json
vs search scene list --application-id <app_id> --json
vs search scene get --application-id <app_id> --scene-id <scene_id> --json

Notes:

• vs app get uses --id, not --application-id.
• vs app dataset-config get must include --dataset-id.
• The current CLI may not have vs search dictionary list; if it returns Unknown search subcommand: dictionary, record this as a missing capability and do not block the main flow.
• vs search run --help may not show service flags such as --json, --output, or --data; do not conclude that a flag is unsupported only because help omits it. You must verify support with probes.

4.2 --data Is a Full Request Override

vs search run --data does not "append fields on top of --query / --page-size"; it overrides the request body.

Incorrect example:

vs search run ... --query "white shoes for dresses" --page-size 10 --data '{}'

This causes the request body to omit query.text and pagination parameters.

A correct candidate payload must explicitly include:

{
  "query": { "text": "user's original query" },
  "page_size": 10,
  "page_number": 1,
  "filter": {
    "op": "must",
    "field": "reverse",
    "conds": ["Footwear"]
  }
}

The CLI usually fills in dataset_id, but for auditability, the report must still record the actual application-id, dataset-id, scene-id, and page-size.

4.3 Request-Level Filter DSL

Prioritize validating the following DSL. Do not use unverified eq, string expressions, or {field:value} maps.

Single field:

{
  "op": "must",
  "field": "category",
  "conds": ["Soccer"]
}

Multi-field AND:

{
  "op": "and",
  "conds": [
    { "op": "must", "field": "category", "conds": ["Soccer"] },
    { "op": "must", "field": "product_type", "conds": ["SHOE"] }
  ]
}

Multiple conds on the same field means OR; multiple must clauses placed in and.conds mean AND.

When validating product_id filters, the common ID field in results is _id; when calculating field_match@10, map product_id to result _id to avoid incorrectly scoring it as 0.

4.4 Authentication Pitfalls in Batch Scripts

Running vs search run ... directly may read secure-store successfully, while spawning vs from a Node/Bash batch script in a restricted sandbox may fail with:

Missing Viking auth. Run `vs auth import-env` ...

This does not necessarily mean the user is not logged in. Handle it as follows:

• First verify auth with the direct command vs auth status --json.
• If the direct command works but the batch script fails, request external execution / real terminal permission for the current environment and rerun the batch script.
• Do not attribute this error to the app or query.

4.5 Query File Format

vs search tune validate accepts text or query.text; do not use an arbitrary field name query.

Recommended JSONL:

{"id":"white-shoes-for-dresses","text":"white shoes for dresses"}
{"id":"TF-soccer-shoes","text":"TF soccer shoes"}

5. Fine-Grained Operations Action Library

By default, the trigger condition should be "search term equals the user's original query" for exact matching and low spillover. Only use broader conditions such as contains, search-term type, or search-term length when the business explicitly wants to cover a class of queries.

Candidates must first pass capability confirmation. Fields must come from the dataset config's filterFields or from verified request-level capabilities; do not invent fields.

If an action-library item has no CLI request-level entry point, write "Not evaluated: the CLI does not expose an equivalent request entry point" in the report; do not generate fake payloads.

Workflow

Step 1. Preflight Checks

Minimum requirements:

vs auth status --json
vs app status --application-id <app_id> --json
vs search scene list --application-id <app_id> --json

Confirm that the app is ready, that there is a unique or specified dataset-id, and that an available search scene exists.

Step 2. Lock the Baseline Scene

Priority:

1. A scene explicitly specified by the user.
2. The scene corresponding to a known best global tuning result that has already been applied.
3. The current default scene.

After locking it, treat it as read-only. Record the scene id, name, whether it is the default, and a summary of the main SearchConfig.

Step 3. Research Capabilities and Save Them

First inspect help:

vs search run --help
vs search tune run --help
vs search tune plan --help
vs search tune validate --help

Then collect read-only information:

vs auth status --json
vs llm status --json
vs app status --application-id <app_id> --json
vs app get --id <app_id> --json
vs app online-config get --application-id <app_id> --json
vs app dataset-config get --application-id <app_id> --dataset-id <dataset_id> --json
vs search scene get --application-id <app_id> --scene-id <baseline_scene_id> --json

You must confirm capabilities with probes:

vs search run --application-id <app_id> --scene-id <scene_id> --dataset-id <dataset_id> \
  --query "<q>" --page-size 2 --json --output <session>/probe-output.json
 
vs search run --application-id <app_id> --scene-id <scene_id> --dataset-id <dataset_id> \
  --query "<q>" --page-size 2 \
  --data '{"query":{"text":"<q>"},"page_size":2,"page_number":1}' \
  --json --output <session>/probe-data.json
 
vs search run --application-id <app_id> --scene-id <scene_id> --dataset-id <dataset_id> \
  --query "<q>" --page-size 2 \
  --data '{"query":{"text":"<q>"},"page_size":2,"page_number":1,"filter":{"op":"must","field":"reverse","conds":["Footwear"]}}' \
  --json --output <session>/probe-filter.json

Generate <session>/cli-capabilities.json, containing at least: