volcengine/searchcli/vs-search-tuning-specify-policy-direction

Viking Search Tuning for Specified Policy Directions (vs-search-tuning-specify-policy-direction)

When to Use

Use this Skill when the user provides specific queries or a type of query, and explicitly specifies an "optimization policy direction" such as boosting the weight of a product type, increasing exposure for seasonal products, or fixing a class of bad cases.

Difference from global vs-search-tuning:

• This Skill does not modify primary recall parameters.
• This Skill does not create, modify, or derive scenes.
• This Skill validates and recommends strategies only through request-parameter passthrough, within the scope of the 8 business intervention methods.
• It is suitable for small-scale targeted tuning with a clear direction; it is not suitable for global similarity-recall strategy search.

Mode selection:

• If the user provides specific queries or bad cases: use Mode A.
• If the user only provides a policy direction and no specific queries: use Mode B. The Agent synthesizes 50 dataset-related queries.

Version Check

Before starting this skill workflow, run vs version check --json. Continue only when status is up-to-date. If status is update-available, stop and tell the user to update the cloned vs repository, then run git pull --ff-only, bash ./scripts/install.sh, and bash ./scripts/install-skills.sh all --target auto --force (PowerShell: scripts/install.ps1 and scripts/install-skills.ps1). If the status is unknown, stop and report that the CLI version could not be verified.

Preconditions

• A Viking Search application id is available.
• A dataset id and baseline search scene id are either provided by the user or can be resolved from read-only CLI checks.
• The user has provided either concrete queries, a query type, or an explicit business policy direction.
• vs auth status --json succeeds before any runtime search probe.
• The requested change can be evaluated through request-parameter passthrough without writing persistent scene, app, dictionary, or recommend configuration.

Commands

• auth status: verify local Viking authentication before runtime probes.
• llm status / search tune llm-check: verify optional LLM judging or query synthesis readiness.
• app status / app get: inspect app readiness and metadata without modifying it.
• app dataset-config get / app online-config get: read dataset-facing fields and online defaults.
• search scene list / search scene get: find and lock a read-only baseline scene.
• search run: run equivalent baseline and candidate requests with full --data payloads.
• search tune run: inspect installed CLI capability only; do not use it for this workflow unless it exposes safe request-level passthrough.

2. Intervention Boundaries

2.1 The 8 Allowed Intervention Methods

Only tune within the following scope. Any method outside these 8 categories must not be used as an official candidate:

1. Key recall guarantee
2. Personalization
3. Hotness boost
4. Boosting / burying
5. Field-based sorting rules
6. Search diversity
7. Recall-result reranking
8. Synonyms

filter / "filtering the item scope" is not one of the 8 categories above. It can only be used as:

• a probe: diagnose whether a field/object type can explain a bad case;
• an optional guard: use when the user explicitly asks to exclude a class of results, protect no-result cases, or protect against competitors/unsellable items;
• a comparison candidate: measure the upper bound, but do not use it by default as the primary strategy for goals like "show more of this type" or "increase exposure."

2.2 Primary Recall Parameters That Must Not Be Changed

Do not modify:

• user_defined_recall_mode
• dense_weight
• text_weight
• query_keyword_match_percent
• max_retrieved_num
• mode

If the baseline scene needs to be read, read it only. Do not write to it.

2.3 Hard Constraints on How Changes Take Effect

Changes may take effect only through request-parameter passthrough:

• Preferred: vs search run --data
• Do not use by default: vs search tune run

Use tune run only when the installed vs search tune run --help explicitly exposes request-level candidate payload passthrough, and after confirming that it will not tune user_defined_recall_mode/dense_weight/text_weight/query_keyword_match_percent/max_retrieved_num/mode. The currently observed search tune run is a similarity-only / recall-parameter tuning entrypoint and does not fit this Skill's boundary.

The following requirements must be met:

• Do not write persistent search scene configuration.
• Do not write app config.
• Do not write or bind dictionaries.
• Do not write recommend scenes.
• Keep the baseline scene read-only throughout. Do not create or derive a working scene.

Elimination axis (the only one): once a method does not support request-level passthrough, remove it from the candidate pool and mark it in the report as "not passthrough-capable -> skipped."

2.4 Known CLI Pitfalls (Must Be Avoided First)

These are empirically observed blockers and should be written into cli-capabilities.json or the report:

• The actual parameter for vs app get is --id <app_id>, not --application-id.
• vs search dictionary list does not exist in some installed CLI versions; it must not be treated as a blocking step. If it does not exist, mark dictionary/synonym binding as unavailable.
• vs search run --data is a full request override, not a merge patch. The payload must contain at least:

``json {"query":{"text":"<query>"},"page_size":10,"page_number":1} ``

• It is still recommended to pass --dataset-id <dataset_id> to the CLI. If the server requires it, dataset_id may also be included in the payload.
• Verified usable filter DSL:

``json {"op":"must","field":"gender","conds":["女子"]} ` `json { "op":"and", "conds":[ {"op":"must","field":"gender","conds":["女子"]}, {"op":"must","field":"reverse","conds":["服装"]} ] } ``

• If running vs ... directly can read the keychain, but invoking vs inside a Node/shell script produces Missing Viking auth or fetch failed, do not bypass it with lower-level APIs. Instead:
• execute the literal vs search run ... command directly;
• or, if the user has already provided AK/SK, pass --ak/--sk / env explicitly;
• or request elevated permission for the exact script command.
• Some OpenAI-compatible models do not support response_format: {"type":"json_object"}. The LLM judge should use a prompt that says "output JSON only" and parse JSON from the model text with fault tolerance.

3. Library of 8 Intervention Methods (Each Method Is Independent)

Field names must be based on the user's actual application schema, CLI help, and live service probes. Do not invent fields that lack a request-level passthrough entrypoint.

3.1 Key Recall Guarantee

Build an independent recall channel for specific items such as new products, high-engagement items, or promotional items, ensuring they are not missed because of insufficient relevance. The relevant item field must first be checked as "used for filtering / category statistics."

• Example: a new SKU "spring linen dress" launches; when searching "dress", force this SKU into the results.
• Applicable when: the expected result is not recalled at all; new product / cold-start support.
• Request-level passthrough requirement: the corresponding request payload field must be verified on the server; otherwise skip.

3.2 Personalization

Under the premise of maintaining relevance, adjust recall and ranking by user-interest tags such as category and brand dimensions based on behavior data.

• Only include as a candidate when the case carries a real user_id.
• If there is no real user_id, skip it and mark this in the report.

3.3 Hotness Boost

On top of relevance, rerank items by hotness score. More clicks, add-to-cart actions, purchases, comments, favorites, and shares mean higher scores.

• Applicable when: popular items are buried or hotness is not reflected.
• You must probe whether there is a request-level hotness/rerank-with-hot entrypoint. Merely having config inside a scene does not mean it can be passed through in a request.

3.4 Boosting / Burying

Products that match a rule are ranked higher or lower accordingly. If the same product matches multiple rules, the weights are accumulated.

• Applicable when: recall is correct but ranking is low; undesired results occupy the top positions.
• You must prove through a probe that the request-level boost/bury payload is accepted by the server and changes TopN. If it is ignored, mark "request field ignored -> skipped."

3.5 Field-Based Sorting Rules

For results with similar relevance, sort by a specified field value in ascending or descending order. Multiple conditions are applied from top to bottom.

• Applicable when: fields such as newness, popularity, or off-shelf status can be sorted and the user's goal is clear.
• Include as a candidate only when the field exists, its type is suitable for sorting, and it supports request-level sort passthrough.

3.6 Search Diversity

Reduce the probability that similar items cluster together. When there are multiple rules, priority decreases from top to bottom.

• Applicable when: Top N is homogeneous, or one brand/SKU/series dominates the screen.
• You must verify that diversity/shuffle rules can be passed through at request level.

3.7 Recall-Result Reranking

Move results that semantically match the retrieval intent more closely to higher positions. This is suitable for natural-language queries and rich-text items.

• You must verify that request-level rerank configuration can be passed through.
• If there is no real rerank entrypoint, do not claim reranking is enabled. You may only rewrite query.text as an "approximate intent expansion" candidate.

3.8 Synonyms

Unify search terms with different expressions but the same meaning so content is not missed because of different wording.

• Generate real synonym candidates only when cli-capabilities.json confirms that a request-level synonym_overrides entrypoint exists.
• Otherwise, use only query.text rewriting for approximate validation, and mark in the report that this is "not a real synonym effect."
• Do not write or bind dictionaries.

4. Failure Mode -> Candidate Method Mapping (Independent Lookup Table)

The Agent automatically labels failure modes based on baseline Top N, then maps them to candidate methods according to the table below. All candidates must first be filtered through cli-capabilities.json.

5. Evaluation Module (Two Decoupled Modes)

Mode A: User-Provided Queries