hermesonehq/hermes-registry/agent-platform-inference

Agent Platform GenAI Inference Skill

This skill provides instructions for authenticating and connecting to Google Cloud Agent Platform to use Generative AI models. It covers both First-Party (Gemini) and Third-Party (OpenMaaS) models.

Safety & Confirmation Tiers (CRITICAL)

Before executing any commands or scripts on behalf of the user, you must adhere to the following safety tiers based on the action requested. (The skill is read-only; other safety tiers are omitted):

1. Tier R: Read-only / Inference (`client.models.generate_content`, `client.chat.completions.create`, `client.completions.create`, `client.embeddings.create`)

• Requires interactive confirmation with 'Yes'/

'No' options before executing model inference on behalf of the user, to prevent unexpected cost or quota consumption. The confirmation prompt must clearly explain the proposed inference execution and its key parameters (e.g., target model ID, SDK choice, input prompt). Natural-language paraphrases without specifying the parameters are NOT sufficient.

• Same-turn restriction: Do not execute the

inference scripts or commands in the same turn as presenting the confirmation prompt. Stop and wait for the user's reply; only execute after explicit 'Yes' / approval.

• Gold Standard Example:

> I will perform model inference with the following parameters. Please > confirm this information before I proceed: > * Model ID: deepseek-ai/deepseek-v3.2-maas > * SDK: OpenAI SDK (via Vertex AI Endpoint) > * Input Prompt: "Explain the concept of quantum computing..." > Do you confirm? [Yes/No]

Phase 0: Environment Setup

CRITICAL: Before running any of the Python sample scripts in the scripts/ directory (e.g., scripts/openmaas_openai_sdk.py), you MUST ensure the environment is correctly initialized by following these steps:

1. Google Cloud Authentication: Authenticate with your Google Cloud

credentials and configure active Application Default Credentials (ADC) for Agent Platform access:

``bash gcloud auth login gcloud auth application-default login ``

1. Enable API (if not already enabled):

``bash gcloud services enable aiplatform.googleapis.com ``

1. Virtual Environment: Create and activate a dedicated local virtual

environment:

``bash python3 -m venv .venv source .venv/bin/activate ``

1. Install Dependencies: Install the required SDKs:

``bash pip install -r scripts/requirements.txt ``

1. Verify Setup (Optional): Run all sample scripts at once to verify the

environment is working end-to-end:

``bash ./scripts/verify_all.sh ``

1. Execution: Advise the user that every time they execute a Python

snippet from this skill, they must ensure this virtual environment is activated first.

<!-- disableFinding(LINE_OVER_80) -->

[!IMPORTANT]

CRITICAL: Model IDs & Availability

* Gemini Models: See [Gemini Models][gemini-models-docs] for valid

Model IDs and Regions.

* OpenMaaS Models: See [Use Open Models on Agent Platform]

(https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/maas/use-open-models)

for Llama, DeepSeek, Qwen, etc.

* Incomplete Lists: The Model IDs listed in this skill are **examples

only** and may be incomplete or outdated.

* Action: Always verify the Model ID and Region using the links above

before generating code.

[gemini-models-docs]: https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/migrate

<!-- enableFinding(LINE_OVER_80) -->

Workflow Decision Tree

1. Model Family Identification: Has the user specified whether they want

to call a Gemini (First-Party) model or an OpenMaaS (Third-Party, e.g. Llama, DeepSeek, Qwen) model?

• No -> Ask the user which model family they want to use. If they

provide a specific model name, infer the family from the name.

• Yes -> Proceed to Step 2.

1. SDK Choice: Which SDK does the user want to use?

• Gemini + GenAI SDK (preferred for Gemini) -> Proceed to

[1. Gemini Models].

• Gemini + legacy Vertex AI SDK -> Proceed to [1. Gemini Models].
• OpenMaaS + OpenAI SDK (preferred for OpenMaaS) -> Proceed to

[2. OpenMaaS Models].

• OpenMaaS + GenAI SDK -> Proceed to [2. OpenMaaS Models].
• Unsure -> Default to the preferred SDK for the chosen family.

1. Troubleshooting: Is the user reporting an error (429 Resource

Exhausted, 400 User Validation, 404 Not Found, etc.)?

• Yes -> Proceed to [3. Troubleshooting & Common Error Codes].
• No -> Proceed with the SDK choice from Step 2.

1. Gemini Models

For Gemini models (e.g., gemini-2.5-pro, gemini-3-flash-preview), the GenAI SDK (google-genai) is the PREFERRED method. The legacy vertexai SDK is still supported but GenAI SDK is recommended for new projects.

[!IMPORTANT]

Preview Models (including Gemini 3.1) are often ONLY available in the

global region. Stable models are available in us-central1 and other

regions.

Choosing the Right SDK

• Gemini Models: GenAI SDK (google-genai) is PREFERRED. Use OpenAI SDK for compatibility, or Legacy SDK (vertexai) if needed.
• OpenMaaS Models: OpenAI SDK is HIGHLY RECOMMENDED. Use GenAI SDK or Legacy SDK if you have specific infrastructure requirements.

Installation

pip install google-genai

Python Example (GenAI SDK - Preferred)

See `scripts/gemini_genai_sdk.py` for the complete code.

Alternative: OpenAI SDK (Chat Completions)

Use the standard OpenAI SDK with the Agent Platform endpoint. This is great for cross-compatibility.

See `scripts/gemini_openai_sdk.py` for the complete code.

Legacy: Agent Platform SDK

The legacy vertexai SDK is still widely used but google-genai is preferred for new Gemini projects.

See `scripts/gemini_vertexai_sdk.py` for the complete code.

Documentation: Google GenAI SDK

Documentation: Agent Platform Gemini Models

2. OpenMaaS Models (Llama, DeepSeek, Qwen, etc.)

For OpenMaaS (Model-as-a-Service) models, the HIGHLY RECOMMENDED approach is to use the standard OpenAI SDK with a specific Vertex AI endpoint.

[!WARNING]

While GenerativeModel can support some OpenMaaS models, it is

discouraged. Use the OpenAI SDK for best compatibility (especially for Chat Completions).

Installation

pip install openai google-auth

Authentication for OpenAI SDK

You MUST use a Google Cloud OAuth access token as the API key for the OpenAI SDK.

import google.auth
from google.auth.transport.requests import Request
def get_gcp_access_token():
    creds, _ = google.auth.default()
    creds.refresh(Request())
    return creds.token

[!NOTE]

Google Cloud access tokens typically expire after 1 hour. The

get_gcp_access_token() function above retrieves a fresh token at the time

it is called.

<!-- disableFinding(LINE_OVER_80) -->

For long-running applications, you implement a refresh mechanism. See Refresh the access token for details.

<!-- enableFinding(LINE_OVER_80) -->

Configuration (Base URL)

<!-- disableFinding(LINE_OVER_80) -->

• Global Endpoint (Recommended for most models requiring global

availability): https://aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/global/endpoints/openapi

• Regional Endpoint:

https://{REGION}-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{REGION}/endpoints/openapi <!-- enableFinding(LINE_OVER_80) -->

Python Example (OpenMaaS - Chat Completions)

See `scripts/openmaas_openai_sdk.py` for the complete code.

[!TIP]

Alternative: Environment Variables

You can set environment variables in your shell instead of updating the code.

```bash

export OPENAI_BASE_URL="https://aiplatform.googleapis.com/v1/projects/YOUR_PROJECT_ID/locations/global/endpoints/openapi"

export OPENAI_API_KEY="$(gcloud auth print-access-token)"

```

Then initialize the client without arguments: client = OpenAI()

Python Example (OpenMaaS - Completions API)

The following models support the legacy Completions API: zai-org/glm-5-maas, moonshotai/kimi-k2-thinking-maas, minimaxai/minimax-m2-maas, deepseek-ai/deepseek-v3.1-maas, and deepseek-ai/deepseek-v3.2-maas.

response = client.completions.create(
    model="deepseek-ai/deepseek-v3.2-maas",
    prompt="Once upon a time",
    max_tokens=100
)
print(response.choices[0].text)

Python Example (OpenMaaS - Embeddings)

# Verify specific Embedding Model ID on Model Garden (e.g., intfloat/multilingual-e5-small)
response = client.embeddings.create(
    model="intfloat/multilingual-e5-large-maas",
    input="The quick brown fox jumps over the lazy dog",
)
print(response.data[0].embedding)

Alternative: GenAI SDK

The google-genai SDK can also access OpenMaaS models via the vertexai backend.

See `scripts/openmaas_genai_sdk.py` for the complete code.

[!IMPORTANT]

Model ID Format: For GenAI SDK with OpenMaaS, you MUST use the full

path: publishers/PUBLISHER/models/MODEL (e.g.,

publishers/zai-org/models/glm-5-maas).

Legacy: Agent Platform SDK (OpenMaaS)

For OpenMaaS, you can also use GenerativeModel (if supported).

See `scripts/openmaas_vertexai_sdk.py` for the complete code.

[!IMPORTANT]

Model ID Format: For Agent Platform SDK with OpenMaaS, you MUST use the

full path: publishers/PUBLISHER/models/MODEL.

Model Reference & Availability

Documentation: Use Open Models on Agent Platform

[!TIP]

Self-Deployment for Control: If you need dedicated hardware

(GPUs/TPUs), guaranteed capacity, or specific regional placement not

offered by MaaS, you can Self-Deploy these models to Agent Platform

Endpoints. Search for the model in Model Garden and click "Deploy" to select

your machine type.

[!IMPORTANT]

Finding Inference Examples: The list above is a starting point. For the

definitive inference snippets (especially for Chat Completions payload

structure):

1. Consult the Use Open Models on Agent Platform

list.

2. Click the link for your specific model (e.g., "DeepSeek-V3") to visit its

Model Garden page.

3. Look for the "Sample Code" or "Use this model" button on the Model

Garden page to get the exact curl or Python code for that specific model

version.

[!NOTE]

This list is INCOMPLETE. See [Use Open Models on Agent Platform]

(https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/maas/use-open-models)

for the full list of supported models.