ueboxai/uebox/ue-ai-render-from-blockout

Painting a blockout with AI

First: is this an AI image or an engine render?

「渲染」 means two completely different things to the person asking, and the two are not substitutes:

When it is ambiguous, ask. One short question costs nothing; an AI image handed to someone who wanted a shot costs them the time they spend discovering it is unusable.

Say what the difference is when you ask, because most people do not think about it until it is pointed out: an AI image is newly painted. It matches the composition and roughly the layout, and nothing in it corresponds to an asset in the project. It cannot be composited, cannot be re-rendered at another resolution, cannot have a pass pulled out of it, and the next image will not match this one exactly.

If they want a real render

Use sequence_render — it renders a Level Sequence through Movie Render Queue. Do not substitute an AI image, and do not improvise a render script with ue_run_python_script: a half-working MRQ script that reports success is worse than not having one.

MRQ renders sequences, not viewports. If there is no sequence for the shot yet, build one first (sequence_camera_keys creates the sequence, the camera and its cuts; a still is a one-frame range), run sequence_audit, then render. Load the ue-sequencer skill for the steps.

The loop

Frame → capture → generate → look. Four tools, in that order, every time.

ue_focus_viewport  names: ["SM_Blockout_Hall"]
ue_screenshot                     → note the returned path
generate_image     prompt: "...", reference_images: ["<that path>"]

The capture is the composition. The prompt is everything the capture cannot show: materials, lighting, time of day, weather, lens, mood, style.

Frame the shot first

The viewport shows whatever the editor camera happens to be pointed at, so an unframed capture usually renders something nobody asked for. ue_focus_viewport takes names — the labels from the World Outliner — and frames those actors.

Before spending a capture, read what focus returned: aim.on_target: false means the camera is not actually looking at the target, transition_settled: false means it is still flying, and occluded_by means something is in the way. Fix the framing, then capture.

For one part of a large object, pass region: "bottom" | "middle" | "top" instead of trying to compute camera coordinates.

Write the prompt for the result, not the blockout

Describe what the picture should become, and explicitly preserve the camera, layout and silhouettes the user wants kept. Do not describe "grey untextured boxes" as the target appearance. Use the image-generation skill for general prompt shaping, reference roles and output checks.

• ✓ weathered limestone hall, warm afternoon sun through high windows, volumetric dust, wide lens, architectural photography
• ✗ a grey box room with some grey box pillars, make it look nice

Chinese or English is fine. Preserve the user's exact requested text, counts and constraints; do not add creative requirements to an already specific request.

State the lighting in the prompt

The capture's brightness is only a baseline when the scene's exposure is locked — the exposure line in the screenshot result says whether it is. Either way, state the lighting you want in the prompt rather than letting the reference decide it: a greybox is usually lit by whatever the template shipped with. Composition, layout, material colour and shadow direction in the capture are trustworthy regardless.

One image first

Every call spends the user's own API credits, and count multiplies the bill. Generate one, look at it, then decide. Ask before generating a batch.

When an iteration is authorized, pass the image the user liked as a reference, describe only the targeted change, and repeat the camera/layout/identity invariants. Label each reference's role if also supplying the blockout. A seed does not guarantee consistency; the current OpenAI/GPT and Gemini adapters do not send it. Ask via ask_user before spending on an unapproved retry; an explicit request to redo already authorizes that retry.

Look at what came back, then say what you see

The image is returned to you directly. Read it and describe it in one or two concrete sentences: what is in the frame, whether it matches the blockout, what is off. success in a tool result means the call went through, not that the picture is right — never report the render as done on the strength of that field alone.

If the render lost the blockout's layout, the reference did not carry: check that reference_images really got the capture path, and that the capture actually shows the subject.

Camera, layout and object count against the capture are yours to check; whether the picture is good is the user's call — do not grade it for them.

Nothing in the project changes along this loop: ue_focus_viewport moves the user's editor camera (say so — it takes their view), and assets appear only if you import an image. There is no blast radius to check beyond that.

Several angles

Repeat frame → capture → generate per requested angle, preserving the same material, lighting and style description. Where the selected integration accepts multiple references, include the approved image for appearance and the new capture for camera/layout, labeling both roles. Check consistency visually; do not promise that a shared seed locks the design.

Delivering the images

Every generated image is saved into the asset library under AIGC/图片 and its absolute path comes back in the result. To put one into the Unreal project — as a reference board, a texture, or a decal source — hand that path to ue_content_import. Do not go looking for the file with shell commands; the path is already in the result.

Failure handling

• No image model configured — the error names the exact place to fix it (preferences → AI →

model source). Relay it; do not try another model id and do not retry.

• Model rejected the reference — some image models are text-to-image only. Say so and offer

either a text-only generation or a different model; do not silently drop the reference.

• Reference path rejected — the tool needs an absolute path. Use the path from

ue_screenshot verbatim rather than rebuilding it.

• Empty or black capture — the camera is not pointed at the subject, or the actors are

hidden in the editor. That is a framing problem, not a generation problem. Re-frame and capture again before spending another generation.

When there is no scene

generate_image works on its own for plain text-to-image — concept art, textures, mood boards, UI sketches. No engine connection, no capture, no reference. The rest of this skill does not apply.