tt-toe/bevy_tui_texture/bevy-tui-texture

bevy_tui_texture — agent usage guide

Versions: bevy 0.20 + ratatui 0.30.2 + wgpu 30 (wgpu must exactly match bevy's pin — bump together; the crate re-exports wgpu and ratatui so downstream code can name matching types without pinning its own copies).

Mental model

• A terminal is a `Tui` component on an ordinary entity. Query it like

anything else (Query<&mut Tui, With<MyMarker>>).

• tui.draw(|frame| ...) is pure CPU — it fills ratatui's buffer and

marks the component dirty only if cells actually changed (byte-identical redraws are free; calling it every frame is the intended pattern).

• The plugin renders dirty terminals entirely in the render world,

directly into the destination Image's GpuImage — in the same frame as the draw, before the camera passes (a structural guarantee from render-graph set ordering). There is no CPU readback, no GPU→GPU copy, and no material touching anywhere — any material type (StandardMaterial, ExtendedMaterial, fully custom) updates automatically with zero registration.

• User systems never need RenderDevice, RenderQueue, Assets<Image>,

Assets<Mesh>, or Assets<StandardMaterial> — not at spawn time, not per frame.

• Per-terminal GPU state (one instance buffer, a globals uniform) is

created lazily on first render and evicted automatically when the Tui's last Handle<Image> drops. The glyph atlas + compositor pipelines are shared across every terminal using the same Fonts (keyed by font identity, not by terminal) and evicted once no live Tui uses that font. No manual cleanup either way.

Choosing a spawn API (top of the ladder first)

1. `TuiRequest` (default choice) — spawn the component, the plugin

materializes the terminal on the next frame:

• TuiRequest::ui(cols, rows, fonts) → bevy_ui terminal (ImageNode +

required Node; your own Node in the spawn tuple wins).

• TuiRequest::world_quad(cols, rows, fonts, height) → 3D quad sized

in world units (height; width follows texture aspect). Face normal is local `+Z` — orient with Transform::from_translation(pos) .with_rotation(Quat::from_rotation_arc(Vec3::Z, camera_pos - pos)). Transform::looking_at is the OPPOSITE convention and shows the back.

• TuiRequest::headless(cols, rows, fonts) → a Tui with no surface —

for AttachTerminal targets (glTF screens) or fully custom meshes/materials.

• Non-default config via .with_config(TerminalConfig { ... }).

1. `AttachTerminal { terminal, material }` — put a headless Tui's

content on an existing mesh entity (e.g. a glTF primitive). The system re-claims automatically until the async glTF loader stops overwriting the material, then goes idle (no archetype churn once settled). AttachMaterial::standard(AlphaMode::Opaque) or ::custom(|image| MyMaterial { ... }). For a self-illuminating screen, route image through the material's EMISSIVE channel (black base_color, white emissive) so content stays visible regardless of scene lighting — see claim_object2_screen in examples/retro_crt.rs.

1. **`TerminalTexture::create(cols, rows, fonts, programmatic_glyphs,

transparent_reset_bg, initial_fill, &mut images) + Tui::from_texture_state(...)** — low-level escape hatch only (see examples/tui_component.rs, examples/shader_mesh.rs`).

For custom render passes / shaders, tui.image_handle() exposes the destination Handle<Image>; look up its GpuImage in the render world and use the re-exported wgpu types against it.

Core recipes

Spawn (no render resources in the signature — this is the whole point):

fn setup(mut commands: Commands) {
    let fonts = Arc::new(Fonts::new(
        Font::new(include_bytes!("assets/fonts/Mplus1Code-Regular.ttf")).unwrap(),
        16, // cell height in px
    ));
    commands.spawn((TuiRequest::ui(80, 25, fonts), Node::default(), MyTerminal));
}

Per-frame draw (register in TerminalSystemSet::Render or UserUpdate):

fn draw(mut screens: Query<&mut Tui, With<MyTerminal>>) {
    let Ok(mut term) = screens.single_mut() else { return }; // REQUIRED:
    // TuiRequest materializes one frame after spawn — tolerate absence.
    term.draw(|frame| { /* ratatui widgets */ });
}

Runtime resize — tui.request_resize(cols, rows) (no GPU work at call site; Image is recreated at the same handle, TerminalDimensions and world-quad mesh aspect update automatically; same-size requests are no-ops). No auto-fit helper exists by design: convert InputEvent::Resize { pixels } via fonts.min_width_px() / fonts.height_px() — full recipe in examples/resize.rs.

Transparency (HUD-style see-through screen):

TuiRequest::world_quad(28, 12, fonts, 3.0).with_config(TerminalConfig {
    transparent_reset_bg: true,          // Color::Reset cells → alpha 0
    alpha_mode: AlphaMode::Blend,        // 3d feature; WorldQuad material
    ..default()
})

Only ratatui::style::Color::Reset backgrounds (ratatui's default) become transparent; explicit background colors stay opaque. initial_fill: [u8; 4] sets the pre-first-draw clear color (default opaque black); initial_draw: Some(Box::new(|frame| ...)) draws real content before the first presented frame (closure must be + Send + Sync).

Fonts:

• Native embedded: Font::new(include_bytes!(...)) (needs 'static).
• Runtime bytes: Font::from_vec(vec) — never Box::leak for this.
• Async/Wasm-safe: `TuiFontSource::Asset { handle:

asset_server.load("fonts/x.ttf"), size_px: 16 } — the request stays pending until the .ttf loads, then materializes. (Named TuiFontSource, not FontSource — bevy's prelude already has a FontSource` and glob imports would collide.)

• CJK/fallback: fonts.add_regular_fonts([...]) etc.; the

bold_italic_fonts feature enables real bold/italic font slots (add_bold_fonts/add_italic_fonts/add_bold_italic_fonts), otherwise bold/italic are faked from regular.

Nerd Fonts icons work with no feature flag: register a Nerd Fonts font (a patched "Nerd Font Mono" as primary, or the symbols font via add_regular_fonts). A symbol (Private Use Area, or an ambiguous-width char its font draws wider than a cell) is fitted by Nerd Fonts' own scaling rules. Follow an icon with a space to get the full-size two-cell icon: "\u{f07b} src". The space keeps its own cell and background; a symbol right after another symbol stays in one cell, so runs keep their alignment (the rule Ghostty uses). Fonts::set_fit_symbols(false) turns this off, before the Fonts is shared. See examples/nerd_font_icons.rs.

Color emoji need the opt-in emoji feature and a font you supply (none ships with the crate):

let mut fonts = Fonts::new(Font::new(text_font_bytes).unwrap(), 16);
// COLRv0, COLRv1, CBDT or sbix; not OT-SVG. Before the `Fonts` is shared.
fonts.add_emoji_fonts([Font::from_vec(emoji_font_bytes).unwrap()]);
let fonts = Arc::new(fonts);

• Draw emoji as ordinary text: ZWJ sequences, flags, keycaps and skin tones

take the two cells ratatui reserves. U+FE0E selects the text form.

• TuiFontSource::Asset builds a single-font Fonts, so it cannot carry an

emoji font. Pass a built Arc<Fonts> instead, on Wasm from include_bytes! or bytes you fetched.

• Without the feature, register a monochrome emoji font with add_fonts: a

color font there gives a solid silhouette (COLR) or a blank cell (PNG).

• Fonts added here never change the cell size. Combining marks on an emoji come

from a text font, so that font must have them.

• A multi-char symbol set directly on a cell is as wide as the whole string,

clamped to two cells.

See examples/emoji.rs (--features emoji).

Styling — every ratatui Modifier renders; use plain ratatui Styles:

• DIM = lower fg alpha (TerminalConfig::dim_alpha, default 128);

HIDDEN = blank cell; UNDERLINED / CROSSED_OUT = bars in the fg color.

• Opt-in features: blink (SLOW_BLINK / RAPID_BLINK, else ignored) and

underline_color (Style::underline_color, else the fg color).

• The cursor from frame.set_cursor_position(...) is drawn as

TerminalConfig::cursor_style (Block default, HollowBlock, Underline, Bar, Hidden).

• TerminalConfig::palette: TerminalPalette::default() keeps historic

colors (DarkGray brighter than Gray), xterm() is standard, .with_ansi16([...]) themes.

• Runtime changes (Tui::set_dim_alpha / set_palette /

set_cursor_style) redraw by themselves; no draw needed.

• Box drawing (all weights, so ratatui's thick/dashed/merged borders),

blocks, Braille, Powerline, sextants/octants (Marker::Sextant / Octant) are drawn procedurally; scrollbar/gauge symbols fall back to procedural glyphs only when no font has them.

Input, focus, and hit testing

Read MessageReader<TerminalEvent> (bevy messages, not EventReader); event.target is always the Tui entity, even for attached surfaces. The payload (event.input: InputEvent) mirrors crossterm::event::Event, not a crate-specific enum, so ratatui-ecosystem input vocabulary (tui-textarea, bevy_ratatui, etc.) maps onto it directly. Variants: Key(KeyEvent { code, modifiers, kind }), Mouse(MouseEvent { kind, column, row, modifiers }), Paste(String), FocusGained, FocusLost, Resize { pixels }. KeyCode here is this crate's crossterm-shaped mirror (bevy_tui_texture::input::KeyCode), NOT bevy::prelude::KeyCode (the physical key) — the prelude deliberately omits it (glob-import collision), so import it explicitly: use bevy_tui_texture::input::KeyCode;.

• `(column, row)` on `MouseEvent` — grid cells, x-then-y.
• `MouseEventKind`: Down(button)/Up(button) (was

MousePress/MouseRelease), Moved (no button held) or Drag(button) (button held while moving — was a single MouseMove for both), ScrollUp/ScrollDown/ScrollLeft/ScrollRight.

• `KeyEventKind`: Press/Repeat/Release — filter

k.kind != KeyEventKind::Release for typical "act on keydown" handling. KeyCode::Char(c) already carries the shifted/layout-resolved character (winit resolves it) — there is no separate CharInput.

• Keyboard needs focus: only the focused terminal receives Key

events. Clicking a terminal focuses it (TerminalInputConfig::focus_button, default left); Tab cycles focus when auto_focus is on. Mouse events need no focus.

• Touch works with no extra code: CursorPosition (the crate's

tracked resource) falls back to the first active touch's position (and the just-released touch's last position on the release frame), and a tap emits MouseEventKind::Down/Up(MouseButton::Left). In your own systems, read Res<CursorPosition> instead of Window::cursor_position() — the latter is None on touch devices (see update_camera_rotation in examples/retro_crt.rs).

• Plugin constructors: TerminalPlugin::default() (all input),

::new(TerminalInputConfig { keyboard_enabled, mouse_enabled, auto_focus, focus_button }), ::without_keyboard(), ::without_mouse(), ::display_only().

• Multi-camera picking works out of the box: rays are built per

active camera via Camera::viewport_to_world (correct for every projection/ScalingMode), prioritized by descending Camera::order, then hit distance.

• **When a 2D UI terminal and a 3D mesh terminal overlap in screen

space, the UI terminal wins** — bevy_ui renders as an overlay on top of the 3D view, so the pick order matches what the user sees.

• `crossterm-compat` feature (native-only): InputEvent::to_crossterm/

from_crossterm for lossy round-trips with real crossterm::event::Event — e.g. feeding a crossterm-shaped widget crate or bridging from a bevy_ratatui adapter. Not available on wasm32.