Debugging Playbook

Diagnostics: logging, crash reports, and the memory watchdog

When the desktop viewer misbehaves — a panic, a wgpu error, a runaway load — the evidence lands in one of three places, all wired together so a crash report is self-contained.

Logging and the Console panel

Logging is tracing. The subscriber lives in ui/shell/src/runner.rs and fans out to two sinks (ui/shell/src/console_log.rs):

  • a console buffer (INFO and above) shown live in the viewer’s Console panel — click Console at the bottom-left of the window;
  • a trace ring (DEBUG/TRACE from the orbitron* crates) kept only in memory for the crash file.

stderr additionally receives WARN and above by default; RUST_LOG overrides the stderr filter (RUST_LOG=info, RUST_LOG=orbitron=debug, per-module directives). Note that a bundled .app launched by double-click has no visible stderr — read the in-app Console panel, read the crash file, or launch the binary directly from a terminal to see the stream.

Events worth knowing are logged at INFO so they survive in the trace ring and the crash file:

  • the selected GPU adapter (name, backend, device type) at renderer init;
  • every file open (session/load.rs) with its path and size;
  • every background task-segment load (background/task_segment.rs) — a start line with the file size and task kind, and a finish line with elapsed time and success. These are the breadcrumb for load-time hangs and memory spikes.

Crash reports

Any panic runs the hook in ui/shell/src/crash_reporter.rs, which writes a crash-<unix-ts>.log to the platform data directory:

macOS  ~/Library/Application Support/dev.Orbitron.Orbitron/crashes/
Linux  ~/.local/share/orbitron/crashes/

Each report contains, in order:

  1. a header — Orbitron version, OS/arch, and the file that was open;
  2. the panic thread, source location, and message;
  3. a full backtrace (Backtrace::force_capture, so it fills in regardless of RUST_BACKTRACE) — this is what turns a symptom location like mesh/pipeline.rs:239 into the call chain that reached it;
  4. the recent trace ring (DEBUG/TRACE) and console buffer (INFO+) leading up to the crash.

On the next launch check_for_crash loads the most recent report into a banner in the Console panel, then deletes the file — the report is shown once, so copy it from the banner (or read it from the folder above) before dismissing. Help → Open Crash Reports Folder (MenuAction::OpenCrashReportsFolder → crash_reporter::reveal_crash_dir) opens that directory in the OS file manager, creating it if needed.

report_header is factored out so its format is unit-tested; if you extend the header, update that test.

Memory watchdog

ui/shell/src/mem_watchdog.rs samples current process RSS and host memory every three seconds. Notice and Critical bands are relative to installed memory and current headroom. A five-sample window also detects uninterrupted growth; three clear samples rearm the warning after recovery. This makes the same workload warn earlier on an 8 or 16 GiB machine than on a 192 GiB workstation.

Each state change records the active operation, current RSS, observed peak RSS, total memory, available memory, and sustained growth in the trace log. The watchdog wakes the event loop and sends the same state to a desktop toast instead of logging alone. It never terminates the process or replaces the current scene. File and task loads expose a Cancel button because those workers have real cancel handles; jobs without a safe cancellation path do not show a misleading button.

Memory guardrail (capacity gate)

core/services/src/capacity/ is the shared pre-flight gate. project(op: &Operation, snap: &MemorySnapshot, cap: Option<HardCap>) -> Verdict is pure and unit-tested (band logic: Ok <70% / Warn 70–100% / Critical >100% of available RAM; an overridable Block above physical+swap; an enforced Block only for a hard cap). CapacityProbe wraps sysinfo (0.38, pinned for the 1.92 toolchain) — total+swap cached at startup, available re-read per gate. Each frontend calls the gate at its heavy entry points and applies the verdict: the desktop shows a confirm dialog (ui/shell/src/capacity_gate.rs), the CLI warns/prompts (automation/cli’s memory_gate), Python emits warnings.warn / raises MemoryError. Projections are logged (tracing, “memory gate: projection” / “…strain memory”) so a tuning benchmark can pair them with actual peak RSS. The per-format multiplier table is a const in the module; tune it there against the projected-vs-actual log.

GPU (wgpu) errors

orbitron_render::install_gpu_error_logger sets Device::on_uncaptured_error so wgpu validation and out-of-memory errors are logged (target wgpu) instead of the default panic-abort. It is installed on every device Orbitron creates — the main window, the off-screen/headless renderer, and the export window — so a bad draw call leaves a diagnostic line rather than taking down the app. Adapter selection is logged alongside it, so a GPU-specific report names the backend it happened on.