Bundles (.orbpack)
6. Working with files on an HPC system
Quantum-chemistry output is often far larger than the part you want to look at. A multi-orbital cube stores every orbital interleaved at every grid point, so a file can be gigabytes while the one orbital you want to see is a few megabytes. Copying the whole thing to a laptop is slow, and parsing it locally can require more memory than the laptop has. Orbitron has no default file-size refusal. It estimates memory from the source type and current free memory, then warns when the operation is tight. You can set an explicit --max-file-size or --max-memory limit when you need a hard boundary.
A bundle solves this by moving the parsing to where the data already is. You run the CLI on the cluster, it produces a single small .orbpack file, and you copy that. There is nothing to install on the remote beyond the orbitron binary itself, no daemon, no port, and no connection to keep alive.
cluster laptop
─────── ──────
run.out ──▶ orbitron canonical export
│
run.orbpack ──── scp ────▶ run.orbpack
│
orbitron info / File ▸ Open6.1 The loop: inspect, then export
Estimate the source-open cost before starting the parser:
orbitron resources water.out
orbitron resources water.out --max-memory 34359738368 --jsonresources reads metadata and at most the first 64 KiB. It reports the source projection, bounded count hints, current machine headroom, and whether the planned open fits the configured policy. It does not validate the whole source or prove that parsing will succeed. A refusal in the report still returns exit 0, so a script should branch on verdict.band or verdict.enforced rather than the process status alone.
Then ask what the file contains before you export it. --dry-run writes nothing and validates the requested sections and dataset index, but it does parse the source after its bounded preflight:
orbitron canonical export water.out --dry-runDry run: water.out
would write: water.orbpack
bounded preflight: nwchem, 3 atoms (observed in the first 64.0 KB)
sections available: structure (3 atoms), trajectory (3 frames), frequency (9 modes),
thermochemistry, electronic structure
(nothing written)For a cube it also reports how many orbitals are inside, which is what you need before asking for one:
orbitron canonical export orbitals.cube --mo 31 --dry-run volumetric datasets available: 90
requested: dataset 31 (valid)Ask for one that does not exist and it fails without writing a bundle:
Error: requested dataset 99 but orbitals.cube has 90Then export:
orbitron canonical export water.out -o water.orbpackBundle: water.orbpack (6.6 KB)
transfer: 245730 source bytes -> 6758 bundle bytes (97.2% reduction)
sections: structure (3 atoms), trajectory (3 frames), frequency (9 modes),
thermochemistry, electronic structure
attachments: 3The transfer line uses exact filesystem byte counts. For a computed orbital it compares the bundle with the wavefunction you selected, not with Orbitron’s temporary CUBE. Directory inputs have no single honest source-byte count, so they report that the source count is unavailable. The report still gives the exact bundle bytes.
6.2 What goes in by default
Everything cheap, nothing expensive. Geometry, trajectory, frequencies, thermochemistry and electronic structure are included automatically — they are kilobytes to a few megabytes. Volumetric grids and the original source file are only ever present because you asked for them.
| Flag | Effect |
|---|---|
| (none) | Structure, trajectory, frequencies, thermochemistry, electronic structure |
--mo N |
Add orbital N (0-based): selected from a cube, or computed from a wavefunction |
--grid-spacing, --grid-padding |
Override the computed grid’s resolution and extent (Å) |
--movecs FILE |
MO coefficients for --mo on an NWChem .out (defaults to the sibling .movecs) |
--no-infer-bonds |
Keep the source’s connection table unchanged, including a missing table |
--with-source |
Embed the original file, so the bundle can be re-parsed later |
--scrub |
Strip absolute paths from provenance before sharing |
--dry-run |
Report what would happen and write nothing |
--json |
Emit a versioned per-source record for scripts and batch reports |
By default, export computes covalent bonds when a source such as XYZ declares no connection table. The bundle records the resulting table as complete, so the desktop, CLI, Python, and browser readers use those stored bonds instead of running distance inference again. --no-infer-bonds is for a faithful record of the source; a reader may still infer connectivity later when it needs bonds.
The source-light default also applies to Orbitron’s canonical cache. The cache keeps the raw source’s hash and path reference but does not make another copy of the file. A later --with-source export from a cache hit reads the file at the currently opened path and embeds those exact bytes on demand.
6.3 Worked examples
A frequency run. You want the vibrational modes from an NWChem job:
# on the cluster
orbitron canonical export cu-freq.out -o cu-freq.orbpack
# Bundle: cu-freq.orbpack (99.1 KB)
# sections: structure (13 atoms), trajectory (68 frames), frequency (39 modes), ...
# on your laptop
scp cluster:~/runs/cu-freq.orbpack .
orbitron info cu-freq.orbpackThat is 22.8 MB of NWChem output reduced to 99 KB, with the trajectory and all 39 modes intact.
One orbital from a large cube. A 90-orbital cube is 613 MB; orbital 31 is not:
orbitron canonical export orbitals.cube --mo 31 -o homo.orbpackAn orbital from a wavefunction. An FCHK or an NWChem run carries the wavefunction but no grid, so there is nothing to select from — the orbital has to be evaluated. Doing that on the cluster is the point: the alternative is shipping the wavefunction, which is the transfer you are trying to avoid.
orbitron canonical export water_orbitals.fchk --mo 4 -o homo.orbpackThat command evaluates the orbital once, on the machine running the export. The bundle stores the resulting little-endian f32 grid, its origin and voxel vectors, and the saved surface appearance. Desktop, CLI, and browser readers decode that attachment and extract the isosurface from it. They do not evaluate the basis functions again, and the bundle does not need to carry the source wavefunction for the surface to render.
The receiving machine still pays for the decoded grid and triangle mesh. Moving the evaluation to a cluster avoids source parsing, wavefunction transfer, and orbital evaluation on the laptop; it does not make the surface itself free to open or draw.
An NWChem .out prints only a truncated MO table, so it needs the companion .movecs alongside it. The sibling file is picked up automatically when its name matches; otherwise name it:
orbitron canonical export run.out --mo 48 --movecs run_scf.movecs -o lumo.orbpackThe grid defaults to 0.20 Å spacing with 4 Å of padding around the molecule, capped at 200³ points. Override either:
orbitron canonical export water_orbitals.fchk --mo 4 --grid-spacing 0.5 -o coarse.orbpackOn the water example that takes the grid from 40×48×43 to 16×20×18, and the bundle from 314 KB to 23 KB. A request that would exceed the point cap is coarsened rather than refused — a coarser grid still shows the orbital, whereas an error would just make you guess a spacing.
A whole run directory. Point at the directory and Orbitron picks the entry point:
orbitron canonical export ~/runs/water/ -o water.orbpackIf the directory holds several candidates it will not guess — it lists them and asks you to name one. Engine scratch files (.cphf_rhs, .db, .movecs and friends) are never swept in; only the run’s real output is read.
Sharing a bundle. Provenance normally records the full path a bundle came from, which is useful for tracing a figure back to its run and less useful when the bundle is attached to a paper:
orbitron canonical export run.out -o run.orbpack --scrubPaths become bare filenames. Content hashes stay, so the data is still verifiable.
6.4 Saving the view: File ▸ Export ▸ Export Bundle…
The CLI has no camera, so a bundle it writes carries no pose. The desktop writes the same bundle plus how the scene is being looked at — camera, appearance preset, background and render mode — in a presentation section.
The CLI can still record the look, which is the part that does not need a window:
orbitron canonical export c60.xyz -o c60.orbpack --appearance signature
orbitron canonical export 1ubq.pdb -o 1ubq.orbpack \
--appearance signature --show cartoon --show atoms --hide cell--appearance takes signature, classic or publication. Without it a bundle says nothing about how it should look, and every viewer falls back to its own default — which is why the same file could come up one way in the desktop and another on a web page.
--show and --hide take cartoon, atoms or cell, and either can be repeated. A layer you do not name is left to the viewer’s own default, so --show cartoon alone does not blank the structure. The cartoon is recorded as a flag rather than as a mesh: the ribbon is derived from the scene and is megabytes for a protein, so each viewer builds its own. --hide cell is the one worth knowing about — a PDB’s CRYST1 record makes a protein periodic as far as a renderer is concerned, so a structure nobody thinks of as a crystal otherwise opens inside a box.
Reopen that bundle and it comes up posed the way you left it, in the desktop or embedded in a web page. That makes a bundle a reasonable way to hand someone a figure: they get the data and your view of it in one file, and can still rotate it.
The bundle is built by re-parsing the file on disk, so unapplied edits in the edit session are not in it — the status line says so when that is the case.
6.5 Sessions are bundles too
File ▸ Save Session writes a .orbpack as well. It is the same bundle Export Bundle produces, plus this window’s own state — panel layout, measurements, selection, annotations, export presets — carried as an opaque attachment beside the portable presentation section.
That state means nothing to the CLI or the browser, so it is not given a place in the schema; anything that does not recognise the attachment ignores it and still reads the data and the view.
A session used to be a JSON file holding UI state and a path to the file it was looking at, so moving or deleting that file left a session that restored your panels onto an empty viewport. The bundle carries the data, which is the point of folding the two together. Sessions saved as .orbitron.json still open.
Volume datasets loaded into the Surfaces panel are part of that portable data. The session stores each decoded grid, its Cartesian placement, and the exact row settings, including contour, signed-contour visibility, colors, opacity, and visibility. A moved session therefore reopens a CUBE, XSF, MRC2014, or CCP4 surface without the original volume file. Source paths and digests remain as provenance; they are not reopen dependencies. Multiple separately loaded volumes retain their dataset order.
| Data | View | Window state | |
|---|---|---|---|
orbitron canonical export |
yes | — | — |
File ▸ Export ▸ Export Bundle… |
yes | yes | — |
File ▸ Save Session… |
yes | yes | yes |
The structure row includes the editable molecular charge/multiplicity, so a chosen charge, spin state, or accepted conformer survives a session save. XYZ does not retain either charge or multiplicity, and MOL/SDF/PDB do not retain Orbitron’s multiplicity; use scene bytes or .orbpack when that state must round-trip.
Open a session with File ▸ Open Session. Opening one with File ▸ Open instead is fine — you get the scene and the view, and the window state is ignored.
6.6 Opening a bundle someone sent you
A bundle is meant to be exchanged, so opening one is reading a file you did not write. Everything an archive says about itself is treated as a claim rather than a fact:
- Sizes are measured, not believed. A zip entry can declare a kilobyte and decompress to a hundred gigabytes, so reads are capped as they happen.
- Attachment bytes are checked against the hash the manifest gives for them. Attachment ids are content hashes, so this is what makes them mean anything; it catches a truncated copy or an archive edited after the fact. It is not a signature — a bundle crafted whole agrees with itself, and nothing records who made it.
- An archive member the manifest never mentions is refused, and so are entry names that would escape the bundle root.
resources.jsonis advisory until the full open. Current bundles put a fixed-shape resource record beforemanifest.jsonand cap its actual uncompressed read at 64 KiB. A full open rederives the record from the canonical manifest and refuses any disagreement, so the small record never becomes an allocation authority.
There is deliberately no compression-ratio rule. A volumetric grid that is mostly zero compresses enormously and is entirely legitimate, so a ratio test would reject real data while a determined bomb padded itself under the threshold.
6.7 Opening a bundle
A bundle is an ordinary Orbitron file. File ▸ Open in the viewer, drag it onto the window, or:
orbitron info run.orbpack
orbitron render run.orbpack -o figure.pngBundles are deliberately partial, so a section you did not export simply is not there. Re-run the export with the flag that includes it.
6.8 From Python
The Python API mirrors the CLI, which matters when you are working in a notebook on the cluster:
import orbitron
orb = orbitron.Orbitron()
info = orb.export_bundle("run.out", "run.orbpack", mo=31)
print(info["source_bytes"], info["bytes"], info["transfer_reduction_percent"])
print(info["sections"])
scene = orb.load("run.orbpack") # same call as any other fileexport_bundle returns path, bytes, source_bytes, transfer_reduction_percent, attachments and sections, and takes the same options as the CLI (with_source, mo, scrub, force). source_bytes and the percentage are None when there is no exact file comparison, such as an in-memory scene or directory input. A negative percentage means the bundle is larger than the source. A test asserts that a bundle written from Python and one written by the CLI produce the same scene, so the two cannot quietly diverge.
Reading works through the ordinary entry points, so a bundle substitutes for the run it came from:
orb.load("run.orbpack") # structure
orb.load_trajectory("run.orbpack") # optimisation frames
orb.analyze_vibrations("run.orbpack") # normal modesMachine-readable export records
Use --json when another program needs to audit an export:
orbitron canonical export run.out -o run.orbpack --json > run.export.jsonThe orbitron.canonical.export/1 record keeps the same keys for success and failure. A success contains a SHA-256 identity for a regular source file and for the finished bundle, exact sections and resource counts, source and bundle bytes, transfer reduction, elapsed milliseconds, and the highest RSS observed by the export’s 50 ms sampler. A directory source has no single-file digest or byte count. Bundle hashing streams through a fixed 64 KiB buffer. It does not read the finished bundle into memory a second time.
A failure leaves bundle-only fields as null, includes the error text, writes the JSON record to stdout, and still exits nonzero. Shell code must test the exit status. Warnings and the normal error message stay on stderr, so redirected stdout remains valid JSON. --json and --dry-run are mutually exclusive because a dry run does not produce a bundle to verify.
For a self-contained web page, hand export_html the bundle rather than a scene:
orbitron.export_html("run.orbpack", "run.html")The page then embeds the bundle and opens it with the viewer’s own reader, so it keeps the frames, the modes, the grids and the recorded view. Passing a Scene still works and still produces a page, but a scene is one static geometry — that is all it can be — and everything else is dropped on the way in.
6.9 In the browser
The WASM viewer reads bundles directly. It has no parser stack — a bundle is already-parsed data, so it only has to deserialize:
const bytes = new Uint8Array(await file.arrayBuffer());
wasm.load_bundle(bytes);
console.log(wasm.bundle_sections(bytes)); // "structure,trajectory,frequency,…"A grid exported with --mo is drawn on load — the browser has marching cubes, so a bundle holding one orbital shows that orbital, at the same default isovalue the desktop picks for its kind (0.05 for an orbital or an ESP, 0.002 for a density). This is the payoff of the --mo flag: the 613 MB cube stays on the cluster and the 1.9 MB bundle renders the surface in a web page.
If the bundle recorded a view, the browser restores it too — camera, halos, cell overlays, theme, background and which layers were showing — so an embedded viewer opens at the pose the desktop was showing rather than at its own default framing. The front page’s ubiquitin tile is exactly that: one 20 KB bundle, no appearance or layer arguments in the embed URL.
Cartoon ribbons are built, not carried. A bundle saved while showing a cartoon records that fact, and the viewer traces the ribbon from the scene when it loads. The mesh never travels: a protein’s is megabytes, while the instruction is one boolean, and every viewer that can draw a ribbon can build one. A 2.3 KB helix bundle draws its cartoon in the browser.
That works because a bundle carries the names of the atom properties it is keyed by. Property ids are indices into a registry each process fills as it parses, so the numbers alone mean nothing elsewhere — a viewer that had never read a PDB could not tell which property was the atom name, and reported a protein as having no backbone. The names travel beside the values and are translated into the reading process’s numbering on open, which is what makes chains, residues, b-factors and occupancies readable from a bundle at all.
A bundle is also the only thing the web viewer can animate: a bare scene is a single geometry, while a bundle carries the optimisation steps or MD frames the run produced, and the normal modes if it was a frequency job. Step through the trajectory with setFrame(i), animate a mode with animateMode(i), or hand it to the embed page and let it play:
/viewer/embed.html?scene=/path/run.orbpack&frames=play&fps=12
/viewer/embed.html?scene=/path/freq.orbpack&mode=2&fps=12A mode is not stored as frames — a bundle keeps the equilibrium geometry and a displacement vector per atom, and the viewer generates the oscillation from them, which is why a 3-atom frequency run animates from 2.4 KB.
See web embedding for the rest of the parameters.
This is why the format lives in its own orbitron-orbpack crate: the parsing crate depends on HDF5 and memory mapping and can never build for the browser, while the format itself compiles to wasm32 cleanly.
6.10 Where to run the export
Exporting parses the source, so it costs roughly what opening that source costs. Do not estimate the job from the eventual .orbpack size. A 2 MB bundle can be the selected result of a source that needed tens of gigabytes to parse.
--dry-run first prints information found in a 64 KiB bounded scan, when the format supports one. It then loads enough of the canonical document to report its exact sections and validate requested datasets. Treat the complete command as a no-write validation, not as a constant-memory preflight for a very large source.
Use orbitron resources SOURCE for that constant-bounded preflight. For an XDATCAR, pass --mode trajectory when the planned operation retains every frame; the default indexed-trajectory mode models bounded indexed access.
If a persisted max_file_size is smaller than the inspected source, the recommended export command includes --max-file-size set to that source’s exact inspected byte length. This is a one-command override for the export the user just assessed. If the file grows before export, the loader still refuses it instead of silently accepting a different, larger source.
6.11 Resource ballparks
Orbitron separates three numbers that are easy to confuse:
- Source-open memory covers parsing the original NWChem, Gaussian, VASP, CUBE, or other file and constructing its scene.
- Scene-display memory covers CPU render records and GPU instance buffers for the atoms and bonds that are actually shown.
- Bundle bytes are the compressed bytes stored or transferred. Compression and optional attachments make bundle size a poor predictor of memory.
Source-open memory
The table below is Orbitron’s current conservative preflight model. Values are incremental host memory for the operation. They exclude Orbitron’s existing resident memory, renderer buffers, later analysis, edit history, and other processes on the machine.
| Source and load shape | Current projection | Evidence |
|---|---|---|
| NWChem, Gaussian log, Molpro, Molcas, or QE text output | source size × 2 | Real NWChem measurements exist; Molcas whole-file parsing is not yet calibrated |
| Gaussian formatted checkpoint | source size × 3 | Conservative format heuristic |
| VASP static structure or scene | source size × 3 | Conservative format heuristic |
| XDATCAR with every frame retained | source size × 5 | Measured loads used 4.11 to 4.38 times the text size |
| Indexed XDATCAR with a bounded frame cache | source size × 1 | Measured indexed access stayed below the source size |
Orbitron .orbpack bundle |
4 × the larger of bundle size and its declared retained-scene floor | A 60.3 MB packed trajectory reached 187 MB above CLI startup; a 15.7 MB million-atom bundle reached 487 MB absolute RSS |
| XYZ, CIF, or PDB geometry | source size × 3 | Conservative format heuristic |
| CUBE or XSF volumetric source | source size × 2 | Measured CUBE loads used 1.18 to 1.22 times the file size; CHGCAR is not yet calibrated |
| MRC2014 or CCP4 map | source size × 2, raised by the exact header grid count when needed | EMD-3001 contains 78,475 f32 voxels in 315,084 bytes. A cold 512³ EMD-38398 parse peaked at 1,091,452,928 bytes, 1.65% above its 1,073,743,872-byte projection. |
| Unclassified source | source size × 4 | Generic fallback |
Small files have an 8 MiB minimum projection because parser and scene setup do not scale to zero.
For supported sources, Orbitron also reads at most 64 KiB of declarations or complete records before parsing. Declared atom, frame, or grid counts can raise the table’s size-based projection when the decoded scene is clearly larger. The desktop, CLI, Python API, and Rust SDK use the same count-aware estimate. A malformed or unsupported header falls back to the table instead of weakening the existing memory check.
Orbitron opens silently only while the projection is at most 70 percent of the memory currently available. Divide the projection by 0.70 to estimate the available-memory target. This is free memory at open time, not installed RAM.
For an NWChem output, the current planning figures are:
| Source size | Parse projection | Available memory for the silent band |
|---|---|---|
| 1 GiB | 2 GiB | about 3 GiB |
| 10 GiB | 20 GiB | about 29 GiB |
| 50 GiB | 100 GiB | about 143 GiB |
| 100 GiB | 200 GiB | about 286 GiB |
A deterministic policy replay uses the 10 GiB row above and assumes 75 percent of installed RAM is currently available:
| Installed RAM | Available RAM | Assessment | User-visible result |
|---|---|---|---|
| 8 GiB | 6 GiB | Block | CLI and TUI refuse; desktop puts Cancel first |
| 16 GiB | 12 GiB | Block | CLI and TUI refuse; desktop puts Cancel first |
| 32 GiB | 24 GiB | Warn | Interactive runs ask with a default-No choice |
| 64 GiB | 48 GiB | Ok | Open silently |
| 192 GiB | 144 GiB | Ok | Open silently |
For a noninteractive CLI run, Warn prints to stderr and proceeds; Block fails without waiting for input. The installed-memory column is context, not the decision input. Orbitron grades against the available-memory column and the physical-memory-plus-swap ceiling reported at that moment.
The desktop uses an in-app choice between Cancel, Open Optimized, and Open Full Detail before parsing a source in a warning band. Cancel is the default: Return, Escape, the Close button, and a click outside the dialog all leave the current document unchanged and start no parser. Exact scene counts can raise the assessment again after parsing but before renderer data is built; that second decision uses the same controls and restores the previous document on Cancel. Optimized presentation keeps labels off and skips bond inference when the source did not store connectivity. At the second decision, both open choices have already paid the source-parsing cost.
Scene counts are a separate interaction warning. At 1,000,000 atoms or 2,000,000 bonds, Orbitron emits a non-blocking large-scene notice even when the memory assessment is Comfortable. This boundary comes from the measured native target below. It discloses selection, labeling, editing, and full-listing work; it does not ask for confirmation or refuse the open. The desktop keeps the notice in the Resources section, orbitron resources includes it in text and JSON, and the TUI writes it before entering the alternate screen. Exact decoded counts replace header-derived counts after parsing.
These are warning figures, not exact runtime promises. Two ordinary mmap-backed NWChem files of 436 MB and 1.84 GB added about 3 to 7 percent of their source size in measured runs. A deliberately dense 1 GiB source with about 16 million short lines reached 2.164 GB absolute peak RSS, including about 16 MB of process baseline above the 2 GiB incremental projection. That case exposed and removed several whole-file line indexes. Do not extrapolate either workload to a 100 GiB file without measuring a representative output on the target system.
Scene-display memory
The current native renderer holds public CPU records, raw upload scratch, and GPU instance buffers. For scenes with two bond segments per atom, its current layout gives these figures:
| Atoms | Bond instances | CPU display records | Exact GPU payload | GPU allocation | Measured process peak |
|---|---|---|---|---|---|
| 10,000 | 20,000 | 5.0 MB | 2.2 MB | 3 MiB | not measured |
| 250,000 | 500,000 | 124 MB | 56 MB | 80 MiB | 278 MB |
| 1,000,000 | 2,000,000 | 496 MB | 224 MB | 320 MiB | 816 MB |
The measured peaks came from short synthetic Metal runs on an M2 Ultra. They show that the current native buffer layout can present those cases on that machine. They do not establish a supported editing limit, sustained frame rate, browser limit, or the memory cost of labels, properties, trajectories, grids, surfaces, picking, selection, analysis caches, or undo history. On Apple Silicon, the GPU allocation shares unified memory with the process. A discrete GPU has a separate VRAM budget.
Orbitron splits atom and bond instances across multiple buffers and draw calls when one table exceeds the selected GPU’s single-buffer limit. The reported GPU allocation is the sum of those buffers, and the allocation record includes the chunk count and maximum instances per buffer. This removes a representation boundary; it does not reduce the total CPU, GPU, or draw cost of the scene.
The current count policy separates measured support from representability:
| Surface | Supported or calibrated | Experimental | Refused |
|---|---|---|---|
| Native desktop | Static viewing through 1,000,000 atoms and 2,000,000 explicit bonds on the measured M2 Ultra | Larger scenes that pass host-memory and selected-device checks | Configured hard memory cap, integer-shape overflow, or a device unable to hold one render instance |
| Browser | Up to 100,000 atom and 200,000 bond instances; measured on hardware WebGPU through the cap | Software WebGL2 remains usable for small scenes but becomes slow well below the cap | Any count above either browser cap |
| CLI and TUI | Data operations that pass the loader and memory policy; ordinary human listings show 100 rows by default | Full JSON or --all output for very large result sets |
Configured hard caps and impossible integer or allocation shapes |
The native row is a tested target, not a universal machine guarantee. Its resource check still depends on available host memory and the selected GPU. Orbitron emits a non-blocking interaction notice at the native measured target. Above it, opening may work, but labels, picking, editing, and dense bond drawing can become impractical because the renderer has no level-of-detail path yet.
The browser viewer has a separate, lower safety policy. It accepts at most 100,000 atom instances and 200,000 bond instances, then shows an error directing the user to the desktop viewer or a smaller exported selection. WebGPU and WebGL2 use the same ceiling because both retain the decoded scene and renderer records inside one browser tab. A faster GPU changes drawing speed, but it does not remove the WASM and host-memory duplication that this first cap bounds. Native desktop, CLI, and TUI workflows do not inherit this fixed count limit.
On an M4 MacBook Air, Chromium hardware WebGPU held about 144 frames per second at 100,000 atoms and again at 100,000 atoms plus 200,000 bonds. The measured browser process trees peaked near 1.31 GB and 1.62 GB respectively. SwiftShader software WebGL2 took about 956 ms per frame for 100,000 atoms and 2.54 seconds per frame at the atom-and-bond cap. Treat software rendering as a small-scene fallback rather than a path to the published browser ceiling.
Viewport textures depend on resolution and multisampling even when the scene is small. These are logical payloads. A graphics backend may add row, image, or heap padding:
| Viewport | Samples | Logical GPU color + depth | Offscreen readback buffer | Returned RGBA pixels |
|---|---|---|---|---|
| 1920 × 1080 onscreen | 1 | 16.6 MB | none | none |
| 1920 × 1080 onscreen | 4 | 74.6 MB | none | none |
| 1920 × 1080 offscreen | 4 | 74.6 MB | 8.3 MB | 8.3 MB |
| 3840 × 2160 onscreen | 4 | 298.6 MB | none | none |
| 3840 × 2160 offscreen | 4 | 298.6 MB | 33.2 MB | 33.2 MB |
Prepared triangle meshes add exact buffer payloads before device padding. The current layout uses 36 bytes per uploaded vertex, 12 bytes per triangle, and a 32-byte material uniform per mesh. Orbitron also retains source positions, normals, triangles, and optional vertex colors on the CPU. Surface extraction cannot be predicted from grid dimensions alone because the triangle count depends on the field and isovalue. Until a surface has been built, the resource report lists that cost as unestimated instead of presenting a precise-looking guess.
What an .orbpack saves
A bundle avoids running the original application parser on the receiving machine. It can also avoid transferring the original source when --with-source is omitted, which is the default. It still has to decode the manifest and selected attachments, construct the scene, and pay the full display and editing cost of that scene.
For example, a 100 GiB NWChem output whose selected result is a 10,000-atom, 20,000-bond final scene can avoid the 200 GiB conservative source-parse projection on the laptop. The current renderer component for that scene is about 5 MB of CPU records and 3 MiB of GPU allocation, plus the application, scene properties, and any attachments included in the bundle.
A bundle with one million atoms and two million bond instances remains heavy. The short native calibration peaked at 816 MB and allocated 320 MiB for GPU instances before adding long trajectories, grids, surfaces, labels, analysis, or edit history. A highly compressed trajectory or grid can also expand far beyond the .orbpack file size when decoded. Use atom, bond, frame, and grid counts when they are available. Do not use compressed bundle bytes alone.
For density maps, a bundle skips reparsing the MRC/CCP4 header and can omit the original map while retaining the normalized f32 grid, transform, cell, contour, colors, and visibility. The EMD-3001 fixture shrank from 315,084 bytes to a 142,855-byte bundle. Opening that bundle still decodes all 78,475 voxels and pays the full surface-extraction and GPU cost. Use the source system to make the bundle when parsing or transferring the original map is the expensive part; do not treat the smaller archive as a low-memory preview.
Schema 3 trajectory bundles store coordinates as packed little-endian f32 values. Native readers keep that full attachment in memory, decode only the requested frame, and cache at most three decoded frame snapshots. For F frames and A atoms, the packed coordinate payload is exactly 12 × F × A bytes before container overhead. The current frame and its scene data add to that payload. Compatibility APIs that return an eager Trajectory still materialize every frame, so use the normal desktop, CLI, TUI, or random-access Rust path for a long trajectory.
The current bundle preflight uses four times the larger of stored bundle bytes and the declared retained-scene floor. That floor includes atom records, resident trajectory positions, selected grid values, and packed trajectory coordinates. In the measured 50,000-frame, 100-atom case, a 60.3 MB bundle with a 60 MB coordinate attachment reached 199.3 MB absolute RSS, about 187 MB above CLI startup. A 15.7 MB bundle carrying one million atoms and a 12 MB coordinate attachment reached 487 MB absolute RSS; its revised projection is 560 MB. The multiplier includes headroom for manifest decoding and scene reconstruction. It does not include renderer buffers, surfaces, analysis, or editing history.
New bundles store those facts in a fixed-shape resources.json ZIP member. The million-atom record is 352 bytes, while its canonical manifest is 114.8 MB uncompressed. A measured orbitron resources run read the 352-byte record in 0.01 seconds at 14.5 MB peak RSS and reported the exact 560 MB projection. It did not read the manifest or any attachment. Schema 2.0 and 3.0 bundles have no record, so Orbitron retains the bounded manifest-inspection path for them.
The fast record is an early warning, not trusted bundle content. During a real open Orbitron still decodes manifest.json, rederives every resource field, and rejects a stale or false record. The same million-atom bundle fully reopened in 1.23 seconds at 486.0 MB peak RSS. Bundling avoids the original XYZ or application parser; it does not remove manifest decoding or scene construction.
Scheduler requests
For the first export from an unmeasured format or corpus, size the memory request from the source projection and the 70 percent headroom rule. A 10 GiB NWChem output projects to 20 GiB, so about 29 GiB must be available for the operation to stay in Orbitron’s silent band. A 32 GiB job is a reasonable first request if the node has little unrelated memory pressure. Record the scheduler’s peak RSS, then adjust later jobs from evidence rather than retaining the conservative first request indefinitely.
Orbitron does not yet predict CPU time or wall time. Most canonical exports do not become faster merely because many CPUs were requested. Start with one Orbitron process per source, time a representative file, and use job arrays for independent files. Add CPUs when the selected workflow actually computes a grid or another parallel analysis. Sum the per-file memory projections before running several exports concurrently on one node.
For a large export, or one that has to read several big files, check your site’s login-node limits first; many cap memory and CPU time per session. If your export is bigger than those allow, run it in a job like any other compute:
#!/bin/bash
#SBATCH -J orbpack
#SBATCH --time=01:00:00
#SBATCH --cpus-per-task=1
#SBATCH --mem=32G
orbitron canonical export "$SCRATCH/run/big.out" -o "$SCRATCH/run/big.orbpack"That request matches the 10 GiB NWChem example. Replace the memory and time with figures for your source type and measured corpus. Slurm suffixes and site policy vary, so check the cluster’s local documentation before copying the request unchanged.
A good habit for very large output is to export at the end of the job that produced it, so the reduction happens once, on nodes already sized for the work, and the bundle is waiting when you want to look.
Preparing many runs
The repository includes a serial loop and Slurm-array recipe for a set of independent calculation outputs. Put one source path on each line of inputs.txt, then run:
ORBITRON=/path/to/orbitron \
./run-list.sh inputs.txt "$SCRATCH/orbitron-prepared"The serial loop keeps one parser active, which is the safe starting point for an unmeasured corpus. Each export writes an orbitron.canonical.export/1 record containing source and bundle hashes, exact bundle counts, byte sizes, elapsed time, sampled peak RSS, and any error. A second run skips a source only when its current source hash and the recorded bundle hash both still match.
For independent Slurm tasks, submit slurm-array.sh with an array limit such as --array=1-100%4. Keep the limit at one until the sum of the worst-case memory projections fits the requested tasks. Slurm enforces the per-task memory request; the example does not add a scheduler dependency to Orbitron.
After the array finishes, run the included collector with the same input list:
python3 collect-report.py collect \
"$ORBITRON_OUTPUT_DIR/records" \
"$ORBITRON_OUTPUT_DIR/report.json" \
"$ORBITRON_OUTPUT_DIR/transfer.txt" \
--input-list "$ORBITRON_INPUT_LIST"The collector returns nonzero for an export failure, a missing record, or a bundle whose SHA-256 no longer matches. transfer.txt contains only verified outputs from the current input list, so an older record left in the output directory cannot enter a later transfer.
6.12 Getting orbitron onto a cluster
Any of the usual routes work. A prebuilt binary from the releases page needs no toolchain. pip install orbitron gives you the Python API, which is often the easiest option if you already have a conda environment there — export_bundle does everything the CLI does. Building from source takes a few minutes and needs only a Rust toolchain; the CLI does not require the desktop’s dependencies.