Bundles (`.orbpack`)

6. Working with files on an HPC system

Quantum-chemistry output is often far larger than the part you want to look at. A multi-orbital cube stores every orbital interleaved at every grid point, so a file can be gigabytes while the one orbital you want to see is a few megabytes. Copying the whole thing to a laptop is slow, and parsing it locally can require more memory than the laptop has. Orbitron has no default file-size refusal. It estimates memory from the source type and current free memory, then warns when the operation is tight. You can set an explicit --max-file-size or --max-memory limit when you need a hard boundary.

A bundle solves this by moving the parsing to where the data already is. You run the CLI on the cluster, it produces a single small .orbpack file, and you copy that. There is nothing to install on the remote beyond the orbitron binary itself, no daemon, no port, and no connection to keep alive.

cluster                                    laptop
───────                                    ──────
run.out  ──▶  orbitron canonical export
                        │
                   run.orbpack  ──── scp ────▶  run.orbpack
                                                    │
                                              orbitron info / File ▸ Open

6.1 The loop: inspect, then export

Estimate the source-open cost before starting the parser:

orbitron resources water.out
orbitron resources water.out --max-memory 34359738368 --json

resources reads metadata and at most the first 64 KiB. It reports the source projection, bounded count hints, current machine headroom, and whether the planned open fits the configured policy. It does not validate the whole source or prove that parsing will succeed. A refusal in the report still returns exit 0, so a script should branch on verdict.band or verdict.enforced rather than the process status alone.

Then ask what the file contains before you export it. --dry-run writes nothing and validates the requested sections and dataset index, but it does parse the source after its bounded preflight:

orbitron canonical export water.out --dry-run

Dry run: water.out
  would write: water.orbpack
  bounded preflight: nwchem, 3 atoms (observed in the first 64.0 KB)
  sections available: structure (3 atoms), trajectory (3 frames), frequency (9 modes),
                      thermochemistry, electronic structure
  (nothing written)

For a cube it also reports how many orbitals are inside, which is what you need before asking for one:

orbitron canonical export orbitals.cube --mo 31 --dry-run

  volumetric datasets available: 90
  requested: dataset 31 (valid)

Ask for one that does not exist and it fails without writing a bundle:

Error: requested dataset 99 but orbitals.cube has 90

Then export:

orbitron canonical export water.out -o water.orbpack

Bundle: water.orbpack (6.6 KB)
  transfer: 245730 source bytes -> 6758 bundle bytes (97.2% reduction)
  sections: structure (3 atoms), trajectory (3 frames), frequency (9 modes),
            thermochemistry, electronic structure
  attachments: 3

The transfer line uses exact filesystem byte counts. For a computed orbital it compares the bundle with the wavefunction you selected, not with Orbitron’s temporary CUBE. Directory inputs have no single honest source-byte count, so they report that the source count is unavailable. The report still gives the exact bundle bytes.

6.2 What goes in by default

Everything cheap, nothing expensive. Geometry, trajectory, frequencies, thermochemistry and electronic structure are included automatically — they are kilobytes to a few megabytes. Volumetric grids and the original source file are only ever present because you asked for them.

Flag	Effect
(none)	Structure, trajectory, frequencies, thermochemistry, electronic structure
`--mo N`	Add orbital `N` (0-based): selected from a cube, or computed from a wavefunction
`--grid-spacing`, `--grid-padding`	Override the computed grid’s resolution and extent (Å)
`--movecs FILE`	MO coefficients for `--mo` on an NWChem `.out` (defaults to the sibling `.movecs`)
`--no-infer-bonds`	Keep the source’s connection table unchanged, including a missing table
`--appearance PRESET`	Record an appearance preset, so the bundle opens wearing it (§6.4)
`--show`, `--hide`	Record a layer as shown or hidden: `cartoon`, `atoms`, `cell`. Repeatable
`--group HOW:SELECTION`	Give a selection its own representation, colour, contacts and residue labels. Repeatable (§6.4)
`--camera FILE`	Record a camera, from the same JSON `orbitron render --camera` reads
`--ribbon-opacity ALPHA`	Record the ribbon opacity, 0 to 1
`--with-source`	Embed the original file, so the bundle can be re-parsed later
`--no-mo-coefficients`	Leave out the orbital coefficient matrices. Energies, occupations and symmetries stay, but no orbital surface can be computed from the bundle. For web embeds, which never read them
`--scrub`	Strip absolute paths from provenance before sharing
`--dry-run`	Report what would happen and write nothing
`--json`	Emit a versioned per-source record for scripts and batch reports
`--pretty`	Pretty-print the manifest JSON
`--force`	Overwrite the output file if it already exists
`--skip-cache`	Skip hydrating the canonical cache after export

By default, export computes covalent bonds when a source such as XYZ declares no connection table. The bundle records the resulting table as complete, so the desktop, CLI, Python, and browser readers use those stored bonds instead of running distance inference again. --no-infer-bonds is for a faithful record of the source; a reader may still infer connectivity later when it needs bonds.

The source-light default also applies to Orbitron’s canonical cache. The cache keeps the raw source’s hash and path reference but does not make another copy of the file. A later --with-source export from a cache hit reads the file at the currently opened path and embeds those exact bytes on demand.

What the manifest holds

A bundle is a ZIP, and its manifest.json is an envelope rather than a single document: a schema version, a list of members — one canonical document per source that was read — the index of the active member a reader opens on, and the warnings the set could not reconcile. Attachments (volumetric grids, packed trajectory coordinates) sit beside it as their own ZIP entries, addressed by content hash.

One source is the case of one member, so an ordinary export is not a different shape from a multi-source one. canonical export takes as many sources as you give it, read as one calculation in the order given:

orbitron canonical export jobA.out jobB.out -o chain.orbpack

That is a study — a restart chain, or unrelated runs looked at together — and a bundle is the only way one reaches the web viewer. What a study does and does not merge is in Several Files, One Calculation.

Bundles written before schema 3.0 do not open

The envelope is container schema major 3. A bundle written before it held a bare single document, which cannot express a calculation made of several sources, and Orbitron refuses it with a message saying so rather than failing as a parse error. Re-export the run; nothing else is needed, and the source files are the only thing required.

6.3 Worked examples

A frequency run. You want the vibrational modes from an NWChem job:

# on the cluster
orbitron canonical export cu-freq.out -o cu-freq.orbpack
# Bundle: cu-freq.orbpack (1.7 MB)
#   sections: structure (13 atoms), trajectory (68 frames), frequency (39 modes), ...

# on your laptop
scp cluster:~/runs/cu-freq.orbpack .
orbitron info cu-freq.orbpack

That is 22.8 MB of NWChem output reduced to 1.7 MB (366 KB with --no-mo-coefficients), with the trajectory and all 39 modes intact.

One orbital from a large cube. A 90-orbital cube is 613 MB; orbital 31 is not:

orbitron canonical export orbitals.cube --mo 31 -o homo.orbpack

An orbital from a wavefunction. An FCHK or an NWChem run carries the wavefunction but no grid, so there is nothing to select from — the orbital has to be evaluated. Doing that on the cluster is the point: the alternative is shipping the wavefunction, which is the transfer you are trying to avoid.

orbitron canonical export water_orbitals.fchk --mo 4 -o homo.orbpack

That command evaluates the orbital once, on the machine running the export. The bundle stores the resulting little-endian f32 grid, its origin and voxel vectors, and the saved surface appearance. Desktop, CLI, and browser readers decode that attachment and extract the isosurface from it. They do not evaluate the basis functions again, and the bundle does not need to carry the source wavefunction for the surface to render.

The receiving machine still pays for the decoded grid and triangle mesh. Moving the evaluation to a cluster avoids source parsing, wavefunction transfer, and orbital evaluation on the laptop; it does not make the surface itself free to open or draw.

An NWChem .out prints only a truncated MO table, so it needs the companion .movecs alongside it. The sibling file is picked up automatically when its name matches; otherwise name it:

orbitron canonical export run.out --mo 48 --movecs run_scf.movecs -o lumo.orbpack

The grid defaults to 0.20 Å spacing with 4 Å of padding around the molecule, capped at 200³ points. Override either:

orbitron canonical export water_orbitals.fchk --mo 4 --grid-spacing 0.5 -o coarse.orbpack

On the water example that takes the grid from 40×48×43 to 16×20×18, and the bundle from 314 KB to 23 KB. A request that would exceed the point cap is coarsened rather than refused — a coarser grid still shows the orbital, whereas an error would just make you guess a spacing.

A whole run directory. Point at the directory and Orbitron picks the entry point:

orbitron canonical export ~/runs/water/ -o water.orbpack

If the directory holds several candidates it will not guess — it lists them and asks you to name one. Engine scratch files (.cphf_rhs, .db, .movecs and friends) are never swept in; only the run’s real output is read.

Sharing a bundle. Provenance normally records the full path a bundle came from, which is useful for tracing a figure back to its run and less useful when the bundle is attached to a paper:

orbitron canonical export run.out -o run.orbpack --scrub

Paths become bare filenames. Content hashes stay, so the data is still verifiable.

6.4 Saving the view: File ▸ Export ▸ Export Bundle…

The desktop writes the same bundle plus how the scene is being looked at — camera, appearance preset, background and render mode — in a presentation section. Frame it with the mouse, export, and the bundle opens at what you were looking at.

The CLI records the same view flag by flag, which is what a job script can do without a window:

orbitron canonical export c60.xyz -o c60.orbpack --appearance signature
orbitron canonical export 1ubq.pdb -o 1ubq.orbpack \
    --appearance signature --show cartoon --show atoms --hide cell

# the same bundle as a binding-site figure: pale ribbon, site in sticks,
# framed on the site rather than on the whole protein
orbitron canonical export 1ubq.pdb -o site.orbpack \
    --show cartoon --ribbon-opacity 0.35 \
    --group "sticks;contacts:resid 45" \
    --camera site-camera.json

--appearance takes signature, ink, glow, classic, publication, licorice, spacefill, presentation, print, lineart, sketch, comic or grayscale. licorice and spacefill record a representation as well as a theme, so a bundle wearing one opens drawn that way. Without it a bundle says nothing about how it should look, and every viewer falls back to its own default — which is why the same file could come up one way in the desktop and another on a web page.

--show and --hide take cartoon, atoms or cell, and either can be repeated. A layer you do not name is left to the viewer’s own default, so --show cartoon alone does not blank the structure. The cartoon is recorded as a flag rather than as a mesh: the ribbon is derived from the scene and is megabytes for a protein, so each viewer builds its own. --hide cell is the one worth knowing about — a PDB’s CRYST1 record makes a protein periodic as far as a renderer is concerned, so a structure nobody thinks of as a crystal otherwise opens inside a box.

--group, --camera and --ribbon-opacity are the three that turn a look into a figure, and they are the same flags orbitron render composes with (§4.3.1 of the CLI guide):

--group <HOW>[;color=<C>][;thickness=<A>][;scale=<X>][;contacts][;labels]:<SELECTION> gives part of the structure its own representation — the active site in sticks, the metal as a sphere, the polar contacts and labels drawn. <HOW> is sticks, ball-and-stick, spheres, cartoon, hidden or default; the colour is #rrggbb, #rgb or r,g,b; thickness sets the group’s bond thickness and scale multiplies its atom radii (both positive numbers); and the flag is repeatable. The group travels as its selection expression, not as a list of atoms, so any viewer holding the scene re-evaluates it and draws the same picture.
--camera FILE reads the same JSON orbitron render --camera does (position, target, distance). Without one every viewer auto-frames the whole structure, which for a binding-site figure leaves the site a speck in the middle of a protein.
--ribbon-opacity ALPHA fades the ribbon to a backdrop the sticks read through, which is the conventional binding-site figure.

Reopen that bundle and it comes up posed the way you left it, in the desktop or embedded in a web page. That makes a bundle a reasonable way to hand someone a figure: they get the data and your view of it in one file, and can still rotate it.

The bundle is built by re-parsing the file on disk, so unapplied edits in the edit session are not in it — the status line says so when that is the case.

6.5 Sessions are bundles too

File ▸ Save Session writes a .orbpack as well. It is the same bundle Export Bundle produces, plus this window’s own state — panel layout, measurements, selection, annotations, export presets — carried as an opaque attachment beside the portable presentation section.

That state means nothing to the CLI or the browser, so it is not given a place in the schema; anything that does not recognise the attachment ignores it and still reads the data and the view.

A session used to be a JSON file holding UI state and a path to the file it was looking at, so moving or deleting that file left a session that restored your panels onto an empty viewport. The bundle carries the data, which is the point of folding the two together. Sessions saved as .orbitron.json still open.

Volume datasets loaded into the Surfaces panel are part of that portable data. The session stores each decoded grid, its Cartesian placement, and the exact row settings, including contour, signed-contour visibility, colors, opacity, and visibility. A moved session therefore reopens a CUBE, XSF, MRC2014, or CCP4 surface without the original volume file. Source paths and digests remain as provenance; they are not reopen dependencies. Multiple separately loaded volumes retain their dataset order.

Molecular surfaces travel too, but as parameters rather than as geometry. The bundle records the surface kind, probe radius, grid spacing, radius table, selection, colouring, which components were drawn, and the row’s colour and opacity — not the hundred thousand triangles. A protein surface as geometry would add tens of megabytes to a file that is otherwise tens of kilobytes, and anything able to draw it can rebuild it from those fields in well under a second, which is what happens on open: the session reopens with the surface already drawn. It is the same choice the format makes for the cartoon ribbon, which travels as an intent rather than as a mesh.

That does mean a reader without the surface code opens the session without the surface, rather than with a picture of one. A bundle carrying molecular surfaces declares schema minor 2 so such a reader can tell.

	Data	View	Window state
`orbitron canonical export`	yes	as flags (§6.4)	—
`File ▸ Export ▸ Export Bundle…`	yes	yes	—
`File ▸ Save Session…`	yes	yes	yes

The structure row includes the editable molecular charge/multiplicity, so a chosen charge, spin state, or accepted conformer survives a session save. XYZ writes both as charge= and multiplicity= on its comment line, which its reader takes back (not the desktop’s XYZ of a scene with overlays, which has no one charge). MOL/SDF/PDB do not retain Orbitron’s multiplicity; use scene bytes or .orbpack when that state must round-trip.

Open a session with File ▸ Open Session. Opening one with File ▸ Open instead is fine — you get the scene and the view, and the window state is ignored.

6.6 Opening a bundle someone sent you

A bundle is meant to be exchanged, so opening one is reading a file you did not write. Everything an archive says about itself is treated as a claim rather than a fact:

Sizes are measured, not believed. A zip entry can declare a kilobyte and decompress to a hundred gigabytes, so reads are capped as they happen.
Attachment bytes are checked against the hash the manifest gives for them. Attachment ids are content hashes, so this is what makes them mean anything; it catches a truncated copy or an archive edited after the fact. It is not a signature — a bundle crafted whole agrees with itself, and nothing records who made it.
An archive member the manifest never mentions is refused, and so are entry names that would escape the bundle root.
The container version is read before the body. A pre-envelope manifest is a bare document rather than a list of members, so a version and shape check comes first and refuses it as the version mismatch it is — otherwise it surfaces as a confusing parse error much later. The message names what to do: re-export the run.
resources.json is advisory until the full open. Current bundles put a fixed-shape resource record before manifest.json and cap its actual uncompressed read at 64 KiB. A full open rederives the record from the canonical manifest and refuses any disagreement, so the small record never becomes an allocation authority.

There is deliberately no compression-ratio rule. A volumetric grid that is mostly zero compresses enormously and is entirely legitimate, so a ratio test would reject real data while a determined bomb padded itself under the threshold.

6.7 Opening a bundle

A bundle is an ordinary Orbitron file. File ▸ Open in the viewer, drag it onto the window, or:

orbitron info run.orbpack
orbitron render run.orbpack -o figure.png

Bundles are deliberately partial, so a section you did not export simply is not there. Re-run the export with the flag that includes it.

A bundle written from several sources opens as the study it was packed from: every member, the one it was left showing, and the warnings it carried. One member is displayed at a time and the task list spans all of them — Several Files, One Calculation has the rest.

An .orbpack from before schema 3.0 is refused rather than opened; re-export the run it came from.

6.8 From Python

The Python API covers the common export, which matters when you are working in a notebook on the cluster:

import orbitron

orb = orbitron.Orbitron()
info = orb.export_bundle("run.out", "run.orbpack", mo=31)
print(info["source_bytes"], info["bytes"], info["transfer_reduction_percent"])
print(info["sections"])

scene = orb.load("run.orbpack")   # same call as any other file

export_bundle returns path, bytes, source_bytes, transfer_reduction_percent, attachments and sections, and takes output, with_source, mo, movecs, grid_spacing, grid_padding, scrub, force and infer_bonds (the inversion of the CLI’s --no-infer-bonds, so it defaults to True). It takes one source: packing several files as one study bundle is CLI-only. So are the appearance and figure flags (--appearance, --show, --hide, --group, --camera, --ribbon-opacity), --no-mo-coefficients, --dry-run, --json, --pretty and --skip-cache. source_bytes and the percentage are None when there is no exact file comparison, such as an in-memory scene or directory input. A negative percentage means the bundle is larger than the source. A test asserts that a bundle written from Python and one written by the CLI produce the same scene, so the two cannot quietly diverge.

Reading works through the ordinary entry points, so a bundle substitutes for the run it came from:

orb.load("run.orbpack")               # structure
orb.load_trajectory("run.orbpack")    # optimisation frames
orb.analyze_vibrations("run.orbpack") # normal modes

Machine-readable export records

Use --json when another program needs to audit an export:

orbitron canonical export run.out -o run.orbpack --json > run.export.json

The orbitron.canonical.export/1 record keeps the same keys for success and failure. A success contains a SHA-256 identity for a regular source file and for the finished bundle, exact sections and resource counts, source and bundle bytes, transfer reduction, elapsed milliseconds, and the highest RSS observed by the export’s 50 ms sampler. A directory source has no single-file digest or byte count. Bundle hashing streams through a fixed 64 KiB buffer. It does not read the finished bundle into memory a second time.

A failure leaves bundle-only fields as null, includes the error text, writes the JSON record to stdout, and still exits nonzero. Shell code must test the exit status. Warnings and the normal error message stay on stderr, so redirected stdout remains valid JSON. --json and --dry-run are mutually exclusive because a dry run does not produce a bundle to verify.

For a self-contained web page, hand export_html the bundle rather than a scene:

orbitron.export_html("run.orbpack", "run.html")

The page then embeds the bundle and opens it with the viewer’s own reader, so it keeps the frames, the modes, the grids and the recorded view. Passing a Scene still works and still produces a page, but a scene is one static geometry — that is all it can be — and everything else is dropped on the way in.

6.9 In the browser

The WASM viewer reads bundles directly. It has no parser stack — a bundle is already-parsed data, so it only has to deserialize:

const bytes = new Uint8Array(await file.arrayBuffer());
wasm.load_bundle(bytes);
console.log(wasm.bundle_sections(bytes));  // "structure,trajectory,frequency,…"

A bundle holding several sources is reached the same way. bundle_members(bytes) returns JSON — one object per source with its index, the name it was read from, the program that wrote it, and whether it is the one the bundle opened on, plus the study’s own warnings — and load_bundle_member(bytes, i) shows a different one:

const { members, active, warnings } = JSON.parse(wasm.bundle_members(bytes));
members.forEach((m) => console.log(m.index, m.name, m.program, m.active));
wasm.load_bundle_member(bytes, 1);

The page passes the bytes back in rather than the viewer keeping a copy: it already holds what it fetched, and a second copy in the wasm heap would double the cost of every bundle to serve a control most pages never use. A single-source bundle returns one entry, so a page has one shape to render rather than two. This is the browser’s only route to a study — it has no parser stack, so a set of raw output files cannot be opened there at all.

A grid exported with --mo is drawn on load — the browser has marching cubes, so a bundle holding one orbital shows that orbital, at the same default isovalue the desktop picks for its kind (0.05 for an orbital or an ESP, 0.002 for a density). This is the payoff of the --mo flag: the 613 MB cube stays on the cluster and the 1.9 MB bundle renders the surface in a web page.

If the bundle recorded a view, the browser restores it too — camera, halos, cell overlays, theme, background and which layers were showing — so an embedded viewer opens at the pose the desktop was showing rather than at its own default framing. The examples page’s ubiquitin tile is nearly that: one 24 KB bundle whose embed URL adds only a lighting preset, with no camera or layer arguments.

Cartoon ribbons are built, not carried. A bundle saved while showing a cartoon records that fact, and the viewer traces the ribbon from the scene when it loads. The mesh never travels: a protein’s is megabytes, while the instruction is one boolean, and every viewer that can draw a ribbon can build one. A 2.3 KB helix bundle draws its cartoon in the browser.

That works because a bundle carries the names of the atom properties it is keyed by. Property ids are indices into a registry each process fills as it parses, so the numbers alone mean nothing elsewhere — a viewer that had never read a PDB could not tell which property was the atom name, and reported a protein as having no backbone. The names travel beside the values and are translated into the reading process’s numbering on open, which is what makes chains, residues, b-factors and occupancies readable from a bundle at all.

The same rule now applies when Python or a desktop export sends a bare Scene.to_bytes() payload to the browser. Its MessagePack envelope carries the definitions actually referenced by atom and bond maps, then the browser remaps their numeric ids into its registry before it evaluates a group expression. That is what makes resname HEM, resid 505, group-derived residue labels, and polar contacts work in a standalone web view of a PDB scene. Old bare scene payloads remain readable as geometry, but cannot recover property names they never carried; re-export them to make browser-side selection portable.

A bundle is also the only thing the web viewer can animate: a bare scene is a single geometry, while a bundle carries the optimisation steps or MD frames the run produced, and the normal modes if it was a frequency job. Step through the trajectory with setFrame(i), animate a mode with animateMode(i), or hand it to the embed page and let it play:

/viewer/embed.html?scene=/path/run.orbpack&frames=play&fps=12
/viewer/embed.html?scene=/path/freq.orbpack&mode=2&fps=12

A mode is not stored as frames — a bundle keeps the equilibrium geometry and a displacement vector per atom, and the viewer generates the oscillation from them, which is why a 3-atom frequency run animates from under 5 KB.

See web embedding for the rest of the parameters.

This is why the format lives in its own orbitron-orbpack crate: the parsing crate depends on HDF5 and memory mapping and can never build for the browser, while the format itself compiles to wasm32 cleanly.

6.10 Where to run the export

Exporting parses the source, so it costs roughly what opening that source costs. Do not estimate the job from the eventual .orbpack size. A 2 MB bundle can be the selected result of a source that needed tens of gigabytes to parse.

--dry-run first prints information found in a 64 KiB bounded scan, when the format supports one. It then loads enough of the canonical document to report its exact sections and validate requested datasets. Treat the complete command as a no-write validation, not as a constant-memory preflight for a very large source.

Use orbitron resources SOURCE for that constant-bounded preflight. For an XDATCAR, pass --mode trajectory when the planned operation retains every frame; the default indexed-trajectory mode models bounded indexed access.

If a persisted max_file_size is smaller than the inspected source, the recommended export command includes --max-file-size set to that source’s exact inspected byte length. This is a one-command override for the export the user just assessed. If the file grows before export, the loader still refuses it instead of silently accepting a different, larger source.

6.11 Resource ballparks

Orbitron separates three numbers that are easy to confuse:

Source-open memory covers parsing the original NWChem, Gaussian, VASP, CUBE, or other file and constructing its scene.
Scene-display memory covers CPU render records and GPU instance buffers for the atoms and bonds that are actually shown.
Bundle bytes are the compressed bytes stored or transferred. Compression and optional attachments make bundle size a poor predictor of memory.

Source-open memory

The table below is Orbitron’s current conservative preflight model. Values are incremental host memory for the operation. They exclude Orbitron’s existing resident memory, renderer buffers, later analysis, edit history, and other processes on the machine.

Source and load shape	Current projection	Evidence
NWChem, Gaussian log, Molpro, Molcas, or QE text output	source size × 2	Real NWChem measurements exist; Molcas whole-file parsing is not yet calibrated
Gaussian formatted checkpoint	source size × 3	Conservative format heuristic
VASP static structure or scene	source size × 3	Conservative format heuristic
XDATCAR with every frame retained	source size × 5	Measured loads used 4.11 to 4.38 times the text size
Indexed XDATCAR with a bounded frame cache	source size × 1	Measured indexed access stayed below the source size
Orbitron `.orbpack` bundle	4 × the larger of bundle size and its declared retained-scene floor	A 60.3 MB packed trajectory reached 187 MB above CLI startup; a 15.7 MB million-atom bundle reached 487 MB absolute RSS
XYZ, CIF, or PDB geometry	source size × 3	Conservative format heuristic
CUBE or XSF volumetric source	source size × 2	Measured CUBE loads used 1.18 to 1.22 times the file size; CHGCAR is not yet calibrated
MRC2014 or CCP4 map	source size × 2, raised by the exact header grid count when needed	EMD-3001 contains 78,475 `f32` voxels in 315,084 bytes. A cold 512³ EMD-38398 parse peaked at 1,091,452,928 bytes, 1.65% above its 1,073,743,872-byte projection.
Unclassified source	source size × 4	Generic fallback

Small files have an 8 MiB minimum projection because parser and scene setup do not scale to zero.

For supported sources, Orbitron also reads at most 64 KiB of declarations or complete records before parsing. Declared atom, frame, or grid counts can raise the table’s size-based projection when the decoded scene is clearly larger. The desktop, CLI and Python API use the same count-aware estimate. A malformed or unsupported header falls back to the table instead of weakening the existing memory check.

Orbitron opens silently only while the projection is at most 70 percent of the memory currently available. Divide the projection by 0.70 to estimate the available-memory target. This is free memory at open time, not installed RAM.

For an NWChem output, the current planning figures are:

Source size	Parse projection	Available memory for the silent band
1 GiB	2 GiB	about 3 GiB
10 GiB	20 GiB	about 29 GiB
50 GiB	100 GiB	about 143 GiB
100 GiB	200 GiB	about 286 GiB

A deterministic policy replay uses the 10 GiB row above and assumes 75 percent of installed RAM is currently available:

Installed RAM	Available RAM	Assessment	User-visible result
8 GiB	6 GiB	Block	CLI and TUI refuse; desktop puts Cancel first
16 GiB	12 GiB	Block	CLI and TUI refuse; desktop puts Cancel first
32 GiB	24 GiB	Warn	Interactive runs ask with a default-No choice
64 GiB	48 GiB	Ok	Open silently
192 GiB	144 GiB	Ok	Open silently

For a noninteractive CLI run, Warn prints to stderr and proceeds; Block fails without waiting for input. The installed-memory column is context, not the decision input. Orbitron grades against the available-memory column and the physical-memory-plus-swap ceiling reported at that moment.

The desktop uses an in-app choice between Cancel, Open Optimized, and Open Full Detail before parsing a source in a warning band. Cancel is the default: Return, Escape, the Close button, and a click outside the dialog all leave the current document unchanged and start no parser. Exact scene counts can raise the assessment again after parsing but before renderer data is built; that second decision uses the same controls and restores the previous document on Cancel. Optimized presentation keeps labels off and skips bond inference when the source did not store connectivity. At the second decision, both open choices have already paid the source-parsing cost.

Scene counts are a separate interaction warning. At 1,000,000 atoms or 2,000,000 bonds, Orbitron emits a non-blocking large-scene notice even when the memory assessment is Comfortable. This boundary comes from the measured native target below. It discloses selection, labeling, editing, and full-listing work; it does not ask for confirmation or refuse the open. The desktop keeps the notice in the Resources section, orbitron resources includes it in text and JSON, and the TUI writes it before entering the alternate screen. Exact decoded counts replace header-derived counts after parsing.

These are warning figures, not exact runtime promises. Two ordinary mmap-backed NWChem files of 436 MB and 1.84 GB added about 3 to 7 percent of their source size in measured runs. A deliberately dense 1 GiB source with about 16 million short lines reached 2.164 GB absolute peak RSS, including about 16 MB of process baseline above the 2 GiB incremental projection. That case exposed and removed several whole-file line indexes. Do not extrapolate either workload to a 100 GiB file without measuring a representative output on the target system.

Scene-display memory

The current native renderer holds public CPU records, raw upload scratch, and GPU instance buffers. For scenes with two bond segments per atom, its current layout gives these figures:

Atoms	Bond instances	CPU display records	Exact GPU payload	GPU allocation	Measured process peak
10,000	20,000	5.0 MB	2.2 MB	3 MiB	not measured
250,000	500,000	124 MB	56 MB	80 MiB	278 MB
1,000,000	2,000,000	496 MB	224 MB	320 MiB	816 MB

The measured peaks came from short synthetic Metal runs on an M2 Ultra. They show that the current native buffer layout can present those cases on that machine. They do not establish a supported editing limit, sustained frame rate, browser limit, or the memory cost of labels, properties, trajectories, grids, surfaces, picking, selection, analysis caches, or undo history. On Apple Silicon, the GPU allocation shares unified memory with the process. A discrete GPU has a separate VRAM budget.

Orbitron splits atom and bond instances across multiple buffers and draw calls when one table exceeds the selected GPU’s single-buffer limit. The reported GPU allocation is the sum of those buffers, and the allocation record includes the chunk count and maximum instances per buffer. This removes a representation boundary; it does not reduce the total CPU, GPU, or draw cost of the scene.

The current count policy separates measured support from representability:

Surface	Supported or calibrated	Experimental	Refused
Native desktop	Static viewing through 1,000,000 atoms and 2,000,000 explicit bonds on the measured M2 Ultra	Larger scenes that pass host-memory and selected-device checks	Configured hard memory cap, integer-shape overflow, or a device unable to hold one render instance
Browser	Up to 100,000 atom and 200,000 bond instances; measured on hardware WebGPU through the cap	Software WebGL2 remains usable for small scenes but becomes slow well below the cap	Any count above either browser cap
CLI and TUI	Data operations that pass the loader and memory policy; ordinary human listings show 100 rows by default	Full JSON or `--all` output for very large result sets	Configured hard caps and impossible integer or allocation shapes

The native row is a tested target, not a universal machine guarantee. Its resource check still depends on available host memory and the selected GPU. Orbitron emits a non-blocking interaction notice at the native measured target. Above it, opening may work, but labels, picking, editing, and dense bond drawing can become impractical because the renderer has no level-of-detail path yet.

The browser viewer has a separate, lower safety policy. It accepts at most 100,000 atom instances and 200,000 bond instances, then shows an error directing the user to the desktop viewer or a smaller exported selection. WebGPU and WebGL2 use the same ceiling because both retain the decoded scene and renderer records inside one browser tab. A faster GPU changes drawing speed, but it does not remove the WASM and host-memory duplication that this first cap bounds. Native desktop, CLI, and TUI workflows do not inherit this fixed count limit.

On an M4 MacBook Air, Chromium hardware WebGPU held about 144 frames per second at 100,000 atoms and again at 100,000 atoms plus 200,000 bonds. The measured browser process trees peaked near 1.31 GB and 1.62 GB respectively. SwiftShader software WebGL2 took about 956 ms per frame for 100,000 atoms and 2.54 seconds per frame at the atom-and-bond cap. Treat software rendering as a small-scene fallback rather than a path to the published browser ceiling.

Viewport textures depend on resolution and multisampling even when the scene is small. These are logical payloads. A graphics backend may add row, image, or heap padding:

Viewport	Samples	Logical GPU color + depth	Offscreen readback buffer	Returned RGBA pixels
1920 × 1080 onscreen	1	16.6 MB	none	none
1920 × 1080 onscreen	4	74.6 MB	none	none
1920 × 1080 offscreen	4	74.6 MB	8.3 MB	8.3 MB
3840 × 2160 onscreen	4	298.6 MB	none	none
3840 × 2160 offscreen	4	298.6 MB	33.2 MB	33.2 MB

Prepared triangle meshes add exact buffer payloads before device padding. The current layout uses 36 bytes per uploaded vertex, 12 bytes per triangle, and a 32-byte material uniform per mesh. Orbitron also retains source positions, normals, triangles, and optional vertex colors on the CPU. Surface extraction cannot be predicted from grid dimensions alone because the triangle count depends on the field and isovalue. Until a surface has been built, the resource report lists that cost as unestimated instead of presenting a precise-looking guess.

What an `.orbpack` saves

A bundle avoids running the original application parser on the receiving machine. It can also avoid transferring the original source when --with-source is omitted, which is the default. It still has to decode the manifest and selected attachments, construct the scene, and pay the full display and editing cost of that scene.

For example, a 100 GiB NWChem output whose selected result is a 10,000-atom, 20,000-bond final scene can avoid the 200 GiB conservative source-parse projection on the laptop. The current renderer component for that scene is about 5 MB of CPU records and 3 MiB of GPU allocation, plus the application, scene properties, and any attachments included in the bundle.

A bundle with one million atoms and two million bond instances remains heavy. The short native calibration peaked at 816 MB and allocated 320 MiB for GPU instances before adding long trajectories, grids, surfaces, labels, analysis, or edit history. A highly compressed trajectory or grid can also expand far beyond the .orbpack file size when decoded. Use atom, bond, frame, and grid counts when they are available. Do not use compressed bundle bytes alone.

For density maps, a bundle skips reparsing the MRC/CCP4 header and can omit the original map while retaining the normalized f32 grid, transform, cell, contour, colors, and visibility. The EMD-3001 fixture shrank from 315,084 bytes to a 142,855-byte bundle. Opening that bundle still decodes all 78,475 voxels and pays the full surface-extraction and GPU cost. Use the source system to make the bundle when parsing or transferring the original map is the expensive part; do not treat the smaller archive as a low-memory preview.

Schema 3 trajectory bundles store coordinates as packed little-endian f32 values. Native readers keep that full attachment in memory, decode only the requested frame, and cache at most three decoded frame snapshots. For F frames and A atoms, the packed coordinate payload is exactly 12 × F × A bytes before container overhead. The current frame and its scene data add to that payload. Compatibility APIs that return an eager Trajectory still materialize every frame, so use the normal desktop, CLI, TUI, or random-access Rust path for a long trajectory.

The current bundle preflight uses four times the larger of stored bundle bytes and the declared retained-scene floor. That floor includes atom records, resident trajectory positions, selected grid values, and packed trajectory coordinates. In the measured 50,000-frame, 100-atom case, a 60.3 MB bundle with a 60 MB coordinate attachment reached 199.3 MB absolute RSS, about 187 MB above CLI startup. A 15.7 MB bundle carrying one million atoms and a 12 MB coordinate attachment reached 487 MB absolute RSS; its revised projection is 560 MB. The multiplier includes headroom for manifest decoding and scene reconstruction. It does not include renderer buffers, surfaces, analysis, or editing history.

New bundles store those facts in a fixed-shape resources.json ZIP member. The million-atom record is 352 bytes, while its canonical manifest is 114.8 MB uncompressed. A measured orbitron resources run read the 352-byte record in 0.01 seconds at 14.5 MB peak RSS and reported the exact 560 MB projection. It did not read the manifest or any attachment. Every bundle written by this Orbitron carries the record, unconditionally; one written without it falls back to bounded manifest inspection. That fallback reads the manifest, so it applies the same version check an open does: a pre-envelope (major 2) bundle is refused on either path rather than inspected.

The fast record is an early warning, not trusted bundle content. During a real open Orbitron still decodes manifest.json, rederives every resource field, and rejects a stale or false record. The same million-atom bundle fully reopened in 1.23 seconds at 486.0 MB peak RSS. Bundling avoids the original XYZ or application parser; it does not remove manifest decoding or scene construction.

Scheduler requests

For the first export from an unmeasured format or corpus, size the memory request from the source projection and the 70 percent headroom rule. A 10 GiB NWChem output projects to 20 GiB, so about 29 GiB must be available for the operation to stay in Orbitron’s silent band. A 32 GiB job is a reasonable first request if the node has little unrelated memory pressure. Record the scheduler’s peak RSS, then adjust later jobs from evidence rather than retaining the conservative first request indefinitely.

Orbitron does not yet predict CPU time or wall time. Most canonical exports do not become faster merely because many CPUs were requested. Start with one Orbitron process per source, time a representative file, and use job arrays for independent files. Add CPUs when the selected workflow actually computes a grid or another parallel analysis. Sum the per-file memory projections before running several exports concurrently on one node.

For a large export, or one that has to read several big files, check your site’s login-node limits first; many cap memory and CPU time per session. If your export is bigger than those allow, run it in a job like any other compute:

#!/bin/bash
#SBATCH -J orbpack
#SBATCH --time=01:00:00
#SBATCH --cpus-per-task=1
#SBATCH --mem=32G

orbitron canonical export "$SCRATCH/run/big.out" -o "$SCRATCH/run/big.orbpack"

That request matches the 10 GiB NWChem example. Replace the memory and time with figures for your source type and measured corpus. Slurm suffixes and site policy vary, so check the cluster’s local documentation before copying the request unchanged.

A good habit for very large output is to export at the end of the job that produced it, so the reduction happens once, on nodes already sized for the work, and the bundle is waiting when you want to look.

Preparing many runs

The repository includes a serial loop and Slurm-array recipe for a set of independent calculation outputs. Put one source path on each line of inputs.txt, then run:

ORBITRON=/path/to/orbitron \
  ./run-list.sh inputs.txt "$SCRATCH/orbitron-prepared"

The serial loop keeps one parser active, which is the safe starting point for an unmeasured corpus. Each export writes an orbitron.canonical.export/1 record containing source and bundle hashes, exact bundle counts, byte sizes, elapsed time, sampled peak RSS, and any error. A second run skips a source only when its current source hash and the recorded bundle hash both still match.

For independent Slurm tasks, submit slurm-array.sh with an array limit such as --array=1-100%4. Keep the limit at one until the sum of the worst-case memory projections fits the requested tasks. Slurm enforces the per-task memory request; the example does not add a scheduler dependency to Orbitron.

After the array finishes, run the included collector with the same input list:

python3 collect-report.py collect \
  "$ORBITRON_OUTPUT_DIR/records" \
  "$ORBITRON_OUTPUT_DIR/report.json" \
  "$ORBITRON_OUTPUT_DIR/transfer.txt" \
  --input-list "$ORBITRON_INPUT_LIST"

The collector returns nonzero for an export failure, a missing record, or a bundle whose SHA-256 no longer matches. transfer.txt contains only verified outputs from the current input list, so an older record left in the output directory cannot enter a later transfer.

6.12 Getting `orbitron` onto a cluster

Any of the usual routes work. A prebuilt binary from the releases page needs no toolchain. A release wheel gives you the Python API, which is often the easiest option if you already have a conda environment there — export_bundle covers the common single-source export. Orbitron is not on PyPI, so install that wheel by its release URL (pip install https://github.com/charliecpeterson/orbitron/releases/download/v0.8.0/orbitron-0.8.0-cp310-abi3-manylinux_2_28_x86_64.whl on Linux; see Installation for the other platforms) rather than with pip install orbitron. Building from source takes a few minutes and needs only a Rust toolchain; the CLI does not require the desktop’s dependencies.