iDaVIE: virtual reality for volumetric scientific data

Radio telescopes such as MeerKAT, ASKAP and APERTIF now produce spectral cubes with the same number of spatial pixels as a decade ago but up to 32,000 spectral channels. Assessing a cube like that one slice at a time is no longer practical, and the structures that matter most, the kinematic ones, are exactly those a stack of 2D projections hides.

iDaVIE renders the whole volume in room-scale virtual reality, where navigating, selecting, masking and measuring happen directly in three dimensions. It is built to sit inside an existing workflow rather than replace it: it reads standard FITS cubes and masks, imports VOTable catalogues, and exports edited masks, cropped subcubes, moment maps and spectral profiles back into the formats the rest of the pipeline expects.

The design rule throughout was to add an immersive interface only where it earns its place. File loading, parameter tuning and debugging stay on the desktop, where a keyboard is better. Navigation, selection and mask painting move into VR, where the flat screen is the bottleneck.

I have worked on iDaVIE since 2018, from the original room-scale proof of concept through to the version 1.0 release. I am first author and corresponding author on the software paper, credited for software, methodology, validation and conceptualisation.

• Built the native C++ plug-in layer that parses FITS, handles World Coordinate System transformations and computes statistics and downsampling.
• Designed the interaction model, including the locomotion and interaction state machines, the selection and mask-painting tools and the voice-command layer.
• Worked on the HLSL ray-marching shader and the rendering trade-offs that keep large cubes inside the VR comfort threshold.
• Coordinated releases, documentation and user support across an international collaboration, and supervised eight honours and MSc students contributing to the codebase.
• Lead the iDaVIE-d prototype, which extends the rendering and interaction stack into the digital planetarium.

Unity's hard ceiling on a single texture is 4 GB, a consequence of 32-bit memory addressing. In practice the binding limit is far lower. iDaVIE defaults to 368 MB, a figure arrived at by trial and error against rendering performance rather than derived from the hardware, and adjustable in configuration for stronger GPUs.

The reason the limit is a correctness requirement rather than a performance target is the 90 Hz VR comfort threshold. Below it, users get sick. That constraint pushes the expensive work out of the frame and into the load pipeline.

1. Parse the cube natively

   The FitsReader plug-in reads the FITS file into a float buffer outside the managed heap. Four-dimensional cubes have their spectral axis resolved automatically where it is unambiguous, and the user is prompted where it is not.

2. Search for downsampling factors that fit

   Three per-axis factors start at 1, 1, 1 and increment until the resulting texture falls under the budget, alternating between the two spatial axes and the spectral one so the cube degrades evenly rather than collapsing along a single dimension.

3. Aggregate in parallel

   Each output voxel becomes the mean or the maximum of its corresponding block in the source cube, selected by configuration. The largest cube benchmarked, 1536³, reduces to a 384 × 384 × 512 texture.

4. Keep the full resolution resident

   The full-resolution cube and mask stay in RAM. When a user crops to a region small enough to fit the budget, downsampling is skipped entirely and they get the real voxels, then can switch back to the overview without touching the disk again.

Median framerate stays close to the headset refresh rate across a sixfold range of native cube dimensions, because the adaptive downsampling absorbs the growth before the renderer sees it. The real cost of a larger dataset shows up in load time and memory instead: the 1536³ cube takes 45 seconds to load and peaks around 22 GB of private memory, against 2.9 seconds and 3.2 GB for a representative science cube.

Loading is all-or-nothing: a full 3D cube from a single header data unit. Very large cubes therefore cost long load times and enough downsampling to wash out features before the user can crop in. Memory scales with the full cube rather than the texture, so insufficient RAM sends the operating system to disk. Subcube bounds and HDU selection are the main targets for version 1.1.

The default maximum-value transfer function serves users looking for bright emission, but it discards absorption features even when they are statistically significant. A hybrid function that takes whichever of the maximum or minimum is larger in absolute value would address this, and a radiative-transfer-based shader would go further.

Visualisation and analysis are still tightly coupled inside the native plug-ins. Separating them so that analysis can arrive as third-party plug-ins is what would let other disciplines adopt the renderer without inheriting the astronomy.

iDaVIE is developed at the Inter-University Institute for Data Intensive Astronomy with collaborators across South Africa, Italy, the Netherlands, the United States and the United Kingdom. The version 1.0 paper carries twelve authors, and iDaVIE-v is developed in collaboration with INAF Osservatorio Astrofisico di Catania.

Although astronomy remains the primary application, the platform has also supported exploratory work with datasets from neuroscience, chemical engineering and astrobiology.