VR-360 Outpainting — teaching a video model to see in every direction at once
I've been fascinated with equirectangular video since 2015, when a colleague in Berlin built his own 360° camera rig out of Sony action cameras and a 3D-printed frame, strapped it to a weather balloon, and launched it into the sky. Watching the footage afterwards felt like magic — especially his stitching algorithm, which turned a cluster of overlapping flat views into one seamless sphere. When I asked him how it worked, he shrugged and said it's just math.
That moment planted something. Eventually I circled back to the question of how you make a machine see in every direction at once.
The long arc
By spring 2025, consumer 360° cameras had gotten good and I'd gotten curious about what generative video models could do with equirectangular footage. Together with great teammates in ADOS Hackathon in Paris in spring 2025, I trained text-to-video and image-to-video LoRAs on top of the LTX model. That work won the model hacking category at the ADOS hackathon, which was encouraging, but the models were fundamentally limited: they could generate 360° clips from scratch, but naturally the quality left a lot to wish for at the stage the OS video models were back then. There wasn't even a concept of IC-LoRAs back then so the models you couldn't even dream of taking an existing flat video and expand it into a full sphere.
That turned out to be a much harder problem. And it's the one I decided to solve for ADOS 2026.
The geometry problem
Here's why it's hard. Ask a video model to "just make this 360°" and it fails, because it has no way to know how a flat rectangle of pixels sits on the inside of a sphere. It's being asked to solve two problems at once: figure out the spherical geometry, and complete the picture. Unsurprisingly, it does neither well.
The insight — which came to me in April 2026 — was to hand the model that geometry explicitly. Take the flat input video, apply an inverse gnomonic projection that maps it correctly onto a black equirectangular canvas at the right position and field of view, and give that to the model as the conditioning input. Now the model only has one job: "complete the rest of the sphere." It can see exactly where the known pixels sit on the sphere, and what the rectilinear perspective looks like in spherical space. The outpainting task becomes filling in everything else, consistent with that correctly mapped window.
This projection step is applied identically during training and at inference. I built a companion set of ComfyUI nodes — ComfyUI-VR-Outpaint-Tools — that handles the projection from flat footage and includes automatic FOV estimation, so users don't need to measure their camera's field of view by hand.
The unglamorous part
The proof-of-concept, trained at ADOS Paris with collaborators Cseti, NebSH, and S4f3ty_Marc, went up on Hugging Face as v0.1 and picked up 33 likes and 122 downloads organically. The trick worked. But when I sat down to build a proper production dataset, I discovered something deflating.
Of the original source videos, roughly 70% were unusable. Projection errors. Burned-in logos. Camera operators and tripods visible in the shot. Drones creeping into frame. I spent hours manually watching 360° footage, clip by clip, flagging everything that would teach the model the wrong thing.
This is the part of ML engineering that doesn't make it into the launch thread. The quality ceiling of your model isn't set by GPU hours — it's set by how carefully you look at your data before training even starts. You need to understand how the model learns, and make sure you are not guiding it into the wrong direction with your data.
I rebuilt the dataset from the ground up: 51 CC-BY-licensed YouTube channels, 4,427 clean clips, augmented by rendering each into 12 reference variants (4 fields of view × 3 aspect ratios) to teach the model to generalize across different framings. That's 53,124 total training pairs generated, of which 13,281 were actually used in the end, with a good randomized selection across the augmentations. Every source channel was individually verified as Creative Commons Attribution-licensed before inclusion, and the license proof was archived alongside the footage — which is why the model card is able to credit 46 individual creators by name with direct channel links.
The run
The production training run — v2, the one that became the 1.0 release — happened over three days in summer 2026 on a rented RTX PRO 6000 Blackwell GPU, with compute sponsored by Lightricks, the company behind the base LTX-2.3 model. The LoRA itself is 654 million trainable parameters (rank/alpha 128, targeting self-attention, cross-attention, and feed-forward layers).
The model converged by step ~3,000 and I let it run to 9,000 just to confirm the plateau held. It did. Loss is a terrible quality signal for this kind of model — the numbers look fine long before the wrap seam actually disappears. I had to build two custom metrics instead: a seam continuity score that measures how well the left and right edges of the equirectangular output match up, and a weight delta tracker to see how much the model was still changing between checkpoints. Both are detailed in the model card, but the short version is: the seam ratio dropped from ~60 at step 1,000 to ~6 at step 3,000, and stayed in the 5–9 band through step 9,000. A residual ~5–6 is the noise floor of the data and config. That is to say, this model does not produce seamless equirectangular videos.
What came out of it
Three checkpoints are released (steps 5000, 7000, and 9000), all fully converged with subtle differences. Step 7000 is the recommended one — best balance of equirectangular consistency and scene variety. The model performs best on semi-static establishing shots (cityscapes, landscapes, slow pans) at ~90–110° FOV with widescreen to cinemascope aspect ratios. It generalizes beyond those conditions, but quality degrades gracefully the further you drift from the training distribution.
The top and bottom caps — zenith and nadir — remain the hardest regions. Fast motion, whip pans, and handheld shake easily break the wrap seam. And the model is trained on outdoor and architectural scenes; interiors and organic subjects work less reliably, though they draw on the base model's own training.
The weights, workflow file, and companion ComfyUI tools are all linked from the Hugging Face model card. The old v0.1 PoC assets are preserved in an _old/ folder — I wanted the release to have a visible before/after, rather than pretending the PoC never happened.
What's next
I'm currently working on a method for seamless equirectangular video generation — 360° video from scratch, rather than outpainting from existing footage. No public timeline yet, but the geometry lessons from this project carry forward directly. I have also started working on a second LoRA in this theme, for VR180 stereo video as a companion to this one.
The weather balloon over Berlin turned out to be the start of a very long trail. Ten years later, I think I've finally caught up to the question it left me with.
---
Model card & weights: huggingface.co/TheBurgstall/VR-360-Outpaint-LTX2.3-IC-LoRA Companion ComfyUI tools: github.com/Burgstall-labs/ComfyUI-VR-Outpaint-Tools Base model: Lightricks/LTX-2.3-22B