Reshoot-Anything: A Self-Supervised Model for In-the-Wild Video Reshooting

Avinash Paliwal  ·  Adithya Iyer  ·  Shivin Yadav  ·  Muhammad Ali Afridi  ·  Midhun Harikumar

Morphic

Abstract

Precise camera control for reshooting dynamic videos is bottlenecked by the severe scarcity of paired multi-view data for non-rigid scenes. We overcome this limitation with a highly scalable self-supervised framework capable of leveraging internet-scale monocular videos. Our core contribution is the generation of pseudo multi-view training triplets, consisting of a source video, a geometric anchor, and a target video. We achieve this by extracting distinct smooth random-walk crop trajectories from a single input video to serve as the source and target views. The anchor is synthetically generated by forward-warping the first frame of the source with a dense tracking field, which effectively simulates the distorted point-cloud inputs expected at inference. Because our independent cropping strategy introduces spatial misalignment and artificial occlusions, the model cannot simply copy information from the current source frame. Instead, it is forced to implicitly learn 4D spatiotemporal structures by actively routing and re-projecting missing high-fidelity textures across distinct times and viewpoints from the source video to reconstruct the target. At inference, our minimally adapted diffusion transformer utilizes a 4D point-cloud derived anchor to achieve state-of-the-art temporal consistency, robust camera control, and high-fidelity novel view synthesis on complex dynamic scenes.

Training Data Construction

From a single monocular video we extract two independent crop trajectories (source & target) and synthetically generate a geometric anchor by forward-warping the first source frame with a dense tracking field.

Figure 1. Our self-supervised training data pipeline. Two independent smooth random-walk crop trajectories are extracted from a single input video to form source & target pseudo-views. The anchor is generated via dense-tracking-guided forward-warping, simulating the point-cloud anchors used at inference.

Pipeline in Action

Inference — input → anchor → reshooting result

Inference — input → anchor → reshooting result

Qualitative Results

Each row shows the original video (left) alongside our reshooting result (right) from a different camera trajectory.

Inference Example

Given one input video we reshoot it from two different anchor camera trajectories simultaneously.

Method Overview

Figure 2. Overview of our conditioning architecture. (1) VAE Encoding: The anchor video (V_a) and source video (V_s) are independently encoded into latents. (2) Conditioning Setup: The anchor latent is combined with a noise latent; the source latent uses an all-ones mask ensuring all content is leveraged. (3) DiT Processing: The two conditioned streams are temporally concatenated and jointly processed through the DiT blocks via self-attention, enabling fine-grained content routing. (4) Source Token Management: An auxiliary reconstruction loss on output source tokens ensures high-fidelity content retention.

Training Triplet Generation

Figure 3. Pseudo multi-view triplet generation. Two independent smooth random-walk crop trajectories are sampled from a single input video. The anchor is synthesized by forward-warping the source's first frame using a combined dense tracking + crop offset flow field.

Comparisons with State-of-the-Art

Our approach consistently outperforms existing methods in dynamic video reshooting, achieving leading performance across video quality, temporal consistency, camera accuracy, and view synchronization metrics.

Bold + underline = best in group. ↑ higher is better, ↓ lower is better.

Ablation Studies

We ablate core architectural and training choices. Our baseline uses a black anchor background, token concatenation through self-attention, and monocular videos without augmentations.

Figure 4. Qualitative ablation on a complex scene with moving smoke and colored lighting. The Synthetic Data Only model fails to capture dynamic smoke. The Cross-Attention model loses fine source details (saxophone texture) and struggles with anchor alignment. Our full model faithfully tracks the anchor while preserving high-fidelity textures and complex dynamics.

Bold + underline = best per column. ↑ higher is better, ↓ lower is better.

BibTeX

@article{paliwal2026reshootanything,
  title   = {Reshoot-Anything: A Self-Supervised Model for In-the-Wild Video Reshooting},
  author  = {Paliwal, Avinash and Iyer, Adithya and Yadav, Shivin and Afridi, Muhammad Ali and Harikumar, Midhun},
  journal = {arXiv preprint arXiv:2604.21776},
  year    = {2026},
  url     = {https://arxiv.org/abs/2604.21776}
}

Acknowledgements

We'd like to thank the members of Morphic for compute resources and motivation, and Dharmesh Kakadia and Isaac Wang for reviewing the paper.