Accurate image alignment based on multi-warp optimization for large parallax

Image alignment with parallax is a challenging computer vision problem. While existing methods employing local-varying smooth warps have enhanced alignment accuracy for complex spacial deformations compared to global homography estimation, they fall short in adequately representing the discontinuous transformations evident in images with large parallax. In this paper, we propose an image alignment method grounded in the estimation and fusion of multiple warping models. To effectively tackle the challenges posed by discontinuous deformations in different regions, our method comprises two key stages: principal region alignment and fine region refinement. In both stages, multiple warping models are initially estimated using feature correspondences and are then concurrently optimized by minimizing pixel-level photometric loss. For each pixel, we select the optimal model that minimizes warping error. Additionally, we introduce a feature grouping method based on Delaunay triangulation. Experiments on real-world images demonstrate the superior alignment accuracy achieved by our proposed method compared to other state-of-the-arts.