Autostitch - Panorama Creation
Stitch images together to create seamless panoramas.
This project implements a complete panorama creation pipeline including spherical warping, feature matching with RANSAC alignment, and advanced blending techniques to combine multiple overlapping photographs into a single seamless panoramic image.
Project Details
| Language: | Python with NumPy, SciPy, OpenCV |
| Key Concepts: | RANSAC, homographies, spherical warping, image blending |
Overview
Panorama creation is a fundamental computer vision task that combines multiple overlapping photographs into a single wide-angle view. This project implements the complete autostitch pipeline, from raw photographs to seamless panoramic images.
The system supports two different alignment approaches: translation-based alignment using spherically warped images (ideal for 360° panoramas), and homography-based alignment for direct image registration (suitable for planar scenes and handheld photography).
Pipeline Architecture
Complete Autostitch Pipeline
- Image Acquisition: Capture overlapping photographs (50% overlap recommended)
- [Optional] Spherical Warping: Transform to spherical coordinates for 360° panoramas
- Feature Extraction: Use ORB detector to find discriminative keypoints
- Feature Matching: Find correspondences between overlapping images
- RANSAC Alignment: Robust estimation of inter-image transformations
- Global Alignment: Chain transformations and correct for drift
- Image Blending: Seamless composition with feathering weights
- Final Processing: Crop and optimize the resulting panorama
Implementation Details
Part 1: Spherical Warping
For 360° panoramas and wide-angle sequences, spherical warping corrects for the curved nature of the camera's field of view.
Spherical Coordinate Transformation:
Radial Distortion Correction:
Camera Parameters Used:
Campus Dataset:f = 595, k1 = -0.15, k2 = 0.00
Yosemite Dataset:
f = 678, k1 = -0.21, k2 = 0.26
Key Insight: Spherical warping linearizes the motion between camera positions, making translation-based alignment much more effective for wide-angle sequences.
Part 2: Motion Models and Alignment
The system supports two motion models for different scenarios:
Translation Model
Use Case: 360° panoramas with spherical warping
DOF: 2 parameters (tx, ty)
RANSAC: 1 correspondence needed
Assumption: Pure camera rotation
Homography Model
Use Case: Planar scenes, handheld photography
DOF: 8 parameters (3×3 matrix)
RANSAC: 4 correspondences needed
Assumption: Planar world or distant scenes
Part 3: RANSAC-Based Alignment
Robust estimation of inter-image transformations using Random Sample Consensus (RANSAC).
RANSAC Algorithm:
- Sample: Randomly select minimal correspondence set
- Estimate: Compute transformation from minimal set
- Evaluate: Count inliers using distance threshold
- Iterate: Repeat for maximum number of trials
- Optimize: Least-squares refinement on best inlier set
Homography Estimation via SVD:
SVD Solution: For the equation Ah = 0, the solution h is the right singular vector corresponding to the smallest singular value of A.
Part 4: Image Blending and Composition
Seamless composition of aligned images using advanced blending techniques.
Feathering Blend
Distance-based weighting function for smooth transitions between images
Alpha Compositing
Weighted accumulation with proper normalization to avoid artifacts
Drift Correction
Linear warping to eliminate vertical misalignment in 360° panoramas
Blending Pipeline:
- Bounding Box Calculation: Determine final panorama dimensions
- Inverse Warping: Resample each image to final coordinates
- Weight Accumulation: Apply feathering weights during composition
- Normalization: Divide by accumulated weights to avoid artifacts
- [360° Only] Seam Blending: Perfect alignment at left/right edges
Feathering Weight Function:
My Results
Implementation Achievements
Successfully implemented the complete autostitch pipeline with both translation and homography-based alignment methods.
Pipeline Visualization
Spherical Warping Results
Final Panorama Results
Technical Challenges and Solutions
Key Implementation Challenges
- Homogeneous Coordinate Handling: Proper normalization when converting back to Cartesian coordinates
- Inverse Warping: Avoiding black pixels and implementing linear interpolation
- Memory Management: Efficient processing of high-resolution image sequences
- Drift Correction: Maintaining alignment in 360° panoramas with accumulating errors
- Exposure Differences: Handling illumination variations between images
Key Learnings
Computer Vision Concepts
- Geometric Transformations: Deep understanding of homographies, translations, and spherical projections
- Robust Estimation: RANSAC algorithm for handling outliers in correspondence matching
- Image Warping: Forward vs. inverse warping, interpolation, and coordinate transformations
- Multi-Image Alignment: Global optimization and drift correction in image sequences
- Computational Photography: Blending algorithms and seamless image composition
Mathematical Insights
- SVD Applications: Using singular value decomposition for homography estimation
- Projective Geometry: Working with homogeneous coordinates and projective transformations
- Optimization Theory: Least-squares refinement and robust statistical estimation
- Spherical Geometry: Coordinate system transformations and distortion models
Extra Credit Explorations
Advanced Techniques Implemented
Exposure Compensation
Automatic brightness adjustment between images
Ghost Removal
Handling moving objects in overlapping regions
Advanced Blending
Multi-resolution pyramid blending for seamless transitions
Applications and Impact
Real-World Applications
Virtual Reality: Creating immersive 360° environments for VR experiences
Architectural Documentation: Complete building documentation and virtual tours
Surveillance Systems: Wide-area monitoring with multiple camera feeds
Scientific Imaging: Large-scale imaging in astronomy, microscopy, and remote sensing
Mobile Photography: Automatic panorama modes in smartphone cameras