Deep CNN-Based Framework For Enhanced Aerial Imagery Registration with Applications to UAV Geolocalization

Ahmed Nassar, Karim Amer, Reda ElHakim, Mohamed ElHelw

Vegard Bergsvik Øvstegård

Fri - 18 Oct 2019

Components

Input images

Input images

• On-board camera(Bird’s eye view)
  • Video frame$_i = S_{i}$
• Reference image(Satellite Map)
  • $R$
  • Known GPS-bounds
• Not Global
  • Cheating with starting positon $= S_1$

$Maths..$

$R$

Calibration

Calibration

• Where is $S_i$ in $R$?
• Affine geometric transformation
  • Map $S_i$ with FOV consideration
    • Scale
    • Orientation
• Remember -> framework is cheating
  • Reduced search space: $r = R(l, w_{s+b})$
• Calibrates on $S_1$ and every $S_{(i+3)}$
• Scale Invariant Feature Transform(SIFT)
• Random Sample Consensus(RANSAC) -> Homography matrix
  • Describes movement of UAV related to R.

SIFT

Homography

Sequential frame registration

Sequential frame registration

• Similar to Calibration stage
• Every non-calibration frame
• Uses ORB -> Homography -> Translation

Oriented FAST and Rotated BRIEF(ORB)

ORB

• Two orders of magnitude faster than SIFT
• Does not handle scale variance robustly

Semantic Segmentation Using U-net

U-net

U-net

Semantic Shape Matching (SSM)

Processing U-net frames

1. Fill gaps(Dilation -> Erosion)
2. Remove outlier noise(Erosion - Dilation)
3. Build dictionary of shapes
4. Brute force matching shapes
5. Check matches

• Calculate homography based on matched shapes
• Adjust location

Semantic Shape Matching (SSM)

• Morphology
  • Buildings
  • Roads

Experiments and Results

• No real-life testing
• Only tested on two cities
  • Potsdam
    • Simulated flight with Google Earth
    • 6.3m average drift for local feature mapping
    • 3.6m average drift for SSM
  • Famagusta
    • YouTube Video
    • 10.4m average drift local feature mapping
    • 5.1m average drift for SSM
    • Lower resolution for $R$ and $S$
• Used manually-labelled GPS coordinates!

Experiments and Results

Experiments and Results

Experiments and Results

• Robustness?
• Computation?

Print

Back