David Nistér, Oleg Naroditsky, James Bergen · 2004
We present a system that estimates the motion of a stereo head or a single moving camera based on video input. The system operates in real-time with low delay and the motion estimates are used for navigational purposes. The front end of the system is a feature tracker. Point features are matched between pairs of frames and linked into image trajectories at video rate. Robust estimates of the camera motion are then produced from the feature tracks using a geometric hypothesize-and-test architecture. This generates what we call visual odometry, i.e. motion estimates from visual input alone. No prior knowledge of the scene nor the motion is necessary. The visual odometry can also be used in conjunction with information from other sources such as GPS, inertia sensors, wheel encoders, etc. The pose estimation method has been applied successfully to video from aerial, automotive and handheld platforms. We focus on results with an autonomous ground vehicle. We give examples of camera trajectories estimated purely from images over previously unseen distances and periods of time. 1.
Cited by 0
David Nistér, Oleg Naroditsky, James Bergen · 2004
We present a system that estimates the motion of a stereo head or a single moving camera based on video input. The system operates in real-time with low delay and the motion estimates are used for navigational purposes. The front end of the system is a feature tracker. Point features are matched between pairs of frames and linked into image trajectories at video rate. Robust estimates of the camera motion are then produced from the feature tracks using a geometric hypothesize-and-test architecture. This generates what we call visual odometry, i.e. motion estimates from visual input alone. No prior knowledge of the scene nor the motion is necessary. The visual odometry can also be used in conjunction with information from other sources such as GPS, inertia sensors, wheel encoders, etc. The pose estimation method has been applied successfully to video from aerial, automotive and handheld platforms. We focus on results with an autonomous ground vehicle. We give examples of camera trajectories estimated purely from images over previously unseen distances and periods of time. 1.
Cited by 0
Anders Hagnelius, Supervisor Cs-umu, Niclas Börlin · 2005
Visual odometry is the process of determining the position and orientation of a camera by analyzing a sequence of images from it. This can be useful for vehicle and robot navigation in locations where other techniques can not be used. In this master thesis a visual odometry system is implemented and evaluated. The system uses three low-cost web cameras mounted in a rig. The camera rig is calibrated i.e. the mutual position and orientation of the cameras are known. Two versions of the Förstner interest operator are used to find feature points in the images. The feature points are matched between the images and there corresponding object points are triangulated. The spatial matching is simplified with help of the known epipolar geometry. Four different triangulation methods are compared to see which gives the best precision of the system. The system uses a relation between the rigid body movement of the largest matchable subset of object points from two different camera positions. The optimal subset of object points is found by a ransac algorithm.
Cited by 0
D. Nistér, Oleg Naroditsky, James R. Bergen · 2004
We present a system that estimates the motion of a stereo head or a single moving camera based on video input. The system operates in real-time with low delay and the motion estimates are used for navigational purposes. The front end of the system is a feature tracker. Point features are matched between pairs of frames and linked into image trajectories at video rate. Robust estimates of the camera motion are then produced from the feature tracks using a geometric hypothesize-and-test architecture. This generates what we call visual odometry, i.e. motion estimates from visual input alone. No prior knowledge of the scene nor the motion is necessary. The visual odometry can also be used in conjunction with information from other sources such as GPS, inertia sensors, wheel encoders, etc. The pose estimation method has been applied successfully to video from aerial, automotive and handheld platforms. We focus on results with an autonomous ground vehicle. We give examples of camera trajectories estimated purely from images over previously unseen distances and periods of time.
Cited by 978
Sebastian P. Kleinschmidt, Bernardo Wagner · 2018
For autonomous localization and navigation, a robot's ego-motion estimation is fundamental. RGB camera-based visual odometry (VO) has proven to be a robust technique used to determine a robot's motion. In situations when direct sunlight, the absence of light or presence of dust as well as smoke make vision difficult, RGB cameras may not provide a sufficient number of RGB features for an accurate and robust visual odometry. In contrast to the visual spectrum of light, imaging modalities like thermal cameras can still be used to identify a stable but small number of image features in the described situations. Unfortunately, the smaller number of image features results in a less accurate VO. In this paper, we present an approach to monocular visual odometry using multimodal image features of different imaging modalities as RGB, thermal and hyperspectral images. By using the strengths of various imaging modalities, the robustness and accuracy of VO can be drastically increased compared to traditional unimodal approaches. The presented method merges different motion hypotheses based on various imaging modalities to create a more accurate motion estimation as well as a map of multimodal image features. The uni- and multimodal motion estimations are evaluated regarding the absolute and relative trajectory errors. The results show that our multimodal approach works robustly in presence of partial sensor failures still creating a multimodal map containing image features of all modalities.
Cited by 11
Kevin M. Judd, Jonathan D. Gammell · The International Journal of Robotics Research · 2024
Visual motion estimation is a well-studied challenge in autonomous navigation. Recent work has focused on addressing multimotion estimation in highly dynamic environments. These environments not only comprise multiple, complex motions but also tend to exhibit significant occlusion. Estimating third-party motions simultaneously with the sensor egomotion is difficult because an object’s observed motion consists of both its true motion and the sensor motion. Most previous works in multimotion estimation simplify this problem by relying on appearance-based object detection or application-specific motion constraints. These approaches are effective in specific applications and environments but do not generalize well to the full multimotion estimation problem (MEP). This paper presents Multimotion Visual Odometry (MVO), a multimotion estimation pipeline that estimates the full SE(3) trajectory of every motion in the scene, including the sensor egomotion, without relying on appearance-based information. MVO extends the traditional visual odometry (VO) pipeline with multimotion segmentation and tracking techniques. It uses physically founded motion priors to extrapolate motions through temporary occlusions and identify the reappearance of motions through motion closure. Evaluations on real-world data from the Oxford Multimotion Dataset (OMD) and the KITTI Vision Benchmark Suite demonstrate that MVO achieves good estimation accuracy compared to similar approaches and is applicable to a variety of multimotion estimation challenges.
Cited by 7Open access
Kevin H. Knuth, Ali Mohammad‐Djafari, Jean‐François Bercher et al. · AIP conference proceedings · 2011
Visual odometry refers to tracking the motion of a body using an onboard vision system. Practical visual odometry systems combine the complementary accuracy characteristics of vision and inertial measurement units. The Mars Exploration Rovers, Spirit and Opportunity, used this type of visual odometry. The visual odometry algorithms in Spirit and Opportunity were based on Bayesian methods, but a number of simplifying approximations were needed to deal with onboard computer limitations. Furthermore, the allowable motion of the rover had to be severely limited so that computations could keep up. Recent advances in computer technology make it feasible to implement a fully Bayesian approach to visual odometry. This approach combines dense stereo vision, dense optical flow, and inertial measurements. As with all true Bayesian methods, it also determines error bars for all estimates. This approach also offers the possibility of using Micro‐Electro Mechanical Systems (MEMS) inertial components, which are more economical, weigh less, and consume less power than conventional inertial components.
Cited by 4
Minjie Lin, Qixin Cao, Haoruo Zhang · 2018
Accurate visual odometry is essential for many fields such as robot navigation and autonomous driving .In this paper, we propose a novel panoramic visual odometry algorithm that is real-time, precise and robust .The main contributions of our work are a series of innovations that address the challenge of effective initialization and robust feature tracking based on panoramic camera. Wide field of view (FOV) is very important for robotic perception. Our algorithm takes advantage of the 360° FOV of a panoramic camera, which results in high accuracy of camera pose estimation and feature tracking stability. We use panoramic images directly without converting them to pinhole images. Through GPU acceleration, our implementation runs at an average 30 frames per second on a consumer laptop. In addition, we have done both indoor and outdoor experiments to validate proposed algorithm, the results show that ... we call our approach PVO (panoramic visual odometry).
Cited by 8
Davide Scaramuzza, Friedrich Fraundorfer · IEEE Robotics & Automation Magazine · 2011
Visual odometry (VO) is the process of estimating the egomotion of an agent (e.g., vehicle, human, and robot) using only the input of a single or multiple cameras attached to it. Application domains include robotics, wearable computing, augmented reality, and automotive. The term VO was coined in 2004 by Nister in his landmark paper [1]. The term was chosen for its similarity to wheel odometry, which incrementally estimates the motion of a vehicle by integrating the number of turns of its wheels over time. Likewise, VO operates by incrementally estimating the pose of the vehicle through examination of the changes that motion induces on the images of its onboard cameras. For VO to work effectively, there should be sufficient illumination in the environment and a static scene with enough texture to allow apparent motion to be extracted. Furthermore, consecutive frames should be captured by ensuring that they have sufficient scene overlap.
Cited by 1,468
Fanqi Xu, Yasin Almalıoğlu, Niki Trigoni · 2025
Learning-based Visual Odometry (VO) has seen significant advancements over the past decades. However, all the existing methods rely on the six degrees of freedom (6-DoF) representation for pose prediction, which is sparse and less conducive for neural network learning. In this work, we introduce a novel dense and distributed representation by modeling VO as ray bundles, referred to as RayVO. This richly parameterized representation is tightly coupled with corresponding spatial features, making it highly effective for neural learning. Additionally, the ray-based approach enables simultaneous prediction of both intrinsic and extrinsic parameters. To prove its effectiveness against the traditional 6-DoF representation, we propose three specialized loss functions for ray’s training: a ray-based loss, a 6-DoF-based loss and a hybrid loss. We extensively evaluate RayVO on both indoor and outdoor benchmark datasets and show that it outperforms the state-of-the-art VO methods.
Cited by 0
BEATRIX TUNG XUE LIN · National University of Singapore · 2025
Master's
Cited by 0
Simon Klenk, Marvin Motzet, Lukas Koestler et al. · arXiv (Cornell University) · 2023
Event cameras offer the exciting possibility of tracking the camera's pose during high-speed motion and in adverse lighting conditions. Despite this promise, existing event-based monocular visual odometry (VO) approaches demonstrate limited performance on recent benchmarks. To address this limitation, some methods resort to additional sensors such as IMUs, stereo event cameras, or frame-based cameras. Nonetheless, these additional sensors limit the application of event cameras in real-world devices since they increase cost and complicate system requirements. Moreover, relying on a frame-based camera makes the system susceptible to motion blur and HDR. To remove the dependency on additional sensors and to push the limits of using only a single event camera, we present Deep Event VO (DEVO), the first monocular event-only system with strong performance on a large number of real-world benchmarks. DEVO sparsely tracks selected event patches over time. A key component of DEVO is a novel deep patch selection mechanism tailored to event data. We significantly decrease the pose tracking error on seven real-world benchmarks by up to 97% compared to event-only methods and often surpass or are close to stereo or inertial methods. Code is available at https://github.com/tum-vision/DEVO
Cited by 0Open access
Xiang Gao, Tao Zhang · 2021
Cited by 0
Zach, Juri · Figshare · 2024
This work presents a novel self-supervised learning framework for deep visual odometry on stereo cameras.
Cited by 0Open access
Zach, Juri · Figshare · 2024
This work presents a novel self-supervised learning framework for deep visual odometry on stereo cameras.
Cited by 0Open access
Sachin Shah, Naitri Rajyaguru, Chahat Deep Singh et al. · IEEE Robotics and Automation Letters · 2024
Autonomous robots often rely on monocular cameras for odometry estimation and navigation. However, the scale ambiguity problem presents a critical barrier to effective monocular visual odometry. In this paper, we present CodedVO, a novel monocular visual odometry method that overcomes the scale ambiguity problem by employing custom optics to physically encode metric depth information into imagery. By incorporating this information into our odometry pipeline, we achieve state-of-the-art performance in monocular visual odometry with a known scale. We evaluate our method in diverse indoor environments and demonstrate its robustness and adaptability. We achieve a 0.08m average trajectory error in odometry evaluation on the ICL-NUIM indoor odometry dataset.
Cited by 0Open access
Daniel DeTone, Tomasz Malisiewicz, Andrew Rabinovich · arXiv (Cornell University) · 2018
We propose a self-supervised learning framework that uses unlabeled monocular video sequences to generate large-scale supervision for training a Visual Odometry (VO) frontend, a network which computes pointwise data associations across images. Our self-improving method enables a VO frontend to learn over time, unlike other VO and SLAM systems which require time-consuming hand-tuning or expensive data collection to adapt to new environments. Our proposed frontend operates on monocular images and consists of a single multi-task convolutional neural network which outputs 2D keypoints locations, keypoint descriptors, and a novel point stability score. We use the output of VO to create a self-supervised dataset of point correspondences to retrain the frontend. When trained using VO at scale on 2.5 million monocular images from ScanNet, the stability classifier automatically discovers a ranking for keypoints that are not likely to help in VO, such as t-junctions across depth discontinuities, features on shadows and highlights, and dynamic objects like people. The resulting frontend outperforms both traditional methods (SIFT, ORB, AKAZE) and deep learning methods (SuperPoint and LF-Net) in a 3D-to-2D pose estimation task on ScanNet.
Cited by 31Open access
Zheng Huai, Guoquan Huang · arXiv (Cornell University) · 2018
In this paper, we propose a novel robocentric formulation of the visual-inertial navigation system (VINS) within a sliding-window filtering framework and design an efficient, lightweight, robocentric visual-inertial odometry (R-VIO) algorithm for consistent motion tracking even in challenging environments using only a monocular camera and a 6-axis IMU. The key idea is to deliberately reformulate the VINS with respect to a moving local frame, rather than a fixed global frame of reference as in the standard world-centric VINS, in order to obtain relative motion estimates of higher accuracy for updating global poses. As an immediate advantage of this robocentric formulation, the proposed R-VIO can start from an arbitrary pose, without the need to align the initial orientation with the global gravitational direction. More importantly, we analytically show that the linearized robocentric VINS does not undergo the observability mismatch issue as in the standard world-centric counterpart which was identified in the literature as the main cause of estimation inconsistency. Additionally, we investigate in-depth the special motions that degrade the performance in the world-centric formulation and show that such degenerate cases can be easily compensated in the proposed robocentric formulation, without resorting to additional sensors as in the world-centric formulation, thus leading to better robustness. The proposed R-VIO algorithm has been extensively tested through both Monte Carlo simulations and real-world experiments with different sensor platforms navigating in different environments, and shown to achieve better (or competitive at least) performance than the state-of-the-art VINS, in terms of consistency, accuracy and efficiency.
Cited by 6Open access
Yuxiang Peng, Chuchu Chen, Guoquan Huang · 2024
As edge devices equipped with cameras and inertial measurement units (IMUs) are emerging, it holds huge implications to endow these mobile devices with spatial computing capability. However, ultra-efficient visual-inertial estimation at the size, weight and power (SWAP)-constrained edge devices to provide accurate 3D motion tracking remains challenging. This is exacerbated by data transfer (between different processors and memory) that consumes significantly more energy than computing itself. To push the state of the art, this paper proposes the first-of-its-kind quantized visual-inertial odometry (QVIO) to offer energy-efficient 3D motion tracking. In particular, we first quantize raw visual measurements in an intuitive way with a given small number of bits and then perform an EKF update with these quantized measurements (termed zQVIO). To improve this ad-hoc quantizer (although it works well in practice), we systematically quantize each measurement residual into a single bit and perform maximum-a-posterior (MAP) estimation. measurements. Thanks to these quantizers, the proposed QVIO estimators significantly reduce the data transfer and thus improve energy efficiency. As shown in our extensive experiments, the proposed residual-quantized VIO (rQVIO) achieves remarkably competing performance even when using an average of only 3.7 bits per measurement, equivalent to a data reduction of 8.6 times compared to transmitting single-precision measurements.
Cited by 8
Peter Hansen, Hatem Alismail, Peter Rander et al. · 2013
Cited by 0