# AirSLAM: An Efficient and Illumination-Robust Point-Line Visual SLAM System Kuan Xu , Yuefan Hao, Shenghai Yuan , Member, IEEE, Chen Wang , Senior Member, IEEE, and Lihua Xie , Fellow, IEEE Abstract—In this article, we present an efficient visual simultaneous localization and mapping (SLAM) system designed to tackle both short-term and long-term illumination challenges. Our system adopts a hybrid approach that combines deep learning techniques for feature detection and matching with traditional back-end optimization methods. Specifically, we propose a unified convolutional neural network that simultaneously extracts keypoints and structural lines. These features are then associated, matched, triangulated, and optimized in a coupled manner. In addition, we introduce a lightweight relocalization pipeline that reuses the built map, where keypoints, lines, and a structure graph are used to match the query frame with the map. To enhance the applicability of the proposed system to real-world robots, we deploy and accelerate the feature detection and matching networks using C++ and NVIDIA TensorRT. Extensive experiments conducted on various datasets demonstrate that our system outperforms other state-of-the-art visual SLAM systems in illumination-challenging environments. Efficiency evaluations show that our system can run at a rate of 73 Hz on a PC and 40 Hz on an embedded platform. Index Terms—Mapping, relocalization, visual simultaneous localization and mapping (SLAM). ## I. INTRODUCTION is essential for robot navigation due to its favorable balance between cost and accuracy [1]. Compared to LiDAR SLAM, vSLAM utilizes more cost-effective and compact sensors to achieve accurate localization, thus broadening its range of potential applications [2]. Moreover, cameras can capture richer and more detailed information, which enhances their potential for providing robust localization. Despite the recent advancements, the present vSLAM systems still struggle with severe lighting conditions [3], [4], [5], [6], which can be summarized into two categories. First, feature detection and tracking often fail due to drastic changes or low light, severely affecting the quality of the estimated trajectory [7], [8]. Second, when the visual map is reused for relocalization, lighting variations could significantly reduce the success rate [9], [10]. In this article, we refer to the first issue as the short-term illumination challenge, which impacts pose estimation between two temporally adjacent frames, and the second as the long-term illumination challenge, which affects matching between the query frame and an existing map. Present methods usually focus on only one of the above challenges. For example, various image enhancement [11], [12], [13] and image normalization algorithms [14], [15] have been developed to ensure robust tracking. These methods primarily focus on maintaining either global or local brightness consistency, yet they often fall short of handling all types of challenging lighting conditions [16]. Some systems have addressed this issue by training a visual odometry (VO) or SLAM network on large datasets containing diverse lighting conditions [17], [18], [19]. However, they have difficulty producing a map suitable for long-term localization. Some methods can provide illuminationrobust relocalization, but they usually require map building under good lighting conditions [20], [21]. In real-world robot applications, these two challenges often arise simultaneously, necessitating a unified system capable of addressing both. Furthermore, many of the aforementioned systems incorporate intricate neural networks, relying on powerful GPUs to run in real-time. They lack the efficiency necessary for deployment on resource-constrained platforms, such as warehouse robots. These limitations impede the transition of vSLAM from laboratory research to industrial applications. In response to these gaps, this article introduces AirSLAM. Observing that line features can improve the accuracy and robustness of vSLAM systems [5], [22], [23], we integrate both point and line features for tracking, mapping, optimization, and relocalization. To achieve a balance between efficiency and performance, we design our system as a hybrid system, employing learning-based methods for feature detection and matching, and traditional geometric approaches for pose and map optimization. In addition, to enhance the efficiency of feature detection, we developed a unified model capable of simultaneously detecting point and line features. We also address long-term localization challenges by proposing a multistage relocalization strategy, which effectively reuses our point-line map. In summary, our contributions are as follows. 1) We propose a novel point line-based vSLAM system that combines the efficiency of traditional optimization techniques with the robustness of learning-based methods. Our system is resilient to both short-term and long-term illumination challenges while remaining efficient enough for deployment on embedded platforms. 2) We develop a unified model for both keypoint and line detection, which we call PLNet. To our knowledge, PLNet is the first model capable of simultaneously detecting both point and line features. Furthermore, we associate these two types of features and jointly utilize them for tracking, mapping, and relocalization tasks. 3) We propose a multistage relocalization method based on both point and line features, utilizing both appearance and geometry information. This method can provide fast and illumination-robust localization in an existing visual map using only a single image. 4) We conduct extensive experiments to demonstrate the efficiency and effectiveness of the proposed methods. The results show that our system achieves accurate and robust mapping and relocalization performance under various illumination-challenging conditions. In addition, our system is also very efficient. It runs at a rate of 73 H on a PC and 40 Hz on an embedded platform. In addition, our engineering contributions include deploying and accelerating feature detection and matching networks using C++ and NVIDIA TensorRT, facilitating their deployment on real robots. We release all the source code at https://github.com/ sair-lab/AirSLAM to benefit the community. This article extends our conference paper, AirVO [22]. AirVO utilizes SuperPoint [24] and LSD [25] for feature detection, and SuperGlue [26] for feature matching. It achieves remarkable performance in environments with changing illumination. However, as a visual-only odometry, it primarily addresses short-term illumination challenges and cannot reuse a map for drift-free relocalization. In addition, despite carefully designed postprocessing operations, the modified LSD is still not stable enough for long-term localization. It relies on image gradient information rather than environmental structural information, rendering it susceptible to varying lighting conditions. In this version, we introduce substantial improvements as follows. 1) We design a unified CNN to detect both point and line features, enhancing the stability of feature detection in illumination-challenging environments. In addition, the more efficient LightGlue [27] is used for feature matching. 2) We extend our system to support both stereo-only and stereo-inertial data, increasing its reliability when an inertial measurement unit (IMU) is available. 3) We incorporate loop closure detection and map optimization, forming a complete vSLAM system. 4) We design a multistage relocalization module based on both point and line features, enabling our system to effectively handle long-term illumination challenges. The rest of this article is organized as follows. In Section II, we discuss the relevant literature. Section III-B presents an overview of the complete system pipeline. The proposed PLNet is presented in Section IV. In Section V, we introduce the visual-inertial odometry based on PLNet. Section VI shows how to optimize the map offline and reuse it online. The detailed experimental results are presented in Section VII to verify the efficiency, accuracy, and robustness of AirSLAM. Finally, Section VIII concludes this article. ## II. RELATED WORK ## A. Keypoint and Line Detectionfor vSLAM 1) Keypoint Detection: Various handcrafted keypoint features e.g., ORB [28], FAST [29], and BRISK [30], have been proposed and applied to VO and vSLAM systems. They are usually efficient but not robust enough in challenging environments [9], [26]. With the development of deep learning techniques, more and more learning-based features are proposed and used to replace the handcrafted features in vSLAM systems. Rong et al. [31] introduced TFeat network [32] to extract descriptors for FAST corners and apply it to a traditional vSLAM pipeline. Tang et al. [33] used a neural network to extract robust keypoints and binary feature descriptors with the same shape as the ORB. Han et al. [34] combined SuperPoint [24] feature extractor with a traditional back-end. Bruno and Colombini [35] proposed LIFT-SLAM, where they use LIFT [36] to extract features. Li et al. [37] replaced the ORB feature with SuperPoint in ORB-SLAM2 and optimized the feature extraction with the Intel OpenVINO toolkit. Zhan et al. [38] proposed a new selfsupervised training scheme for learning-based features, using bundle adjustment and bilevel optimization as a supervision signal. Some other learning-based features, e.g., R2D2 [39] and DISK [40], and D2Net [41], are also being attempted to be applied to vSLAM systems, although they are not yet efficient enough [42], [43]. 2) Line Detection: Currently, most point-line-based vSLAM systems use the LSD [25] or EDLines [44] to detect line features because of their good efficiency [5], [45], [46], [47], [48]. Although many learning-based line detection methods, e.g., SOLD2 [49], AirLine [50], and HAWP [51], have been proposed and shown better robustness in challenging environments, they are difficult to apply to real-time vSLAM systems due to lacking efficiency. For example, Kannapiran et al. [23] proposed StereoVO, where they choose SuperPoint [24] and SOLD2 [49] to detect keypoints and line segments, respectively. Despite achieving good performance in dynamic lighting conditions, StereoVO can only run at a rate of about 7 Hz on a good GPU. ## B. Short-Term Illumination Challenge Several handcrafted methods have been proposed to improve the robustness of VO and vSLAM to challenging illumination. DSO [52] models brightness changes and jointly optimizes camera poses and photometric parameters. DRMS [11] and AFE-ORB-SLAM [12] utilize various image enhancements. Some systems try different methods, such as ZNCC, the locallyscaled sum of squared differences (LSSD), and dense descriptor computation, to achieve robust tracking [14], [15], [53]. These methods mainly focus on either global or local illumination change for all kinds of images, however, lighting conditions often affect the scene differently in different areas [16]. Other related methods include that of Huang and Liu [54], which presents a multifeature extraction algorithm to extract two kinds of image features when a single-feature algorithm fails to extract enough feature points. Kim et al. [55] employed a patchbased affine illumination model during direct motion estimation. Chen and Heckman [56] minimized the normalized information distance with nonlinear least square optimization for image registration. Alismail et al. [57] proposed a binary feature descriptor using a descriptor assumption to avoid brightness constancy. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/78ced2416ef27902373881f5e84a2d42e03a126aa2598f58ee890ede71927e00.jpg) Fig. 1. Proposed system consists of three main parts: Online stereo VO/VIO, offline map optimization, and online relocalization. The VO/VIO module uses the mapping image sequences to build an initial map. Then, the initial map is processed offline and an optimized map is outputted. The optimized map can be used for the one-shot relocalization. Compared with handcrafted methods, learning-based methods have shown better performance. Savinykh et al. [8] proposed DarkSLAM, where generative adversarial network [58] was used to enhance input images. Singh et al. [59] compared different learning-based image enhancement methods for vSLAM in low-light environments. TartanVO [17], DROID-SLAM [18], and iSLAM [19] train their VO or SLAM networks on the TartanAir dataset [60], which is a large simulation dataset that contains various lighting conditions, therefore, they are very robust in challenging environments. However, they usually require good GPUs and long training times. Besides, DROID-SLAM runs very slowly and is difficult to apply to real-time applications on resource-constrained platforms. TartanVO and iSLAM are more efficient, but they cannot achieve performance as accurately as traditional vSLAM systems. ## C. Long-Term Illumination Challenge Currently, most SLAM systems still use the bag of words (BoW) [61] for loop closure detection and relocalization due to its good balance between efficiency and effectiveness [4], [62], [63]. To make the relocalization more robust to large illumination variations, Labbé and Michaud [20] proposed the multisession relocalization method, where they combined multiple maps generated at different times and in various illumination conditions. DXSLAM [37] uses NetVLAD [64] for the coarse image retrieval and SuperPoint with a binary descriptor for keypoint matching between the query frame and candidates. Another similar task in the robotics and computer vision communities is the visual place recognition (VPR) problem, where many researchers handle the localization problem with image retrieval methods [64], [65]. These VPR solutions try to find images most similar to the query image from a database. They usually cannot directly provide accurate pose estimation which is needed in robot applications. Sarlin et al. [9] addressed this and proposed Hloc. They use a global retrieval to obtain several ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/248a36843e9c1fccc1c4d158cff37a037ba71f5789a2121f2e2558d5fe736a31.jpg) Fig. 2. We visualize the feature map (top right) and detected keypoints (bottom left) of a keypoint detection model, and the detected structural lines (bottom right) of a line detection model. The overlap of keypoints and junctions, and the edge information in the feature map inspire the design of our PLNet. Hloc toolbox has integrated many image retrieval methods, local feature extractors, and matching methods, and it is currently the SOTA system. Yan et al. [66] proposed a long-term visual localization method for mobile platforms, however, they relied on other sensors, e.g., GPS, compass, and gravity sensor, for the coarse location retrieval. ## III. SYSTEM OVERVIEW ## A. Notations In this article, R represents the set of real numbers, and Rm denotes the m-dimensional real vector space. The transpose of a vector or matrix is written as $( \cdot ) ^ { \top }$ . For a vector $\mathbf { x } \in \mathbb { R } ^ { m }$ $\| \mathbf { x } \|$ represents its Euclidean norm, while $\| \mathbf { x } \| _ { \Sigma } ^ { 2 }$ is shorthand for $\mathbf { x } ^ { \top } \Sigma \mathbf { x }$ . The transformation, rotation, and translation from the b-coordinate system to the a-coordinate system are denoted by $\mathbf { T } _ { a b } \in \mathrm { S E } ( 3 ) , \mathbf { R } _ { a b } \in \mathrm { S O } ( 3 )$ , and $\mathbf { t } _ { a b } \in \mathbb { R } ^ { 3 }$ , respectively. We use $( \cdot ) _ { c }$ to indicate a vector in the camera frame and $( \cdot ) _ { w }$ to indicate a vector in the global world frame. Throughout the article, super/subscripts may be omitted for brevity, as long as the meaning remains unambiguous within the given context. ## B. System Architecture We believe that a practical vSLAM system should possess the following features. 1) High Efficiency: The system should have real-time performance on resource-constrained platforms. 2) Scalability: The system should be easily extensible for various purposes and real-world applications. 3) Easy to Deploy: The system should be easy to deploy on real robots and capable of achieving robust localization. Therefore, we design a system as shown in Fig. 1. The proposed system is a hybrid system as we need the robustness of data-driven approaches and the accuracy of geometric methods. It consists of the following three components. 1) Stereo VO/VIO: We propose a point line-based visual odometry that can handle both stereo and stereo-inertial ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/f68c1c99d7d22e98bbf7533ec184dbb6f2cf9b80526a2ac0a28d059162d45133.jpg) Fig. 3. Framework of the proposed PLNet. It consists of the shared backbone, the keypoint module, and the line module. 2) Offline Map Optimization: We implement several commonly used plugins, such as loop detection, pose graph optimization, and global bundle adjustment. The system is easily extensible for other map-processing purposes by adding customized plugins. For example, we have implemented a plugin to train a scene-dependent junction vocabulary using the endpoints of line features, which is utilized in our lightweight multistage relocalization. 3) Lightweight Relocalization: We propose a multistage relocalization method that improves efficiency while maintaining effectiveness. In the first stage, keypoints and line features are detected using the proposed PLNet, and several candidates are retrieved using a keypoint vocabulary trained on a large dataset. In the second stage, most false candidates are quickly filtered out using a scene-dependent junction vocabulary and a structure graph. In the third stage, feature matching is performed between the query frame and the remaining candidates to find the best match and estimate the pose of the query frame. Since feature matching in the third stage is typically time-consuming, the filtering process in the second stage enhances the efficiency of our system compared to other two-stage relocalization systems. We transfer some time-consuming processes, e.g., loop closure detection, pose graph optimization, and global bundle adjustment, to the offline stage. This improves the efficiency of our online mapping module. In many practical applications, such as warehouse robotics, a map is typically built by one robot and then reused by others. Our system is designed with these applications in mind. The lightweight mapping and map reuse modules can be easily deployed on resource-constrained robots, while the offline optimization module can run on a more powerful computer for various map manipulations, such as map editing and visualization. The mapping robot uploads the initial map to the computer, which then distributes the optimized map to other robots, ensuring drift-free relocalization. In the following sections, we introduce our feature detection and VO pipeline in Section IV and Section V, respectively. The offline optimization and relocalization modules are presented in Section VI. ## IV. FEATURE DETECTION ## A. Motivation With advancements in deep learning technology, learningbased feature detection methods have demonstrated more stable performance in illumination-challenging environments compared to traditional methods. However, existing point-linebased VO/VIO and SLAM systems typically detect keypoints and line features separately. While it is acceptable for handcrafted methods due to their efficiency, the simultaneous application of keypoint detection and line detection networks in VO/VIO or SLAM systems, especially in stereo configurations, often hinders real-time performance on resource-constrained platforms. Consequently, we aim to design an efficient unified model that can detect keypoints and line features concurrently. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/f2d5d24f3b4ca54baa282f3e2b559e18cec2ada618c142cd93a6da1d63ff3daf.jpg) Fig. 4. We use four parameters, d, θ, $\theta _ { 1 } , \theta _ { 2 } ,$ , to encode a line into a point p within the attraction region field. However, achieving a unified model for keypoint and line detection is challenging, as these tasks typically require different real-image datasets and training procedures. Keypoint detection models are generally trained on large datasets comprising diverse images and depend on either a boosting step or the correspondences of image pairs for training [24], [39], [40]. For line detection, we find wireframe parsing methods [51], [67] can provide stronger geometric cues than the self-supervised models [49], [68] as they are able to detect longer and more complete lines. The wireframe includes all prominent straight lines and their junctions within the scene, providing an efficient and accurate representation of large-scale geometry and object shapes [69]. However, these methods are trained on the Wireframe dataset [69], which is limited in size with only 5462 discontinuous images. In the following sections, we will address this challenge and demonstrate how to train a unified model capable of performing both tasks. It is important to note that in this article, the term “line detection” refers specifically to the wireframe parsing task. ## B. Architecture Design As shown in Fig. 2, we have two findings when visualizing the results of the keypoint and line detection networks: 1) mostjunctions (endpoints oflines) detected by the line detection model are also selected as keypoints by the keypoint detection model; 2) the feature maps outputted by the keypoint detection model contain the edge information. Therefore, we argue that a line detection model can be built on the backbone of a pretrained keypoint detection model. Based on this assumption, we design the PLNet to detect keypoints and lines in a unified framework. As shown in Fig. 3, it consists of the shared backbone, the keypoint module, and the line module. Backbone: We follow SuperPoint [24] to design the backbone for its good efficiency and effectiveness. It uses 8 convolutional layers and 3 max-pooling layers. The input is the grayscale image sized $H \times W$ . The outputs are $H \times \bar { W } \times 6 4 , \frac { \bar { H } } { 2 } \times \frac { W } { 2 } \times 6 4$ $\begin{array} { r } { \frac { \hat { H } } { 4 } \times \frac { W } { 4 } \times 1 2 8 , \frac { H } { 8 } \times \frac { W } { 8 } \times 1 2 8 } \end{array}$ feature maps. Keypoint Module: We also follow SuperPoint [24] to design the keypoint detection header. It has two branches: the score branch and the descriptor branch. The inputs are $\textstyle { \frac { H } { 8 } } \times { \frac { W } { 8 } } \times 1 2 8$ feature maps outputted by the backbone. The score branch outputs a tensor sized $\frac { H } { 8 } \times \frac { W } { 8 } \times 6 5$ . The 65 channels correspond to an 8 × 8 grid region and a dustbin indicating no keypoint. The tensor is processed by a softmax and then resized to $H \times W$ . The descriptor branch outputs a tensor sized $\textstyle { \frac { H } { 8 } } \times { \frac { W } { 8 } } \times 2 5 6$ , which is used for interpolation to compute descriptors of keypoints. Line Module: This module takes feature maps from the backbone as inputs. It consists of a U-Net-like CNN and the line detection header. We modify the U-Net [70] to make it contain fewer convolutional layers and thus be more efficient. The U-Net-like CNN is to increase the receptive field as detecting lines requires a larger receptive field than detecting keypoints. The EPD LOIAlign [51] is used to process the outputs of the line module and finally outputs junctions and lines. ## C. Network Training Due to the training problem described in Section IV-A and the assumption in Section IV-B, we train our PLNet in two rounds. In the first round, only the backbone and the keypoint detection module are trained, which means we need to train a keypoint detection network. In the second round, the backbone and the keypoint detection module are fixed, and we only train the line detection module on the Wireframe dataset. We skip the details of the first round as they are very similar to [24]. Instead, we present the training of the line detection module. Line Encoding: We adopt the attraction region field [51] to encode line segments. As shown in Fig. 4, for a line segment $\bf { l } = ( x _ { 1 } , x _ { 2 } )$ , where $\mathbf { x _ { 1 } }$ and $\mathbf { x _ { 2 } }$ are two endpoints of l, and a point p in the attraction region of l, four parameters and p are used to encode l: $$ \mathbf { p } \left( 1 \right) = \left( d , \theta , \theta _ { 1 } , \theta _ { 2 } \right)\tag{1} $$ where d is the distance from p to l, θ is the angle between l and the v-axis of the image, $\theta _ { 1 }$ is the angle between $\mathbf { p x _ { 1 } }$ and the perpendicular line from p to l, and $\theta _ { 2 }$ is the angle between px2 and the perpendicular line. The network can predict these four parameters for point p and then l can be decoded through $$ \left. \begin{array} { r l } { { \bf l } = d \cdot \left[ \cos \theta } & { { } - \sin \theta \right] \left[ \begin{array} { c c } { { \bf l } } & { { \bf l } } \\ { { \tan \theta _ { 1 } } } & { { \tan \theta _ { 2 } } } \end{array} \right] + \left[ { \bf p } \quad { \bf p } \right] . } \end{array} \right.\tag{2} $$ Line Prediction: The line detection module outputs a tensor sized $\textstyle { \frac { H } { 4 } } \times { \frac { W } { 4 } } \times 4$ to predict parameters in (1) and a heatmap to predict junctions. For each decoded line segment by (2), two junctions closest to its endpoints will be selected to form a line proposal with it. Proposals with the same junctions will be deduplicated and only one is retained. Then, the EPD LOIAlign [51] and a head classifier are applied to decide whether the line proposal is a true line feature. Line Module Training: We use the L1 loss to supervise the prediction of parameters in (1) and the binary cross-entropy loss to supervise the junction heatmap and the head classifier. The total loss is the sum of them. As shown in Fig. 5, to improve the robustness of line detection in illumination-challenging environments, seven types of photometric data augmentation are applied to process training images. The training uses the ADAM optimizer [71] with the learning rate $l r = 4 e { - } 4$ in the first 35 epochs and $l r = 4 e { - } 5$ in the last 5 epochs. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/5d845a6fd19f5f15769ed5243880f9c3f90c9fd2903b5ca9609d493ddff2c28a.jpg) Fig. 5. We use seven types of photometric data augmentation to train our PLNet to make it more robust to challenging illumination. ## V. STEREO VISUAL ODOMETRY ## A. Overview The proposed point line-based stereo visual odometry is shown in Fig. 6. It is a hybrid VO system utilizing both the learning-based front-end and the traditional optimization backend. For each stereo image pair, we first employ the proposed PLNet to extract keypoints and line features. Then, a GNN (LightGlue [27]) is used to match keypoints. In parallel, we associate line features with keypoints and match them using the keypoint matching results. After that, we perform an initial pose estimation and reject outliers. Based on the results, we triangulate the 2-D features of keyframes and insert them into the map. Finally, the local bundle adjustment will be performed to optimize points, lines, and keyframe poses. In the meantime, if an IMU is accessible, its measurements will be processed using the IMU preintegration method [72], and added to the initial pose estimation and local bundle adjustment. Applying both learning-based feature detection and matching methods to the stereo VO is time-consuming. Therefore, to improve efficiency, the following three techniques are utilized in our system. 1) for keyframes, we extract features on both left and right images and perform stereo matching to estimate the real scale. But for nonkeyframes, we only process the left image. Besides, we use some lenient criteria to make the selected keyframes in our system very sparse, so the runtime and resource consumption of feature detection and matching in our system are close to that of a monocular system; 2) we convert the inference code of the CNN and GNN from Python to C++, and deploy them using ONNX and NVIDIA TensorRT, where the 16-bit floating-point arithmetic replaces the 32-bit floating-point arithmetic; 3) we design a multithread pipeline. A producer-consumer model is used to split the system into two main threads, i.e., the front-end thread and the back-end thread. The front-end thread extracts and matches features while the back-end thread performs the initial pose estimation, keyframe insertion, and local bundle adjustment. ## B. Feature Matching We use LightGlue [27] to match keypoints. For line features, most of the current VO and SLAM systems use the LBD algorithm [73] or tracking sample points to match them. However, the LBD algorithm extracts the descriptor from a local band region of the line, so it suffers from unstable line detection due to challenging illumination or viewpoint changes. Tracking sample points can match the line detected with different lengths in two frames, but current SLAM systems usually use optical flow to track the sample points, which have a bad performance when the light conditions change rapidly or violently. Some learning-based line feature descriptors [49] are also proposed, however, they are rarely used in current SLAM systems due to the increased time complexity. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/d3afb4ba99ab34c6f93331ad6df04b3ad58aea8770dbef62f60484ff2024a067.jpg) Fig. 6. Framework of our visual(-inertial) odometry. The system is split into two main threads, which are represented by two different colored regions. Note tha the IMU input is not strictly required. The system is optional to use stereo data or stereo-inertial data. Therefore, to address both the effectiveness problem and efficiency problem, we design a fast and robust line-matching method for illumination-challenging conditions. First, we associate keypoints with line segments through their distances. Assume that M keypoints and N line segments are detected on the image, where the ith keypoint is denoted as $\mathbf { p } _ { i } = ( x _ { i } , y _ { i } )$ and the jth line segment is denoted as $1 _ { j } =$ $( A _ { j } , B _ { j } , C _ { j } , x _ { j , 1 } , y _ { j , 1 } , x _ { j , 2 } , y _ { j , 2 } )$ , where $( A _ { j } , B _ { j } , C _ { j } )$ are line parameters of $\mathbf { \dot { l } } _ { j }$ and $( x _ { j , 1 } , y _ { j , 1 } , x _ { j , 2 } , y _ { j , 2 } )$ are the endpoints. We first compute the distance between $\mathbf { p } _ { i }$ and $1 _ { j }$ through $$ d _ { i j } = d \left( \mathbf { p } _ { i } , \mathbf { l } _ { j } \right) = \frac { \left| A _ { j } \cdot x _ { i } + B _ { j } \cdot y _ { i } + C _ { j } \right| } { \sqrt { A _ { j } ^ { 2 } + B _ { j } ^ { 2 } } } .\tag{3} $$ If $d _ { i j } < 3$ and the projection of $\mathbf { p } _ { i }$ on the coordinate axis lies within the projections of line segment endpoints, i.e., $\operatorname* { m i n } ( x _ { j , 1 } , x _ { j , 2 } ) \leq x _ { i } \leq \operatorname* { m a x } ( x _ { j , 1 } , x _ { j , 2 } )$ or min $( y _ { j , 1 } , y _ { j , 2 } ) \le$ $y _ { i } \le \operatorname* { m a x } ( y _ { j , 1 } , y _ { j , 2 } )$ , we will say $\mathbf { p } _ { i }$ belongs to $1 _ { j } .$ Then, the line segments on two images can be matched based on the point-matching result of these two images. For ${ \mathbf { l } } _ { k , m }$ on image $k$ and $^ { 1 _ { k + 1 , n } }$ on image $k + 1$ , we compute a score $S _ { m n }$ to represent the confidence of that they are the same line $$ S _ { m n } = \frac { N _ { p m } } { \operatorname* { m i n } ( N _ { k , m } , N _ { k + 1 , n } ) }\tag{4} $$ where $N _ { p m }$ is the matching number between point features belonging to ${ \mathbf { l } } _ { k , m }$ and point features belonging to $\mathrm { l } _ { k + 1 , n } . \ : N _ { k , m }$ and $N _ { k + 1 , n }$ are the numbers of point features belonging to l and $\mathbf { l } _ { k + 1 , n } ,$ respectively. Then, if $S _ { m n } > \delta _ { S }$ and $N _ { p m } > \delta _ { N }$ where $\delta _ { S }$ and $\delta _ { N }$ are two preset thresholds, we will regard $\mathbf { l } _ { k , m }$ and $^ { 1 _ { k + 1 , n } }$ as the same line. This coupled feature matching method allows our line matching to share the robust performance of keypoint matching while being highly efficient due to that it does not need another line-matching network. ## C. 3-D Feature Processing In this part, we will introduce our 3-D feature processing methods, including 3-D feature representation, triangulation, i.e., constructing 3-D features from 2-D features, and reprojection, i.e., projecting 3-D features to the image plane. For 3-D point processing, a 3-D point is denoted as $\mathbf { X } \in \mathbb { R } ^ { 3 }$ . Light-Glue [27] is utilized to match keypoints between the left and right images. Successfully matched keypoints are triangulated using stereo disparity information, while unmatched keypoints are triangulated leveraging multiframe observations. The projection of 3-D points onto the image plane is modeled using either the pinhole camera model or the fisheye camera model, depending on the specific camera in use. We skip the details of 3-D point processing in our system as they are similar to other point-based VO and SLAM systems [4], [74]. On the contrary, compared with 3-D points, 3-D lines have more degrees of freedom, and they are easier to degenerate when being triangulated. Therefore, the 3-D line processing will be illustrated in detail. 1) 3-D Line Representation: We use Plücker coordinates [75] to represent a 3-D spatial line $$ \mathbf { L } = { \binom { \mathbf { n } } { \mathbf { v } } } \in \mathbb { R } ^ { 6 }\tag{5} $$ where v is the direction vector of the line and n is the normal vector of the plane determined by the line and the origin. Plücker coordinates are used for 3-D line triangulation, transformation, and projection. It is over-parameterized because it is a 6-D vector, but a 3-D line has only four degrees of freedom. In the graph optimization stage, the extra degrees of freedom will increase the computational cost and cause the numerical instability of the system [76]. Therefore, we also use orthonormal representation [75] to represent a 3-D line $$ ( \mathbf { U } , \mathbf { W } ) \in \mathrm { S O } ( 3 ) \times \mathrm { S O } ( 2 ) .\tag{6} $$ The relationship between Plücker coordinates and orthonormal representation is similar to SO(3) and so(3). Orthonormal representation can be obtained from Plücker coordinates by $$ \mathbf { L } = [ \mathbf { n } \mid \mathbf { v } ] = \underbrace { \left[ { \frac { \mathbf { n } } { \| \mathbf { n } \| } } { \frac { \mathbf { v } } { \| \mathbf { v } \| } } { \frac { \mathbf { n } \times \mathbf { v } } { \| \mathbf { n } \times \mathbf { v } \| } } \right] } _ { \mathbf { U } \in \mathrm { S O } ( 3 ) } \underbrace { \left[ \| \mathbf { n } \| \quad \mathbf { 0 } \right] } _ { \Sigma _ { 3 \times 2 } }\tag{7} $$ where $\Sigma _ { 3 \times 2 }$ is a diagonal matrix and its two nonzero entries defined up to scale can be represented by an SO(2) matrix $$ \mathbf { W } = { \frac { 1 } { \sqrt { \left\| \mathbf { n } \right\| ^ { 2 } + \left\| \mathbf { v } \right\| ^ { 2 } } } } \left[ \left\| \mathbf { n } \right\| \ - \left\| \mathbf { v } \right\| \right] \in \mathrm { S O } ( 2 ) .\tag{8} $$ In practice, this conversion can be done simply and quickly with the QR decomposition. 2) Triangulation: Triangulation is to initialize a 3-D line from two or more 2-D line features. In our system, we use two methods to triangulate a 3-D line. The first is similar to the line triangulation algorithm B in [77], where the pose of a 3-D line can be computed from two planes. To achieve this, we select two line segments, $\mathbf { l } _ { 1 }$ and $^ { 1 _ { 2 } , }$ on two images, which are two observations of a 3-D line. Note that the two images can come from the stereo pair of the same keyframe or two different keyframes. $\mathbf { l } _ { 1 }$ and $1 _ { 2 }$ can be back-projected and construct two 3-D planes, $\pi _ { 1 }$ and $\pi _ { 2 }$ . Then, the 3-D line can be regarded as the intersection of $\pi _ { 1 }$ and $\pi _ { 2 }$ However, triangulating a 3-D line is more difficult than triangulating a 3-D point, because it suffers more from degenerate motions [77]. Therefore, we also employ a second line triangulation method if the above method fails, where points are utilized to compute the 3-D line. In Section V-B, we have associated point features with line features. So to initialize a 3-D line, two triangulated points $\mathbf { X } _ { 1 }$ and $\mathbf { X } _ { 2 }$ , which belong to this line and have the shortest distance from this line on the image plane are selected. Then, the Plücker coordinates of this line can be obtained through $$ \mathbf { L } = \left[ \mathbf { \bar { v } } \right] = \left[ \mathbf { \bar { X } } _ { 1 } \times \mathbf { \bar { X } } _ { 2 } \right] .\tag{9} $$ This method requires little extra computation because the selected 3-D points have been triangulated in the point triangulating stage. It is very efficient and robust. 3) Reprojection: Reprojection is used to compute the reprojection errors. We use Plücker coordinates to transform and reproject 3-D lines. First, we convert the 3-D line from the world frame to the camera frame $$ \mathbf { L } _ { c } = \left[ \mathbf { n } _ { c } \right] = \left[ \mathbf { R } _ { c w } \quad \left[ \mathbf { t } _ { c w } \right] _ { \times } \mathbf { R } _ { c w } \right] \left[ \mathbf { n } _ { w } \right] = \mathbf { H } _ { c w } \mathbf { L } _ { w }\tag{10} $$ where $\mathbf { L } _ { c }$ and $\mathbf { L } _ { w }$ are Plücker coordinates of 3D line in the camera frame and world frame, respectively. $\mathbf { R } _ { c w } \in \mathrm { S O } ( 3 )$ is the rotation matrix from world frame to camera frame and $\mathbf { t } _ { c w } \in \mathbb { R } ^ { 3 }$ is the translation vector. $[ \cdot ] _ { \times }$ denotes the skew-symmetric matrix ofa vector and $\mathbf { H } _ { c w }$ is the transformation matrix of3D lines from world frame to camera frame. Then, the 3-D line $\mathbf { L } _ { c }$ can be projected to the image plane through a line projection matrix $\mathbf { P } _ { c }$ $$ 1 = { \binom { A } { B } } = \mathbf { P } _ { c } \mathbf { L } _ { c [ : 3 ] } = { \left[ \begin{array} { l l l } { f _ { x } } & { 0 } & { 0 } \\ { 0 } & { f _ { y } } & { 0 } \\ { - f _ { y } c _ { x } } & { - f _ { x } c _ { y } } & { f _ { x } f _ { y } } \end{array} \right] } \mathbf { n } _ { c }\tag{11} $$ where $\mathbf { l } = [ A \quad B \quad C ] ^ { \top }$ is the reprojected 2-D line on image plane. $\mathbf { L } _ { c [ : 3 ] }$ donates the first three rows of vector $\mathbf { L } _ { c }$ ## D. Keyframe Selection Observing that the learning-based data association method used in our system is able to track two frames that have a large baseline, so different from the frame-by-frame tracking strategy used in other VO or SLAM systems, we only match the current frame with the last keyframe. We argue this strategy can reduce the accumulated tracking error. Therefore, the keyframe selection is essential for our system. On the one hand, as described in Section V-A, we want to make keyframes sparse to reduce the consumption of computational resources. On the other hand, the sparser the keyframes, the more likely tracking failure happens. To balance the efficiency and the tracking robustness, a frame will be selected as a keyframe if any of the following conditions is satisfied. 1) The tracked features are less than $\alpha _ { 1 } \cdot N _ { s }$ 2) The average parallax of tracked features between the current frame and the last keyframe is larger than $\alpha _ { 2 } \cdot \sqrt { W H }$ 3) The number of tracked features is less than $N _ { k f }$ In the above, $\alpha _ { 1 } , \alpha _ { 2 } .$ , and $N _ { k f }$ are all preset thresholds. $N _ { s }$ is the number of detected features. W and H, respectively, represent the width and height of the input image. ## E. Local Graph Optimization To improve the accuracy, we perform the local bundle adjustment when a new keyframe is inserted. $N _ { o }$ latest neighboring keyframes are selected to construct a local graph, where map points, 3-D lines, and keyframes are vertices and pose constraints are edges. We use point constraints and line constraints as well as IMU constraints if an IMU is accessible. Their related error terms are defined as follows. 1) Point Reprojection Error: If the frame i can observe the 3-D map point $\mathbf { X } _ { p }$ , then the reprojection error is defined as $$ \mathbf { r } _ { i , X _ { p } } = \tilde { \mathbf { x } } _ { i , p } - \pi \left( \mathbf { R } _ { c w } \mathbf { X } _ { p } + \mathbf { t } _ { c w } \right)\tag{12} $$ where $\tilde { \mathbf { x } } _ { i , p }$ is the observation of $\mathbf { X } _ { p }$ on frame i and $\pi ( \cdot )$ represents the camera projection. 2) Line Reprojection Error: If the frame i can observe the 3-D line $\mathbf { L } _ { q }$ , then the reprojection error is defined as $$ \mathbf { r } _ { i , L _ { q } } = e _ { l } \left( \widetilde { \mathbf { l } } _ { i , q } , \mathbf { P } _ { c } \left( \mathbf { H } _ { c w } \mathbf { L } _ { q } \right) _ { [ : 3 ] } \right) \in \mathbb { R } ^ { 2 } ,\tag{13a} $$ $$ e _ { l } \left( \tilde { \mathbf { l } } _ { i , q } , \mathbf { l } _ { i , q } \right) = \left[ d \left( \tilde { \mathbf { p } } _ { i , q 1 } , \mathbf { l } _ { i , q } \right) \quad d \left( \tilde { \mathbf { p } } _ { i , q 2 } , \mathbf { l } _ { i , q } \right) \right] ^ { \top }\tag{13b} $$ where $\tilde { \mathbf { l } } _ { i , q }$ is the observation of $\mathbf { L } _ { q }$ on frame $i , \tilde { { \bf p } } _ { i , q 1 }$ and $\tilde { \mathbf { p } } _ { i , q 2 }$ are the endpoints of $\tilde { \mathrm { l } } _ { i , q } ,$ and $d ( \mathbf { p } , \mathbf { l } )$ is the distance between point p and line l which is computed through (3). 3) IMU Residuals: We first follow [72] to preintegrate IMU measurements between the frame i and the frame j $$ \Delta \tilde { \mathbf { R } } _ { i j } = \prod _ { k = i } ^ { j - 1 } \mathrm { E x p } \Big ( \Big ( \tilde { \omega } _ { k } - \mathbf { b } _ { k } ^ { g } - \pmb { \eta } _ { k } ^ { g d } \Big ) \Delta t \Big ) ,\tag{14a} $$ $$ \Delta \tilde { \mathbf { v } } _ { i j } = \sum _ { k = i } ^ { j - 1 } \Delta \tilde { \mathbf { R } } _ { i k } \Big ( \tilde { \mathbf { a } } _ { k } - \mathbf { b } _ { k } ^ { a } - \pmb { \eta } _ { k } ^ { a d } \Big ) \Delta t ,\tag{14b} $$ $$ \Delta \tilde { \mathbf { p } } _ { i j } = \sum _ { k = i } ^ { j - 1 } \left( \Delta \tilde { \mathbf { v } } _ { i k } \Delta t + \frac { 1 } { 2 } \Delta \tilde { \mathbf { R } } _ { i k } \Big ( \tilde { \mathbf { a } } _ { k } - \mathbf { b } _ { k } ^ { a } - \boldsymbol { \eta } _ { k } ^ { a d } \Big ) \Delta t ^ { 2 } \right)\tag{14c} $$ where $\tilde { \omega } _ { k }$ and $\tilde { \mathbf { a } } _ { k }$ are, respectively, the angular velocity and the acceleration. $\mathbf { b } _ { k } ^ { g }$ and ${ \bf b } _ { k } ^ { a }$ are biases of the sensor and they are modeled as constants between two keyframes through $\mathbf { b } _ { k } ^ { g } =$ $\mathbf { b } _ { k + 1 } ^ { g }$ and $\mathbf { b } _ { k } ^ { a } = \mathbf { b } _ { k + 1 } ^ { a } . \eta _ { k } ^ { g d }$ and $\eta _ { k } ^ { a d }$ are Gaussian noises. Then IMU residuals are defined as $$ \mathbf { r } _ { \Delta R _ { i j } } = \mathrm { L o g } \left( \left( \Delta \tilde { \mathbf { R } } _ { i j } \mathrm { E x p } \left( \frac { \partial \Delta \mathbf { R } _ { i j } } { \partial \mathbf { b } ^ { g } } \delta \mathbf { b } ^ { g } \right) \right) ^ { \top } \mathbf { R } _ { i } ^ { \top } \mathbf { R } _ { j } \right) ,\tag{15a} $$ $$ \begin{array} { l } { { \displaystyle { \bf r } _ { \Delta v _ { i j } } = { \bf R } _ { i } ^ { \top } \left( { \bf v } _ { j } - { \bf v } _ { i } - { \bf g } \Delta t _ { i j } \right) } \ ~ } \\ { { \displaystyle ~ - \left( \Delta \tilde { \bf v } _ { i j } + \frac { \partial \Delta { \bf v } _ { i j } } { \partial { \bf b } ^ { g } } \delta { \bf b } ^ { g } + \frac { \partial \Delta { \bf v } _ { i j } } { \partial { \bf b } ^ { a } } \delta { \bf b } ^ { a } \right) } , } \end{array}\tag{15b} $$ $$ \begin{array} { l } { { \displaystyle { \bf r } _ { \Delta p _ { i j } } = { \bf R } _ { i } ^ { \top } \left( { \bf p } _ { j } - { \bf p } _ { i } - { \bf v } _ { i } \Delta t _ { i j } - \frac { 1 } { 2 } { \bf g } \Delta t _ { i j } ^ { 2 } \right) } } \\ { { \displaystyle ~ - \left( \Delta \tilde { \bf p } _ { i j } + \frac { \partial \Delta { \bf p } _ { i j } } { \partial { \bf b } ^ { g } } \delta { \bf b } ^ { g } + \frac { \partial \Delta { \bf p } _ { i j } } { \partial { \bf b } ^ { a } } \delta { \bf b } ^ { a } \right) } , } \end{array}\tag{15c} $$ $$ \mathbf { r } _ { b _ { i j } } = \left[ \left( \mathbf { b } _ { j } ^ { g } - \mathbf { b } _ { i } ^ { g } \right) ^ { \top } \quad \left( \mathbf { b } _ { j } ^ { a } - \mathbf { b } _ { i } ^ { a } \right) ^ { \top } \right] ^ { \top }\tag{15d} $$ Where g is the gravity vector in world coordinates. In our system, we combine the initialization process in [4] and [62] to estimate g and initial values of biases. The factor graph is optimized by the g2o toolbox [78]. The cost function is defined as $$ \begin{array} { r l } { \mathbf { E } = \displaystyle \sum \| \mathbf { r } _ { i , X _ { p } } \| _ { \Sigma _ { P } } ^ { 2 } + \sum \| \mathbf { r } _ { i , L _ { q } } \| _ { \Sigma _ { L } } ^ { 2 } + \sum \| \mathbf { r } _ { \Delta R _ { i j } } \| _ { \Sigma _ { \Delta R } } ^ { 2 } } & { { } } \\ { + \displaystyle \sum \| \mathbf { r } _ { \Delta v _ { i j } } \| _ { \Sigma _ { \Delta v } } ^ { 2 } + \sum \| \mathbf { r } _ { \Delta p _ { i j } } \| _ { \Sigma _ { \Delta p } } ^ { 2 } + \sum \| \mathbf { r } _ { b _ { i j } } \| _ { \Sigma _ { b } } ^ { 2 } . } \end{array}\tag{16a} $$ We use the Levenberg-Marquardt optimizer to minimize the cost function. The point and line outliers are also rejected in the optimization if their corresponding residuals are too large. ## F. Initial Map As described in Section III-B, our map is optimized offline. Therefore, keyframes, map points, and 3-D lines will be saved to the disk for subsequent optimization when the visual odometry is finished. For each keyframe, we save its index, pose, keypoints, keypoint descriptors, line features, andjunctions. The correspondences between 2-D features and 3-D features are also recorded. To make the map faster to save, load, and transfer across different devices, the above information is stored in binary form, which also makes the initial map much smaller than the raw data. For example, on the OIVIO dataset [79], our initial map size is only about 2% of the raw data size. ## VI. MAP OPTIMIZATION AND REUSE ## A. Offline Map Optimization This part aims to process an initial map generated by our VO module and outputs the optimized map that can be used for drift-free relocalization. Our offline map optimization module consists of the following several map-processing plugins. 1) Loop Closure Detection: Similar to most current vSLAM systems, we use a coarse-to-fine pipeline to detect loop closures. Our loop closure detection relies on DBoW2 [61] to retrieve candidates and LightGlue [27] to match features. We train a vocabulary for the keypoint detected by our PLNet on a database that contains 35 k images. These images are selected from several large datasets [80], [81], [82] that include both indoor and outdoor scenes. The vocabulary has 4 layers, with 10 nodes at each layer, so it contains 10 000 words. Coarse Candidate Selection: This step aims to find three candidates most similar to a keyframe $\kappa _ { i }$ from a set $S _ { 1 } = \{ \boldsymbol { K } _ { j } \mid j <$ $i \}$ . Note that we do not add keyframes with an index greater than $\kappa _ { i }$ to the set because this may miss some loop pairs. We build a co-visibility graph for all keyframes where two are connected if they obverse at last one feature. All keyframes connected with $\kappa _ { i }$ will be first removed from $S _ { 1 }$ . Then we compute a similarity score between $\kappa _ { i }$ and each keyframe in $S _ { 1 }$ using DBoW2. Only keyframes with a score greater than $0 . 3 \cdot S _ { \mathrm { m a x } }$ will be kept in $S _ { 1 }$ , where $S _ { \mathrm { m a x } }$ is the maximum computed score. After that, we group the remaining keyframes. If two keyframes can observe more than ten features in common, they will be in the same group. For each group, we sum up the scores of the keyframes in this group and use it as the group score. Only the top three groups with the highest scores will be retained. Then, we select one keyframe with the highest score within the group as the candidate from each group. These three candidates will be processed in the subsequent steps. Fine Feature Matching: For each selected candidate, we match its features with $\kappa _ { i } .$ . Then the relative pose estimation with outlier rejection will be performed. The candidate will form a valid loop pair with $\kappa _ { i }$ if the inliers exceed 50. 2) Map Merging: A 3-D feature observed by both frames of a loop pair is usually mistakenly used as two features. Therefore, in this part, we aim to merge the duplicated point and line features observed by loop pairs. For keypoint features, we use the above feature-matching results between loop pairs. If two matched keypoints are associated with two different map points, they will be regarded as duplicated features and only one map point will be retained. The correspondence between 2-D keypoints and 3-D map points, as well as the connections in the co-visibility graph, will also be updated. For line features, we first associate 3-D lines and map points through the 2-D–3-D feature correspondence and 2-D point-line association built in Section V-B. Then, we detect 3-D line pairs that associate with the same map points. If two 3-D lines share more than three associated map points, they will be regarded as duplicated and only one 3-D line will be retained. 3) Global Bundle Adjustment: We perform the global bundle adjustment (GBA) after merging duplicated features. The form of the residuals and loss functions is similar to that in Section V-E; however, unlike Section V-E, all keyframes and features are jointly optimized, and loop closure residuals are also incorporated into the optimization process. In the initial stage of optimization, the reprojection errors of merged features are relatively large due to the VO drift error, so we first iterate 50 times without outlier rejection to optimize the variables to a good rough position, and then iterate another 40 times with outlier rejection. We find that when the map is large, the initial 50 iterations can not optimize the variables to a satisfactory position. To address this, we first perform pose graph optimization (PGO) before the global bundle adjustment if a map contains more than 80 k map points. Only the keyframe poses will be adjusted in the PGO and the cost function is defined as follows: $$ \mathbf { E } _ { p g o } = \sum \| \mathrm { L o g } \left( \Delta \tilde { \mathbf { T } } _ { i j } ^ { - 1 } \mathbf { T } _ { i } ^ { - 1 } \mathbf { T } _ { j } \right) \| _ { \Sigma _ { i j } } ^ { 2 }\tag{17} $$ where $\mathbf { T } _ { i } \in \mathrm { S E } ( 3 )$ and $\mathbf { T } _ { j } \in \mathrm { S E } ( 3 )$ are poses of $\kappa _ { i }$ and $\boldsymbol { \mathcal { K } } _ { j }$ , respectively. $\operatorname { L o g } ( \cdot ) = \log ( \cdot ) ^ { \vee } : \operatorname { S E } ( 3 ) \to \operatorname { s e } ( 3 )$ is the Logarithm map proposed in [83]. $\kappa _ { i }$ and $\kappa _ { j }$ should either be adjacent or form a loop pair. After the pose graph optimization, the positions of map points and 3-D lines will also be adjusted along with the keyframes in which they are first observed. The systems with online loop detection usually perform the GBA after detecting a new loop, so they undergo repeated GBAs when a scene contains many loops. In contrast, our offline map optimization module only does the GBA after all loop closures are detected, allowing us to reduce the optimization iterations significantly compared with them. 4) Scene-Dependent Vocabulary: We train a junction vocabulary aiming to be used for relocalization. The vocabulary is built on the junctions of keyframes in the map so it is scenedependent. Compared with the keypoint vocabulary trained in Section VI-A1, the database used to train thejunction vocabulary is generally much smaller, so we set the number of layers to 3, with 10 nodes in each layer. The junction vocabulary is tiny, i.e., about 1 MB, as it only contains 1000 words. Its detailed usage will be introduced in Section VI-B. 5) Optimized Map: We save the optimized map for subsequent map reuse. Compared with the initial map in Section V-F, more information is saved such as the bag of words for each keyframe, the global co-visibility graph, and the scene-dependent junction vocabulary. In the meantime, the number of 3-D features has decreased due to the fusion of duplicate map points and 3-D lines. Therefore, the optimized map occupies a similar memory to the initial map. ## B. Map Reuse In this part, we present our illumination-robust relocalization using an existing optimized map. In most vSLAM systems, recognizing revisited places typically needs two steps: 1) retrieving $N _ { k c }$ keyframe candidates; and 2) performing feature matching and estimating relative pose. The second step is usually time consuming, so selecting a proper $N _ { k c }$ is very important. A larger $N _ { k c }$ will reduce the system’s efficiency while a smaller $N _ { k c }$ may prevent the correct candidate from being recalled. For example, in the loop closing module of ORB-SLAM3 [62], only the three most similar keyframes retrieved by DBoW2 [61] are used for better efficiency. It works well as two frames in a loop pair usually have a short time interval and thus the lighting conditions are relatively similar. But for challenging tasks, such as the day/night relocalization problem, retrieving so few candidates usually results in a low recall rate. However, retrieving more candidates needs to perform feature matching and pose estimation more times for each query frame, which makes it difficult to deploy for real-time applications. To address this problem, we propose an efficient multistage relocalization method to make the optimized map usable in different lighting conditions. Our insight is that if most of the false candidates can be quickly filtered out, then the efficiency can be improved while maintaining or even improving the relocalization recall rate. Therefore, we add another step to the two-step pipeline mentioned above. We next introduce the proposed multistage pipeline in detail. 1) First Step: This step is to retrieve the similar keyframes in the map that are similar to the query frame. For each input monocular image, we detect keypoints, junctions, and line features using our PLNet. Then a pipeline similar to the “coarse candidate selection” in Section VI-A1 will be executed, but with two differences. The first difference is that we do not filter out candidates using the co-visibility graph as the query frame is not in the graph. The second is that all candidates, not just three, will be retained for the next step. 2) Second Step: This step filters out most of the candidates selected in the first step using junctions and line features. For query frame $\textstyle { \mathcal { K } } _ { q }$ and each candidate $\kappa _ { b } .$ , we first match their junctions by finding the same words through thejunction vocabulary trained in Section VI-A4. We use $\{ ( q _ { i } , b _ { i } ) \mid q _ { i } \in \mathcal { K } _ { q } , b _ { i } \in \mathcal { K } _ { b } \}$ to denote the matching pairs. Then we construct two structure graphs, i.e., $G _ { q } ^ { J }$ and $G _ { b } ^ { J }$ , for $\textstyle { \mathcal { K } } _ { q }$ and $\displaystyle { \mathcal { K } } _ { b }$ , respectively. The vertices are matched junctions, i.e., $V _ { q } ^ { J } = \{ q _ { i } \ | \ q _ { i } \in \mathcal { K } _ { q } \}$ and $V _ { b } ^ { J } = \{ b _ { i } \mid b _ { i } \in { \mathcal { K } } _ { b } \}$ . The related adjacent matrices that describe the connection between vertices are defined as $$ \mathbf { A } _ { q } ^ { J } = \left[ \begin{array} { c c c } { q _ { 1 1 } } & { \cdots } & { q _ { 1 n } } \\ { \vdots } & { \ddots } & { \vdots } \\ { q _ { n 1 } } & { \cdots } & { q _ { n n } } \end{array} \right] , \mathbf { A } _ { b } ^ { J } = \left[ \begin{array} { c c c } { b _ { 1 1 } } & { \cdots } & { b _ { 1 n } } \\ { \vdots } & { \ddots } & { \vdots } \\ { b _ { n 1 } } & { \cdots } & { b _ { n n } } \end{array} \right]\tag{18} $$ where n is the number of junction-matching pairs. $q _ { i j }$ is set to 1 if the junction $q _ { i }$ and $q _ { j }$ are two endpoints of the same line, otherwise, it is set to 0. The same goes for $b _ { i j }$ . Then, the graph similarity of $G _ { q } ^ { J }$ and $G _ { b } ^ { J }$ can be computed through $$ S _ { q b } ^ { G } = \sum 1 - | q _ { i j } - b _ { i j } | .\tag{19} $$ We also compute a junction similarity score $S _ { q b } ^ { J }$ using the junction vocabulary and the DBoW2 algorithm. Finally, the similarity score of $\textstyle { \mathcal { K } } _ { q }$ and $\displaystyle { \mathcal { K } } _ { b }$ is given by combining the keypoint similarity, junction similarity, and structure graph similarity $$ S _ { q b } = S _ { q b } ^ { K } + S _ { q b } ^ { J } \cdot \left( 1 + \frac { S _ { q b } ^ { G } } { n } \right)\tag{20} $$ where $S _ { q b } ^ { K }$ is the keypoint similarity of $\textstyle { \mathcal { K } } _ { q }$ and $\displaystyle { \mathcal { K } } _ { b }$ computed in the first step. We compute the similarity score with the query frame for each candidate, and only the top 3 candidates with the highest similarity scores will be retained for the next step. Analysis: We next analyze the second step. In the normal two-step pipeline that uses the DBoW method, only appearance information is used to retrieve candidates. The structural information, i.e., the necessity of the consistent spatial distribution of features between the query frame and candidate, is ignored in the first step and only used in the second step. However, in SOLD2 the illumination-challenging scenes, the structural information is essential as it is invariant to lighting conditions. In our second step, a portion of the structural information is utilized to select candidates. First, our PLNet uses the wireframe-parsing method to detect structural lines, which are more stable in illuminationchallenging environments. Second, the similarity computed in (20) utilizes both the appearance information and the structural information. Therefore, our system can achieve good performance in illumination-challenging environments although using the efficient DBoW method. The second step is also highly efficient. On the one hand, junctions are usually much less than keypoints. In normal scenes, our PLNet can detect more than 400 good keypoints but only about 50 junctions. On the other hand, the junction vocabulary is tiny and only contains 1000 words. Therefore, matching junctions using DBoW2, constructing junction graphs, and computing similarity scores are all executed very efficiently. The experiment shows that the second step can be done within 0.7 ms. More results will be presented in Section VII. 3) Third Step: The third step aims to estimate the pose of the query frame. We first use LightGlue to match features between the query frame and the retained candidates. The candidate with the most matching inliers will be selected as the best candidate. Then, based on the matching results of the query frame and the best candidate, we can associate the query keypoints with map points. Finally, a PnP problem is solved with RANSAC to estimate the pose. The pose will be considered valid if the inliers exceed 20. ## VII. EXPERIMENTS In this section, we present the experiment results. The remainder of this section is organized as follows. In Section VII-A, we evaluate the line detection performance of the proposed PLNet. In Section VII-B, we evaluate the mapping accuracy of our system by comparing it with other SOTA VO or SLAM systems. In Section VII-C, we test our system in three illuminationchallenging scenarios: 1) onboard illumination; 2) dynamic illumination; and 3) low illumination. The comparison of these three scenarios will show the excellent robustness of our system. In Section VII-D, we assess the performance of the proposed map reuse module in addressing the day/night localization challenges, i.e., mapping during the day and relocalization at night. In Section VII-E, we present the ablation study. In Section VII-F, we evaluate the efficiency. We use two platforms in the experiments. Most evaluations are conducted on a personal computer with an Intel i9-13900 CPU and a NVIDIA GeForce RTX 4080 GPU. In the efficiency experiment in Section VII-F, we also deploy AirSLAM on an NVIDIA Jetson Orin to prove that our system can achieve good accuracy and efficiency on the embedded platform. ## A. Line Detection In this section, we evaluate the performance of our PLNet. As described in Section IV-B, we follow SuperPoint [24] to design and train our backbone and keypoint detection module, and we can even use the pretrained model of SuperPoint, therefore, we do not evaluate the keypoint detection anymore. Instead, we assess the performance of the line detection module by comparing it with SOTA systems, as it is trained with a fixed backbone, which is different from other line detectors. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/b2691b7b134a340ab484e34c4395a44bfe61c24c97244d4caca24e9a7044aeed.jpg) ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/b97dc882b489519ef36a7523e063847e86a1afeab40a86877fae2ae8619ae1f2.jpg) ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/fdef875380edf607b5dfae1cc7936f3da046320dc40c2aa28b83e550b59aed0c.jpg) Fig. 7. Line detection comparison between our PLNet (a wireframe parsing method) and SOLD2 (a nonwireframe-parsing method). The red lines are detected line features and the green points are endpoints of lines. Our PLNet aims to detect structural lines while SOLD2 detects more general lines with significant gradients, such as the patterns on the floor and walls. TABLE I COMPARISON OF VARIOUS WIREFRAME PARSING METHODS
Methods1Wireframe DatasetYorkUrban DatasetFPS
$\overline { { \mathrm { s A P } ^ { 5 } } }$ $\overline { { \mathrm { s A P } ^ { \mathrm { I 0 } } } }$ $\overline { { \mathrm { s A P } ^ { \mathrm { I 5 } } } }$ sAP5 $\overline { { \mathrm { s A P } ^ { \mathrm { I 0 } } } }$ $\overline { { \mathrm { s A P } ^ { \mathrm { I 5 } } } }$
AFM [85]18.524.427.57.39.411.110.4*
AFM++ [86]27.732.434.89.511.613.28.0*
L-CNN [67]59.763.665.325.027.128.329.6
LETR [87]59.265.267.723.927.629.72.0
F-Clip [88]64.368.369.128.631.032.482.3
ELSD [89]64.368.970.927.630.231.842.6*
HAWPv2 [51]65.769.771.328.931.232.685.2
HAWPv2 [51] PLNet (Ours)63.6 65.267.7 69.269.5 70.926.6 29.329.0 32.030.385.2 79.4†
The top two results are highlighted and underlined in order. 1 Methods represented using the font are evaluated with color inputs and methods represented using the font are evaluated with grayscale inputs These numbers are cited from the original paper. † The FPS of our PLNet is the speed of detecting both keypoints and lines for the Python implementation. 1) Datasets and Baseliens: This experiment is conducted on the Wireframe dataset [69] and the YorkUrban dataset [84]. The Wireframe dataset contains 5000 training images and 462 test images that are all collected in man-made environments. We use them to train and test our PLNet. To validate the generalization ability, we also compare various methods on the YorkUrban dataset, which contains 102 test images. All the training and test images are resized to $5 1 2 \times 5 1 2$ . We compare our method with AFM [85], AFM++ [86], L-CNN [67], LETR [87], F-Clip [88], ELSD [89], and HAWPv2 [51]. 2) Evaluation Metrics: We evaluate both the accuracy and efficiency of the line detection. For accuracy, the structural average precision (sAP) [67] is the most challenging metric of the wireframe parsing task. It is inspired by the mean average precision commonly used in object detection. A detected line $\tilde { l } = ( \tilde { \bf p } _ { 1 } , \tilde { \bf p } _ { 2 } )$ is a True Positive (TP) if and only if it satisfies the following: $$ \operatorname* { m i n } _ { ( \mathbf { p } _ { 1 } , \mathbf { p } _ { 2 } ) \in \mathcal { L } } \| \mathbf { p } _ { 1 } - \tilde { \mathbf { p } } _ { 1 } \| ^ { 2 } + \| \mathbf { p } _ { 2 } - \tilde { \mathbf { p } } _ { 2 } \| ^ { 2 } \leq \vartheta\tag{21} $$ where L is the set ofground truth, and ϑ is a predefined threshold. We follow the previous methods to set ϑ to 5, 10, and 15, then the corresponding sAP scores are represented by $\mathrm { s A P ^ { 5 } , s A P ^ { 1 0 } }$ and $\mathrm { s A P ^ { 1 5 } }$ , respectively. For efficiency, we use the frames per second (FPS) to evaluate various systems. TABLE II TRANSLATIONAL ERROR (RMSE) ON THE EUROC DATASET (UNIT: M), THE BEST RESULTS ARE IN BOLD
M1Sensors s1I¹Features p¹ L1MH01
MH02MH03MH04MH05Sequence V101V102V103V201V202V203 $\mathrm { A v g } ^ { 2 }$
DROID-SLAM [18]√×X√ ×0.1630.1210.2420.3990.2700.1030.1650.1580.1020.1150.2040.186
VINS-Fusion [4]×」√√0.1630.1780.3160.3310.1750.1020.0990.1120.1100.1240.2520.178
Struct-VIO [47]√×√X √ √0.1190.1000.2830.2750.2560.0750.1970.1610.0810.1520.1770.171
PLF-VINS [90]X√√√ √0.1430.1780.2210.2400.2600.0690.0990.1660.0830.1250.1830.161
Wit opKimera-VIO [63]X√√√ X0.1100.1000.1600.2400.3500.0500.0800.0700.0800.1000.2100.141
OKVIS [91]X√√√ X0.1970.1080.1220.1380.2720.0400.0670.1200.0550.1500.2400.137
AirVIO (Ours)X√√√ √0.0740.0600.1140.1670.1250.0330.1320.2380.0360.0830.1680.113
iSLAM [19]×√」×0.3020.4600.3630.9360.4780.3550.3910.3010.4520.4161.1330.508
UV-SLAM [92]√X一√0.1610.1790.1760.2910.1890.0770.0710.0940.0780.0850.1250.139
Kimera [93]X√一√0.0900.1100.1200.1600.1800.0500.0600.1300.0500.0700.2300.114
OpenVINS [94]X√√√0.0720.1430.0860.1730.2470.0550.0600.0590.0540.0470.1410.103
Structure-PLP-SLAM [95]×√X√0.0460.0560.0480.0710.0710.0910.0660.0650.0610.0610.1660.073
VINS-Fusion [4]X√√√ × × ×0.0520.0400.0520.1240.0880.0460.0530.1080.0400.0810.0980.071
Maplab [96]√×√√0.0410.0260.0450.1100.0670.0390.0450.0800.0530.0840.1960.071
SP-Loop [97]X√√√0.0700.0440.0680.1000.0900.0420.0340.0820.0380.0540.1000.066
PL-SLAM [5]X√√√0.0420.0520.0400.0640.0700.0420.0460.0690.0610.0570.1260.061
Basalt [15]X ×√√ √√0.080 0.0600.0500.1000.0800.0400.0200.0300.0300.0200.0590.052
DVI-SLAM [98]√√×0.0420.0460.0810.0720.0690.0590.0340.0280.0400.039 0.0550.051
ORB-SLAM3 [62]√√ ×√×0.0360.0330.0350.0510.0820.0380.0140.024 0.0320.0140.0240.035
DROID-SLAM [18]X√ √√×0.0150.0130.0350.0480.0400.0370.011 0.0200.0180.0150.0170.024
AirSLAM (Ours)X√√√0.0190.0130.0250.0560.0510.0320.014 0.0250.0140.0180.0680.030
1 M denotes the monocular camera, S denotes the stereo camera, I denotes the IMU, P denotes the keypoint feature, and L denotes the line feature 2 The average error of the successful sequences. 3) Results and Analysis: We present the results in Table I. The top-performing results are distinctly highlighted and underlined in order. It can be seen that our PLNet achieves the second-best performance on the Wireframe dataset and the best performance on the YorkUrban dataset. On the Wireframe dataset, HAWPv2, the best method, only outperforms our PLNet by 0.5, 0.5, and 0.4 points in $\mathrm { s A P ^ { \bar { 5 } } , \ s A \bar { P } ^ { 1 0 } }$ , and $\mathrm { s A P ^ { 1 5 } }$ , respectively. On the YorkUrban dataset, our method surpasses the second-best method by 0.4, 0.8, and 0.9 points on these three metrics, respectively. Overall, we can conclude that our PLNet achieves comparable accuracy with SOTA methods. Generalizability Analysis: We can also conclude that the generalizability of our PLNet is better than other methods. This conclusion is based on two comparative results between our method and HAWPv2, which is the current best wireframe parsing method. First, on the Wireframe dataset, which also serves as the training dataset, HAWPv2 outperforms our PLNet. However, on the YorkUrban dataset, it is surpassed by our method. Second, the previous methods are all evaluated with color inputs in their original paper. Considering that grayscale images are also widely used in vSLAM systems, we train our PLNet with grayscale inputs. We also retrain HAWPv2 and evaluate it using grayscale images for comparison. The result shows that our PLNet significantly outperforms HAWPv2 on both datasets when the inputs are grayscale images. We think the better generalizability comes from our backbone. Other methods are trained on only 5000 images of the Wireframe dataset, while our backbone is trained on a large diverse dataset, which gives it a stronger feature extraction capability. Efficiency Analysis:It is worth noting that the FPS of our method in Table I is the speed of detecting both keypoints and lines, while other methods can only output lines. Nevertheless, our PLNet remains one ofthe fastest methods due to the design of the shared backbone. PLNet processes each image only 0.86 ms slower than the fastest algorithm, i.e., HAWPv2. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/e3401e2d00a9d5f3e7e43ce0d39fe2796d7e59fac67d40976dbbf4ae6d68f849.jpg) Fig. 8. Comparison based on the OIVIO dataset. The vertical axis is the proportion of pose errors that are less than the given alignment error threshold on the horizontal axis. Our AirSLAM achieves the most accurate result. TABLE III RMSE (M) ON THE OIVIO DATASET, THE BEST RESULTS ARE IN BOLD
SequenceKimeraPL- SLAMBasaltDROID- SLAMORB- SLAM3Ours
MN_015_GV_010.1691.2380.2160.2860.0660.054
MN_015_GV_022.4080.8530.1530.0810.0690.052
MN_050_GV_01F1.1430.1860.1730.0630.062
MN_050_GV_02F0.9210.1030.0800.0530.048
MN_100_GV_01F0.8310.1970.1840.0510.064
MN_100_GV_022.2380.6090.0920.0900.0630.042
TN_015_GV_010.3001.5790.1480.1880.0530.057
TN_050_GV_010.2801.7360.5210.3130.0820.065
TN_100_GV_010.2641.3120.1160.1790.0860.078
Average一1.3580.1920.1750.0650.058
F represents tracking failure or large drift error. Note that the selected baselines are all wireframe parsing methods. The nonwireframe-parsing line detection methods, such as SOLD2 [49] and DeepLSD [68], are not added to the comparison as it is unfair to do so. As shown in Fig. 7, the wireframe parsing techniques aim to detect structural lines. They are usually evaluated using the sAP and compared with the ground truth. The nonwireframe-parsing methods can detect more general lines with significant gradients, however, they often detect a long line segment as multiple short line segments, which results in their poor sAP performance. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/dbae6ade22ade6a9ce0ad00c4ddf17deabe3c96a6378d25b522bd3a70a708ea7.jpg) Fig. 9. Our feature detection and matching on a challenging sequence in UMA-VI dataset. The red lines represent detected line features and the colored lines across images indicate feature association. The image may suddenly go dark due to turning off the lights, which is very difficult for vSLAM systems. ## B. Mapping Accuracy In this section, we evaluate the mapping accuracy of our system under well-illuminated conditions. The EuRoC dataset [99] is one of the most widely used datasets for vSLAM, so we use it for the accuracy evaluation. We compare our method only with systems capable of estimating the real scale, so the selected baselines are either visual-inertial systems, stereo systems, or those incorporating both. We incorporate traditional methods, learning-based systems, and hybrid systems into the comparison. We use AirVIO to represent our system without loop detection. The root mean square error (RMSE) is used as the metric and computed by the evo [100]. The comparison results are presented in Table II. We evaluate the systems with and without loop detection on eleven sequences. For the comparison without loop detection, our method outperforms other VIO methods: we achieve the best results on 8 out of 11 sequences. The average translational error of AirVIO is 20% lower than the second-best system, i.e., Kimera-VIO. For the comparison with loop detection, our system achieves comparable performance with ORB-SLAM3 and DROID-SLAM, which are SOTA traditional and learning-based visual SLAM systems, respectively. Another conclusion that can be drawn from Table II is that loop detection significantly improves the accuracy of our system. The average error of our system decreases by 74% after the loop detection. ## C. Mapping Robustness Although many vSLAM systems have achieved impressive accuracy as shown in the previous Section VII-B, complex lighting conditions usually render them ineffective when deployed in real applications. Therefore, in this section, we evaluate the robustness of various vSLAM systems to lighting conditions. We select several representative SOTA systems as baselines. They are ORB-SLAM3 [62], an accurate feature-based system, DROID-SLAM [18], a learning-based hybrid system, Basalt [15], a system that achieves illumination-robust optical flow tracking with the LSSD algorithm, Kimera [93], a direct visual-inertial SLAM system, and OKIVS [91], a system proven to be illumination-robust in our previous work [22]. We test these methods and our system in three scenarios: 1) onboard illumination; 2) dynamic illumination; and 3) low-lighting environments. We first present the evaluation results in Section VII-C1, Section VII-C2, and Section VII-C3, respectively, and then give an overall analysis in Section VII-C4. TABLE IV RMSE (M) ON THE UMA-VI DATASET, THE BEST RESULTS ARE IN BOLD
SequencePL- SLAMORB- SLAM3BasaltOKVISDROID- SLAMOurs
conference-csc12.697F1.2701.1180.7110.490
conference-csc21.596F0.6820.4700.1350.091
conference-csc3F0.4260.4690.0880.7240.088
lab-module-csc-revF0.0630.4860.8610.3640.504
lab-module-cscFF0.4030.5790.3190.979
long-walk-engFF5.0463.005F1.801
third-floor-csc14.4780.8630.4200.2870.0480.070
third-floor-csc26.0680.1490.5900.2710.8900.127
two-floors-csc1FF0.7600.1540.3410.066
two-floors-csc2FF1.2110.6790.2990.190
F represents tracking failure or large drift error. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/61c384beaf0e80af4317b7488392f63f61e3a32b0dd80a5617d4b5dd3fee91e4.jpg) Fig. 10. We use the gamma nonlinearity to generate image sequences with low illumination. i is the brightness level. A and γ are parameters to control the image brightness. The smaller A and B are, the darker the image. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/fa1371840070701b59847324a258d5ee25ab6514af402da37e6ce11a6e592837.jpg) Fig. 11. Comparison results on the Dark EuRoC dataset. The higher the level of low illumination on the x-axis, the darker the image. Basalt is the most stable system while our system is more accurate. 1) Onboard Illumination: We utilize the OIVIO dataset [79] to assess the performance of various systems with onboard illumination. The OIVIO dataset collects visual-inertial data in tunnels and mines. In each sequence, the scene is illuminated by an onboard light of approximately 1300, 4500, or 9000 lumens. We used all nine sequences with ground truth acquired by the Leica TCRP1203 R300. As no loop closure exists in the selected sequences, it is fair to compare the VO systems with the SLAM systems. The performance of translational error is presented in Table III. The most accurate results are in bold, and F represents that the tracking is lost for more than 10 s or the RMSE exceeds 10 m. It can be seen that our method achieves the most accurate results on 7 out of 9 sequences and the smallest average error. The onboard illumination has almost no impact on our AirSLAM and ORB-SLAM3, however, it reduces the accuracy of OKVIS, Basalt, and PL-SLAM. Kimera suffers a lot from such illumination conditions. It even experiences tracking failures and large drift errors on three sequences. We show a comparison of our method with selected baselines on the OIVIO TN\_100\_GV\_01 sequence in Fig. 8. In this case, the robot goes through a mine with onboard illumination. The distance is about 150 m and the average speed is about 0.84 m/s. The plot shows the proportion of pose errors on the horizontal axis that are less than the given alignment error threshold on the horizontal axis. Our system achieves a more accurate result than other systems on this sequence. 2) Dynamic Illumination: The UMA-VI dataset is a visualinertial dataset gathered in challenging scenarios with handheld custom sensors. We selected sequences with illumination changes to evaluate our system. As shown in Fig. 9, it contains many subsequences where the image suddenly goes dark due to turning off the lights. In the subsequence of the top row in Fig. 9, the total darkness lasted for 0.16 s, and low illumination lasted for 0.64 s. In the subsequence of the bottom row, the total darkness lasted for 0.32 s, and low illumination lasted for 0.4 s. It is more challenging than the OIVIO dataset for vSLAM systems. As the ground-truth poses are only available at the beginning and the end of each sequence, we disabled the loop closure part from all the evaluated methods. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/4f21407d96727e7eb495c6c4bb3dec64123f1160bc0177cb185c1a48ee846574.jpg) Fig. 12. Some image pair samples for the mapping and relocalization in the TartanAir Day/Night Localization dataset. Due to the differences in capture viewpoints and scene depths, not all the image pairs have a valid overlap. The translational errors are presented in Table IV. The most accurate results are in bold, and F represents that the tracking is lost for more than 10 s or the RMSE exceeds 10 m. It can be seen that our AirSLAM outperforms other methods. Our system achieves the best results on 7 out of 10 sequences. The UMA-VI dataset is so challenging that PL-SLAM and ORB-SLAM3 fail on most sequences. Although OKVIS and Basalt, like our system, can complete all the sequences, their accuracy is significantly lower than ours. The average RMSEs of OKVIS and Basalt are around 1.134 m and 0.724 m, respectively, while ours is around 0.441 m, which means our average error is only 62.6% of OKVIS and 38.9% of Basalt. 3) Low Illumination: Inspired by [16], we process a publicly available sequence by adjusting the brightness levels of its images. Then the processed sequences are used to evaluate the performance of various SLAM systems in low-illumination conditions. We select the “V2\_01\_easy” of the EuRoC dataset as the base sequence. The image brightness is adjusted using the gamma nonlinearity $$ V _ { o u t } = A V _ { i n } ^ { \frac { 1 } { \gamma } }\tag{22} $$ where $V _ { i n }$ and $V _ { o u t }$ are normalized input and output pixel values, respectively. A and $\gamma$ control the maximum brightness and contrast. We set 12 adjustment levels and use $L _ { i }$ to denote the ith level. $L _ { 0 }$ represents the original sequence, i.e., $A _ { 0 } = 1$ and $\gamma _ { 0 } = 1$ . When $i \in [ 1 , 1 2 ]$ $A _ { i }$ and $\gamma _ { i }$ alternate in descending order to make the image progressively darker. Fig. 10 shows the values of $A _ { i }$ and $\gamma _ { i } ,$ , and the processed image in each level. We name the processed dataset “Dark EuRoC”. We present the comparison result in Fig. 11. As the errors of PL-SLAM are much greater than other methods, we do not show its result. Tracking failures and large drift errors, i.e., the RMSE is more than 1 m, are also marked. It can be seen that low illumination has varying degrees of impact on different systems. In this scenario, optical flow-based methods demonstrate greater stability compared to feature-based methods. Nonetheless, our approach achieves comparable performance to Basalt and DROID-SLAM, which utilize sparse and dense optical flow, respectively. The RMSEs of ORB-SLAM3 and OKVIS increase as the brightness decreases. They even experience tracking failures or large drift errors on $L _ { 1 0 }$ and $L _ { 1 1 }$ TABLE V RELOCALIZATION COMPARISON ON THE TARTANAIR DAY/NIGHT LOCALIZATION DATASET, THE BEST RESULTS ARE IN BOLD
Global Feature1+ Matching¹+ Local Feature1 $\mathrm { F P S } ^ { 2 }$ Recall Rate of Sequences (%)
P000P001P002P003P004P005P006P007P008P009P010P011Avg
[6] 0] $\mathrm { N V } + \mathrm { N N } + \mathrm { S O S N e t }$ 12.44.310.642.510.5 19.033.9 20.126.1 65.97.2 27.527.2 49.032.76.524.97.719.5
$\mathrm { N V } + \mathrm { N N } + \mathrm { D } 2 { \cdot } \mathrm { N e t }$ 10.3422.620.787.469.064.371.423.445.0
$\mathrm { N V } + \mathrm { N N } + \mathrm { R } 2 \mathrm { D } 2$ 8.1415.732.392.422.752.775.514.484.954.867.760.726.050.0
$\mathrm { N V } + \mathrm { N N } + \mathrm { S P }$ 30.269.232.888.717.253.572.631.183.872.669.689.534.059.6
$\mathrm { N V } + \mathrm { A L } + \mathrm { S O S N e t }$ 6.129.030.355.619.649.342.412.745.838.925.247.011.334.0
$\mathrm { N V } + \mathrm { L G } + \mathrm { S I F T }$ 10.845.931.383.058.552.464.718.878.857.243.679.833.653.7
$\mathrm { N V } + \mathrm { L G } + \mathrm { D I S K }$ 12.922.935.497.962.960.984.233.889.990.085.574.933.464.3
$\mathrm { N V } + \mathrm { L G } + \mathrm { S P }$ 20.694.735.498.470.963.585.833.895.9 96.797.191.499.249.876.3
$\mathrm { D I R } + \mathrm { L G } + \mathrm { S P }$ 19.187.228.8 35.499.5 97.769.8 70.364.2 64.385.2 86.132.897.188.196.050.474.7
$\mathrm { O p e n I B L } + \mathrm { L G } + \mathrm { S P }$ 21.388.636.434.796.491.3 94.095.7 85.199.7 96.945.1 45.175.4 75.6
$\mathrm { E P + L G + S P }$ 22.399.7100.068.364.585.834.496.5
AirSLAM (Ours)48.889.578.887.894.688.285.670.072.883.478.881.454.580.5
1 NV is NetVLAD, DIR is AP-GeM/DIR, EP is EigenPlace, NN is Nearest Neighbor Matching, AL is AdaLAM, LG is LightGlue, and SP is SuperPoint. 2 Running time of relocalization measured in Frame Per Second (FPS). 4) Result Analysis: We think the above three lighting conditions affect a visual system in different ways. The OIVIO dataset collects sequences in dark environments with only onboard illumination, so the light source moves along with the robot, which results in two effects. On the one hand, the lighting is uneven in the environment. The direction the robot is facing and the area closer to the robot is brighter than other areas. The uneven image brightness may lead to the uneven distribution of features. On the other hand, when the robot moves, the lighting of the same area will change, resulting in different brightness in different frames. The assumption of brightness constancy in some systems will be affected in such conditions. The UMA-VI dataset is collected under dynamic lighting conditions, where the dynamic lighting is caused by the sudden switching of lights or moving between indoor and outdoor environments. The image brightness variations in the UMA-VI dataset are much more intense than those in the OIVIO dataset, which may even make the extracted feature descriptor inconsistent in consecutive frames. In low-illumination environments, both the brightness and contrast of captured images are very low, making the vSLAM system more difficult to detect enough good features and extract distinct descriptors. We summarize the above experiment results with the following conclusions. First, the systems that use descriptors for matching are more robust than the direct methods in illumination-dynamic environments. On the OIVIO dataset, our AirSLAM and ORB-SLAM3 outperform the other systems significantly. On the UMA-VI dataset, our method and OKVIS achieve the best and the second-best results, respectively. This is reasonable as the brightness constancy assumption constrains the direct methods. Despite Basalt uses LSSD to enhance its optical flow tracking, its accuracy still decreases significantly in these two scenarios. Second, the direct methods are more stable in the low illumination environments. This is because descriptor-based SLAM systems rely on enough high-quality features and descriptors, which are difficult to obtain on low brightness and contrast images. The direct methods use corners that are easier to detect, so the low illumination has less impact on them. Third, thanks to the robust feature detection and matching, the illumination robustness of our system is far better than that of other systems. AirSLAM achieves relatively high accuracy in these three illumination-challenging scenarios. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/099e23beabe04ff4be54dcff872f4646b5f01c77ab0c9a7b2d70eeeb35fce63d.jpg) Fig. 13. Point-line map of the P000 sequence built by our AirSLAM. The red points are mappoints and the blue lines are 3D lines. ## D. Map Reuse 1) Dataset: As mapping and relocalization in the same well-illuminated environment are no longer difficult for many current vSLAM systems, we only evaluate our map reuse module under illumination-challenging conditions, i.e., the day/night localization task. We use the “abandoned\_factory” and “abandoned\_factory\_night” scenes in the TartanAir dataset [60] as they can provide consecutive stereo image sequences for the SLAM mapping and the corresponding accurate ground truth for the evaluation. The images in these two scenes are collected during the day and at night, respectively. We use the sequences in the “abandoned\_factory” scene to build maps. Then, for each mapping image, the images with a relative distance of less than 3 m and a relative angle of less than $1 5 ^ { \circ }$ from it in the “abandoned\_factory\_night” scene are selected as query images. We call the generated mapping and relocalization sequences the “TartanAir Day/Night Localization” dataset. Fig. 12 shows some sample pairs for mapping and relocalization. It is worth noting that due to the differences in capture viewpoints and scene depths, the query image selected based on the relative distance and angle may not always have valid overlapping with the mapping images. TABLE VI ABLATION STUDY OF LINE FEATURES FOR MAPPING
EuRoCOIVIOUMA-VIDark-EuRoC
w/o Lines0.0460.0720.5450.035
Ours0.0300.0580.4410.029
We present the RMSE (m) of our system with and without line features. 2) Baseline: We have tried several traditional vSLAM systems, e.g., ORB-SLAM3 [62] and SOTA learning-based onestage relocalization methods, e.g., ACE [101] on the TartanAir Day/Night Localization dataset, and find they perform badly: Their relocalization recall rates are below 1% . Therefore, we only present the comparison results of our systems and some VPR methods. The Hloc toolbox [9] uses the structure from motion (SFM) method to build maps and has integrated many image retrieval methods, local feature extractors, and matching methods for localization. We mainly compare our system with these methods. Specifically, the NetVLAD [64], AP-GeM/DIR [102], OpenIBL [103], and EigenPlaces [104] are used to extract global features, the SuperPoint [24], SIFT [105], D2-Net [41], SOSNet [106], R2D2 [39], and DISK [40] are used to extract local features, and the LightGlue [27], AdaLAM [107], and Nearest Neighbor Matching are used to match features. We combine these methods into various “global feature detection + local feature matcher + local feature detection” pipelines for the mapping and relocalization. We do not add DXSLAM [37] to the comparison as it uses NetVLAD and SuperPoint with the binary descriptor, which has been included in the above pipelines. 3) Results: To achieve a fair comparison and balance the efficiency and effectiveness, we extract 400 local features and retrieve 3 candidates in the coarse localization stage for all methods. Unlike vSLAM systems that have the keyframe selection mechanism, the SFM mapping optimizes all input images, so it is very slow when mapping with original sequences. Therefore, to accelerate the SFM mapping while ensuring its mapping frames are more than our keyframes, we sample its mapping sequences by selecting one frame every four frames. We show a point-line map built by our AirSLAM in Fig. 13. The relocalization results are presented in Table V. We give the running time (FPS) and the relocalization recall rate of each method. We define a successful relocalization if the estimated pose of the query frame is within 2 m and $1 5 ^ { \circ }$ of the ground truth. It can be seen that our AirSLAM outperforms other methods in terms ofboth efficiency and recall rate. Our system achieves the best results on 5 out of 11 sequences. AirSLAM has an average recall rate 4.2% higher than the second-best algorithm and is about 2.4 times faster than it. 4) Analysis: We find that our system is more stable than the VPR methods on the TartanAir Day/Night Localization dataset. On several sequences, e.g., P000, P002, and P010, some VPR methods achieve remarkable results, with recall rates close to 100% . However, on some other sequences, e.g., P001 and P006, their recall rates are less than 40% . In contrast, our system maintains a recall rate of 70% to 90% on most sequences. TABLE VII RELOCALIZATION ABLATION STUDY
${ \mathrm { S e q . } }$ $N _ { C } { = } 3$ $N _ { C } { = } 5$ $N _ { C } { = } 1 0$
w/o G.Oursw/o G.Oursw/o G.Ours
P00077.989.584.392.588.594.0
P00169.278.879.383.885.487.9
P00275.487.880.987.885.388.0
P00386.494.692.495.593.995.9
P00481.588.285.390.989.092.1
P00578.485.682.988.888.491.5
P00662.470.072.372.778.677.2
P00760.672.863.773.769.076.2
P00877.283.480.084.481.785.8
P00963.478.870.681.776.984.8
P01070.781.475.484.080.986.9
P01148.154.555.158.962.064.0
$\operatorname { A v g } .$ 70.980.576.982.981.685.4
The recall rates (%) of our system with and without the structure graph during map reuse. To clarify this, we examined each sequence and roughly categorized the images into three types. As shown in Fig. 12, the first type of image is captured with the camera relatively far from the features, so there is a significant overlap between each day/night image pair. In addition, these images contain distinct buildings and landmarks. The second type of image pair also has a significant overlap. However, the common regions in these image pairs do not contain large buildings and landmarks. The third type of image is captured with the camera very close to the features. Although the camera distance between the day/night image pair is not large, they have almost no overlap, making their local feature matching impossible. We find that the VPR methods perform very well on the first type of image but perform poorly on the second type of image. Therefore, their recall rates are very low on P001 and P006, which contain more of the second type of images. This may be because their global features are usually trained on datasets that have a lot of distinct buildings and landmarks, which makes them rely more on such semantic cues to retrieve similar images. By contrast, our system is based on the DBoW method, which only utilizes the low-level local features of images, so it achieves similar performance on the first and second types ofimages. This also proves the strong generalizability of our system. However, neither our system nor VPR methods can process the pairs with little overlap due to relying on local feature matching. Such image pairs are abundant in the P011. ## E. Ablation Study 1) Line Feature in Mapping: In this section, we evaluate the impact of line features on mapping performance. To this end, we remove line features from our system and compute the RMSE on the EuRoC, OIVIO, UMA-VI, and Dark-EuRoC datasets. The results are summarized in Table VI, where “w/o Lines” refers to our system without line features. As shown in the table, incorporating line features reduces the RMSE by 34.8%, 19.4%, 19.1%, and 17.1% on the four datasets, respectively. These results demonstrate that line features substantially enhance our system and improve the mapping accuracy. We believe this improvement comes from two aspects. On the one hand, a line feature is more likely to be observed across multiple frames compared to a point feature, thereby constraining more frames during pose optimization. This consistent constraint across multiple frames enhances the accuracy of pose estimation. On the other hand, we use the tracking results of a set of points to track a line feature, which makes the tracking of a single line feature more stable than that of a single point. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/106a21fabe63d227843b2f4f22ffc942160b728072f63f27b6c265a31d1b8fbf.jpg) (a) ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/fa32deadb00a5f018ded072e9dbda35a920f7893c540eba8900ac33cb1879263.jpg) (b) Fig. 14. For odometry efficiency, we disable the loop detection from all the methods. The GPU usage of systems using GPU is also indicated. For mapping efficiency, we compute the total mapping time and compare it with Hloc. (a) Odometry Efficiency. (b) Mapping Efficiency. TABLE VIII EVALUATION OF CPU AND GPU CONTRIBUTIONS TO OUR SYSTEM
Front-endBack-end
FD+FM1Others
ORB-SLAM332.9s25.2s39.6s
Ours17.5s8.4s49.3s
1 Total runtime for feature detection and feature matching. andrepresent the modules running on the GPU and CPU, respectively. 2) Structure Graph in Relocalization: We also verify the effectiveness of the proposed relocalization method. This experiment is conducted on the TartanAir Day/Night Localization dataset. We compare the systems with and without the second step proposed in Section VI-B2. The results are presented in Table VII, where “w/o $\mathrm { G } . ^ { \dag }$ denotes our system without the structure graph, and $N _ { \mathcal { C } }$ denotes the candidate number for local feature matching. It shows that using junctions, line features, and structure graphs to filter out relocalization candidates significantly improves recall rates. AirSLAM outperforms w/o G. across all sequences, and when $N _ { \mathcal { C } }$ is 3, 5, and 10, the average improvements are 9.6%, 6.0%, and 3.8%, respectively, which demonstrates the effectiveness of the proposed method. ## F. Efficiency Analysis Efficiency is essential for robotics applications, so we also evaluate the efficiency of the proposed system. We first compare the running time of our AirSLAM with several SOTA VO and SLAM systems on a computer with an Intel i9-13900 CPU and an NVIDIA RTX 4080 GPU. Then, we deploy AirSLAM on an NVIDIA Jetson AGX Orin to verify the efficiency and performance of our system on the embedded platform. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/b0e9a37500387fe364eb2e94277d020007525681386948c0fbd56ae085a23feb.jpg) Fig. 15. Accuracy comparison of our system on an NVIDIA Jetson Orin and a PC. The vertical axis represents ATE in meters. TABLE IX EFFICIENCY COMPARISON OF OUR SYSTEM ON TWO PLATFORMS
PlatformRuntimeCPU Usage (%)GPU Usage (MB)
VIO (FPS)Optim! (s)
Ours-Jetson40.357.8224.7989
Ours-PC73.155.5217.83076
1 The runtime of the offline map optimization. 1) Odometry Efficiency: The VO/VIO efficiency experiment is conducted on the MH\_01\_easy sequence of the EuRoC dataset. We compare our AirSLAM with several SOTA systems. The loop detection and GBA are disabled from all the systems for a fair comparison. The metrics are the runtime per frame and the CPU usage. The results are presented in Fig. 14(a), where 100% CPU usage means 1 CPU core is utilized. It should be additionally noted that DROID-SLAM actually uses 32 CPU cores, and its CPU usage in Fig. 14(a) is only for a compact presentation. Our system is very efficient, achieving a rate of 73 FPS. In addition, due to extracting and matching features using the GPU, our system requires relatively less CPU resources. We also test the GPU usage. It shows that DROID-SLAM requires about 8 GB GPU memory, while our AirSLAM only requires around 3 GB. To investigate the contributions of the CPU and GPU to the efficiency ofour system, we measured the total runtime ofmodules executed on the GPU and CPU using the MH\_01\_easy sequence and compared it with ORB-SLAM3. The loop detection and GBA are disabled from both systems. The results are presented in Table VIII. It can be seen that our feature detection and matching are highly efficient due to the use of the GPU resources and our system design, which processes only the left image in nonkeyframes. Note that we perform the initial pose estimation in the back-end, whereas ORB-SLAM3 includes this process in the front-end. This leads to the differences in the runtime of the “Others” and “Back-end” components between the two systems. 2) Mapping Efficiency: We also evaluate the mapping time, i.e., the total runtime for building the initial map and offline optimizing the map. As we compare our system with Hloc using the TartanAir dataset in the map reuse experiment, we use the same baseline and dataset in this experiment. The average mapping time per frame may differ when the map size varies, therefore, we measure the mapping time with different numbers of input images. The results are presented in Fig. 14(b), where n× means our system is n times faster than Hloc. It can be seen that our system is much more efficient than Hloc, especially as the input images increase. Besides, Hloc can only use monocular images to build a map without the real scale, and the map only contains point features, while our system can build the point-line map and estimate the real scale using a stereo camera and an IMU. Therefore, our system is more stable and practical for robotics applications than Hloc. 3) Embedded Platform: We use 8 sequences in the EuRoC dataset to evaluate the efficiency of AirSLAM on the embedded platform. The suffixes, $\mathrm { i . e . , }$ , “-Jetson” and $\mathbf { \tilde { \Sigma } } ^ { 6 6 } \mathbf { - P C } ^ { 9 }$ , are added to distinguish results on different platforms. On the Jetson, we modify three parameters in our system to improve efficiency. First, we reduced the number of detected keypoints from 350 to 300. Second, we change two parameters in Section V-D to make keyframes sparser, i.e., $\alpha _ { 1 }$ and $\alpha _ { 2 }$ are changed from 0.65 and 0.1 to 0.5 and 0.2, respectively. The other parameters are the same on these two platforms. The comparisons of efficiency and absolute trajectory error (ATE) are presented in Table IX and Fig. 15, respectively. Our AirSLAM can run at a rate of 40 Hz on the Jetson while only consuming 2 CPU cores and 989 MB GPU memory. We find the runtime of the offline map optimization is very close on these two platforms. This is because AirSLAM-Jetson selects fewer keyframes than AirSLAM-PC, so the loop closure and GBA are faster. ## VIII. CONCLUSION In this work, we presented an efficient and illumination-robust hybrid vSLAM system. To be robust to challenging illumination, the proposed system employs a CNN to detect both keypoints and structural lines. Then these two features are associated and tracked using a GNN. To enhance the efficiency, we proposed PLNet, a unified model capable ofsimultaneously detecting both point and line features. Furthermore, a multistage relocalization method based on both appearance and geometry information was proposed for efficient map reuse. We designed the system with an architecture that includes online mapping, offline optimization, and online relocalization, making it easier to deploy on real robots. Extensive experiments show that the proposed system outperforms other SOTA vSLAM systems in terms of accuracy, efficiency, and robustness in illumination-challenging environments. Despite its remarkable performance, the proposed system still has limitations. Like other point-line-based SLAM systems, our AirSLAM relies on enough line features, so it is best to apply it to man-made environments. This is because our system was originally designed for warehouse robots. In unstructured environments, the system degrades into a point-only system. ## REFERENCES [1] A. M. Barros, M. Michel, Y. Moline, G. Corre, and F. Carrel, “A comprehensive survey of visual SLAM algorithms,” Robotics, vol. 11, no. 1, pp. 24–50, 2022. [2] S. Yuan, H. Wang, and L. Xie, “Survey on localization systems and algorithms for unmanned systems,” Unmanned Syst., vol. 9, no. 2, pp. 129–163, 2021. [3] R. Mur-Artal, J. M. M. Montiel, and J. D. Tardos, “ORB-SLAM: A versatile and accurate monocular slam system,” IEEE Trans. Robot., vol. 31, no. 5, pp. 1147–1163, Oct. 2015. [4] T. Qin, P. Li, and S. Shen, “Vins-mono: A robust and versatile monocular visual-inertial state estimator,” IEEE Trans. Robot., vol. 34, no. 4, pp. 1004–1020, Aug. 2018. [5] R. Gomez-Ojeda, F.-A. Moreno, D. Zuniga-Noël, D. Scaramuzza, and J. Gonzalez-Jimenez, “Pl-SLAM: A stereo SLAM system through the combination of points and line segments,” IEEE Trans. Robot., vol. 35, no. 3, pp. 734–746, Jun. 2019. [6] C. Cadena et al., “Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age,” IEEE Trans. Robot., vol. 32, no. 6, pp. 1309–1332, Dec. 2016. [7] D. Zuñiga-Noël, A. Jaenal, R. Gomez-Ojeda, and J. Gonzalez-Jimenez, “The uma-vi dataset: Visual–inertial odometry in low-textured and dynamic illumination environments,” Int. J. Robot. Res., vol. 39, no. 9, pp. 1052–1060, 2020. [8] A. Savinykh et al., “Darkslam: Gan-assisted visual slam for reliable operation in low-light conditions,” in Proc. IEEE 95th Veh. Technol. Conf., IEEE, 2022, pp. 1–6. [9] P.-E. Sarlin, C. Cadena, R. Siegwart, and M. Dymczyk, “From coarse to fine: Robust hierarchical localization at large scale,” in Proc. IEEE/CVF Conf. Comput. Vis. pattern Recognit., 2019, pp. 12716–12725. [10] C. Toft et al., “Long-term visual localization revisited,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 4, pp. 2074–2088, Apr. 2022. [11] Q. Gu, P. Liu, J. Zhou, X. Peng, and Y. Zhang, “Drms: Dim-light robust monocular simultaneous localization and mapping,” in Proc. 2021 Int. Conf. Comput., Control Robot., IEEE, 2021, pp. 267–271. [12] L. Yu, E. Yang, and B. Yang, “Afe-orb-slam: Robust monocular vslam based on adaptive fast threshold and image enhancement for complex lighting environments,” J. Intell. Robotic Syst., vol. 105, no. 2, pp. 1–14, 2022. [13] R. Gomez-Ojeda, Z. Zhang, J. Gonzalez-Jimenez, and D. Scaramuzza, “Learning-based image enhancement for visual odometry in challenging hdr environments,” in Proc. 2018 IEEE Int. Conf. Robot. Automat., IEEE, 2018, pp. 805–811. [14] G. G. Scandaroli, M. Meilland, and R. Richa, “Improving ncc-based direct visual tracking,” in Proc. Eur. Conf. Comput. Vis., Springer, 2012, pp. 442–455. [15] V. Usenko, N. Demmel, D. Schubert, J. Stückler, and D. Cremers, “Visual-inertial mapping with non-linear factor recovery,” IEEE Robot. Automat. Lett., vol. 5, no. 2, pp. 422–429, Apr. 2020. [16] S. Park, T. Schöps, and M. Pollefeys, “Illumination change robustness in direct visual SLAM,” in Proc. 2017 IEEE Int. Conf. Robot. Automat., IEEE, 2017, pp. 4523–4530. [17] W. Wang, Y. Hu, and S. Scherer, “Tartanvo: A generalizable learningbased VO,” in Proc. Conf. Robot Learn., 2021, pp. 1761–1772. [18] Z. Teed and J. Deng, “Droid-SLAM: Deep visual SLAM for monocular, stereo, and RGB-D cameras,” Adv. Neural Inf. Process. Syst., vol. 34, pp. 16558–16569, 2021. [19] T. Fu, S. Su, Y. Lu, and C. Wang, “iSLAM: Imperative SLAM,” IEEE Robot. Automat. Lett., vol. 9, no. 5, pp. 4607–4614, May 2024. [Online]. Available: https://arxiv.org/pdf/2306.07894.pdf [20] M. Labbé and F. Michaud, “Multi-session visual slam for illuminationinvariant re-localization in indoor environments,” Front. Robot. AI, vol. 9, 2022, Art. no. 801886. [21] M. Labbé and F. Michaud, “Rtab-map as an open-source lidar and visual simultaneous localization and mapping library for large-scale and longterm online operation,” J. Field Robot., vol. 36, no. 2, pp. 416–446, 2019. [22] K. Xu, Y. Hao, S. Yuan, C. Wang, and L. Xie, “Airvo: An illuminationrobust point-line visual odometry,” in Proc. 2023 IEEE/RSJ Int. Conf. Intell. Robots Syst., IEEE, 2023, pp. 3429–3436. [23] S. Kannapiran et al., “Stereo visual odometry with deep learning-based point and line feature matching using an attention graph neural network,” in Proc. 2023 IEEE/RSJ Int. Conf. Intell. Robots Syst., IEEE, 2023, pp. 3491–3498. [24] D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superpoint: Selfsupervised interest point detection and description,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops, 2018, pp. 224–236. [25] R. G. Von Gioi, J. Jakubowicz, J.-M. Morel, and G. Randall, “LSD: A line segment detector,” Image Process. On Line, vol. 2, pp. 35–55, 2012. [26] P.-E. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superglue: Learning feature matching with graph neural networks,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 4938–4947. [27] P. Lindenberger, P.-E. Sarlin, and M. Pollefeys, “Lightglue: Local feature matching at light speed,” in Proc. IEEE/CVF Int. Conf. Comput. Vis., 2023, pp. 17627–17638. [28] E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, “ORB: An efficient alternative to SIFT or SURF,” in Proc. 2011 Int. Conf. Comput. Vis., IEEE, 2011, pp. 2564–2571. [29] D. G. Viswanathan, “Features from accelerated segment test (fast),” in Proc. 10th Workshop Image Anal. Multimedia Interactive Serv., London, U.K., 2009, pp. 6–8. [30] S. Leutenegger, M. Chli, and R. Y. Siegwart, “Brisk: Binary robust invariant scalable keypoints,” in Proc. 2011 Int. Conf. Comput. Vis., IEEE, 2011, pp. 2548–2555. [31] R. Kang, J. Shi, X. Li, Y. Liu, and X. Liu, “Df-SLAM: A. deep-learning enhanced visual slam system based on deep local features,” 2019, arXiv:1901.07223. [32] V. Balntas, E. Riba, D. Ponsa, and K. Mikolajczyk, “Learning local feature descriptors with triplets and shallow convolutional neural networks,” in Proc. Bmvc, 2016, vol. 1, no. 2, p. 3. [33] J. Tang, L. Ericson, J. Folkesson, and P. Jensfelt, “Gcnv2: Efficient correspondence prediction for real-time SLAM,” IEEE Robot. Automat. Lett., vol. 4, no. 4, pp. 3505–3512, Oct. 2019. [34] X. Han, Y. Tao, Z. Li, R. Cen, and F. Xue, “SuperPointVO: A lightweight visual odometry based on CNN feature extraction,” in Proc. IEEE 5th Int. Conf. Automat., Control Robot. Eng., 2020, pp. 685–691. [35] H. M. S. Bruno and E. L. Colombini, “LIFT-SLAM: A deep-learning feature-based monocular visual SLAM method,” Neurocomputing, vol. 455, pp. 97–110, 2021. [36] K. M. Yi, E. Trulls, V. Lepetit, and P. Fua, “Lift: Learned invariant feature transform,” in Proc. Eur. Conf. Comput. Vis., Springer, 2016, pp. 467–483. [37] D. Li et al., “Dxslam: A robust and efficient visual SLAM system with deep features,” in Proc. 2020 IEEE/RSJ Int. Conf. Intell. Robots Syst., IEEE, 2020, pp. 4958–4965. [38] Z. Zhan, D. Gao, Y.-J. Lin, Y. Xia, and C. Wang, “iMatching: Imperative correspondence learning,” in Proc. Eur. Conf. Comput. Vis., 2024, pp. 183–200. [Online]. Available: https://arxiv.org/abs/2312.02141 [39] J. Revaud, C. De Souza, M. Humenberger, and P. Weinzaepfel, “R2d2: Reliable and repeatable detector and descriptor,”Adv. Neural Inf. Process. Syst., vol. 32, pp. 12414–12424, 2019. [40] M. Tyszkiewicz, P. Fua, and E. Trulls, “Disk: Learning local features with policy gradient,” Adv. Neural Inf. Process. Syst., vol. 33, pp. 14254–14265, 2020. [41] M. Dusmanu et al., “D2-net: A trainable cnn for joint description and detection of local features,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2019, pp. 8092–8101. [42] H. Hu, L. Sackewitz, and M. Lauer, “Joint learning of feature detector and descriptor for visual SLAM,” in Proc. 2021 IEEE Intell. Veh. Symp., IEEE, 2021, pp. 928–933. [43] C. Anagnostopoulos, A. S. Lalos, P. Kapsalas, D. Van Nguyen, and C. Stylios, “Reviewing deep learning-based feature extractors in a novel automotive SLAM framework,” in Proc. 31st Mediterranean Conf. Control Automat., IEEE, 2023, pp. 107–112. [44] C. Akinlar and C. Topal, “Edlines: A real-time line segment detector with a false detection control,” Pattern Recognit. Lett., vol. 32, no. 13, pp. 1633–1642, 2011. [45] Y. He, J. Zhao, Y. Guo, W. He, and K. Yuan, “PL-VIO: Tightly-coupled monocular visual–inertial odometry using point and line features,” Sensors, vol. 18, no. 4, 2018, Art. no. 1159. [46] L. Zhou, G. Huang, Y. Mao, S. Wang, and M. Kaess, “Edplvo: Efficient direct point-line visual odometry,” in Proc. 2022 Int. Conf. Robot. Automat., IEEE, 2022, pp. 7559–7565. [47] D. Zou, Y. Wu, L. Pei, H. Ling, and W. Yu, “Structvio: Visual-inertia odometry with structural regularity of man-made environments,” IEEE Trans. Robot., vol. 35, no. 4, pp. 999–1013, Aug. 2019. [48] Q. Chen et al., “VPL-SLAM: A vertical line supported point line monocular slam system,” IEEE Trans. Intell. Transp. Syst., vol. 25, no. 8, pp. 9749–9761, Aug. 2024. [49] R. Pautrat∗, J.-T. Lin∗, V. Larsson, M. R. Oswald, and M. Pollefeys, “Sold2: Self-supervised occlusion-aware line description and detection,” in Proc. Comput. Vis. Pattern Recognit., 2021, pp. 11368–11378. [50] X. Lin and C. Wang, “AirLine: Efficient learnable line detection with local edge voting,” in Proc. IEEE/RSJInt. Conf. Intell. Robots Syst., 2023, pp. 3270–3277. [Online]. Available: https://arxiv.org/abs/2303.16500 [51] N. Xue et al., “Holistically-attracted wireframe parsing: From supervised to self-supervised learning,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 12, pp. 14727–14744, Dec. 2023. [52] J. Engel, V. Koltun, and D. Cremers, “Direct sparse odometry,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 3, pp. 611–625, Mar. 2018. [53] A. Crivellaro and V. Lepetit, “Robust 3D tracking with descriptor fields,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2014, pp. 3414–3421. [54] J. Huang and S. Liu, “Robust simultaneous localization and mapping in low-light environment,” Comput. Animation Virtual Worlds, vol. 30, no. 3-4, 2019, Art. no. e1895. [55] P. Kim, H. Lee, and H. J. Kim, “Autonomous flight with robust visual odometry under dynamic lighting conditions,” Auton. Robots, vol. 43, no. 6, pp. 1605–1622, 2019. [56] Z. Chen and C. Heckman, “Robust pose estimation based on normalized information distance,” in Proc. 2021 IEEE/RSJ Int. Conf. Intell. Robots Syst., IEEE, 2021, pp. 2217–2223. [57] H. Alismail, M. Kaess, B. Browning, and S. Lucey, “Direct visual odometry in low light using binary descriptors,” IEEE Robot. Automat. Lett., vol. 2, no. 2, pp. 444–451, Apr. 2017. [58] A. Creswell, T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta, and A. A. Bharath, “Generative adversarial networks: An overview,” IEEE signal Process. Mag., vol. 35, no. 1, pp. 53–65, Jan. 2018. [59] S. P. Singh, B. Mazotti, S. Mayilvahanan, G. Li, D. M. Rajani, and M. Ghaffari, “Twilight SLAM: A comparative study of low-light visual SLAM pipelines,” 2023, arXiv:2304.11310. [60] W. Wang et al., “Tartanair: A dataset to push the limits of visual SLAM,” in Proc. 2020 IEEE/RSJ Int. Conf. Intell. Robots Syst., IEEE, 2020, pp. 4909–4916. [61] D. Gálvez-López and J. D. Tardos, “Bags of binary words for fast place recognition in image sequences,” IEEE Trans. Robot., vol. 28, no. 5, pp. 1188–1197, Oct. 2012 . [62] C. Campos, R. Elvira, J. J. G. Rodríguez, J. M. Montiel, and J. D. Tardós, “ORB-SLAM3: An accurate open-source library for visual, visual–inertial, and multimap SLAM,” IEEE Trans. Robot., vol. 37, no. 6, pp. 1874–1890, Dec. 2021. [63] A. Rosinol, M. Abate, Y. Chang, and L. Carlone, “Kimera: An opensource library for real-time metric-semantic localization and mapping,” in Proc. 2020 IEEE Int. Conf. Robot. Automat., 2020, pp. 1689–1696. [64] R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, and J. Sivic, “Netvlad: Cnn architecture for weakly supervised place recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 5297–5307. [65] N. Keetha et al., “Anyloc: Towards universal visual place recognition,” IEEE Robot. Automat. Lett., vol. 9, no. 2, pp. 1286–1293, Feb. 2024. [66] S. Yan et al., “Long-term visual localization with mobile sensors,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 17245–17255. [67] Y. Zhou, H. Qi, and Y. Ma, “End-to-end wireframe parsing,” in Proc. IEEE/CVF Int. Conf. Comput. Vis., 2019, pp. 962–971. [68] R. Pautrat, D. Barath, V. Larsson, M. R. Oswald, and M. Pollefeys, “Deeplsd: Line segment detection and refinement with deep image gradients,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 17327–17336. [69] K. Huang, Y. Wang, Z. Zhou, T. Ding, S. Gao, and Y. Ma, “Learning to parse wireframes in images of man-made environments,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2018, pp. 626–635. [70] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in Proc. Med. Image Comput. Comput.-Assist. Interv. 2015: 18th Int. Conf., Munich, Germany, Springer, Oct. 2015, pp. 234–241. [71] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proc. 3rd Int. Conf. Learn. Representations, 2015. [Online]. Available: https://arxiv.org/abs/1412.6980 [72] C. Forster, L. Carlone, F. Dellaert, and D. Scaramuzza, “On-manifold preintegration for real-time visual–inertial odometry,” IEEE Trans. Robot., vol. 33, no. 1, pp. 1–21, Feb. 2017. [73] L. Zhang and R. Koch, “An efficient and robust line segment matching approach based on lbd descriptor and pairwise geometric consistency,” J. Vis. Commun. Image Representation, vol. 24, no. 7, pp. 794–805, 2013. [74] R. Mur-Artal and J. D. Tardós, “ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras,” IEEE Trans. Robot., vol. 33, no. 5, pp. 1255–1262, Oct. 2017. [75] A. Bartoli and P. Sturm, “Structure-from-motion using lines: Representation, triangulation, and bundle adjustment,” Comput. Vis. Image Understanding, vol. 100, no. 3, pp. 416–441, 2005. [76] X. Zuo, X. Xie, Y. Liu, and G. Huang, “Robust visual SLAM with point and line features,” in Proc. 2017 IEEE/RSJ Int. Conf. Intell. Robots Syst., IEEE, 2017, pp. 1775–1782. [77] Y. Yang, P. Geneva, K. Eckenhoff, and G. Huang, “Visual-inertial odometry with point and line features,” in Proc. 2019 IEEE/RSJ Int. Conf. Intell. Robots Syst., IEEE, 2019, pp. 2447–2454. [78] R. Kümmerle, G. Grisetti, H. Strasdat, K. Konolige, and W. Burgard, “G2o: A general framework for graph optimization,” in Proc. 2011 IEEE Int. Conf. Robot. Automat., 2011, pp. 3607–3613. [79] M. Kasper, S. McGuire, and C. Heckman, “A benchmark for visual-inertial odometry systems employing onboard illumination,” in Proc. 2019 IEEE/RSJ Int. Conf. Intell. Robots Syst., 2019, pp. 5256–5263. [80] A. Torii, J. Sivic, T. Pajdla, and M. Okutomi, “Visual place recognition with repetitive structures,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2013, pp. 883–890. [81] H. Taira et al., “Inloc: Indoor visual localization with dense matching and view synthesis,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2018, pp. 7199–7209. [82] A. Torii, R. Arandjelovic, J. Sivic, M. Okutomi, and T. Pajdla, “24/7 place recognition by view synthesis,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2015, pp. 1808–1817. [83] Y. Wang and G. S. Chirikjian, “Nonparametric second-order theory of error propagation on motion groups,” Int. J. Robot. Res., vol. 27, no. 11-12, pp. 1258–1273, 2008. [84] P. Denis, J. H. Elder, and F. J. Estrada, “Efficient edge-based methods for estimating Manhattan frames in urban imagery,” in Proc. Comput. Vis.– ECCV 2008: 10th Eur. Conf. Comput. Vis., Marseille, France, Springer, Oct. 2008, pp. 197–210. [85] N. Xue, S. Bai, F. Wang, G.-S. Xia, T. Wu, and L. Zhang, “Learning attraction field representation for robust line segment detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2019, pp. 1595–1603. [86] N. Xue et al., “Learning regional attraction for line segment detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 6, pp. 1998–2013, Jun. 2021. [87] Y. Xu, W. Xu, D. Cheung, and Z. Tu, “Line segment detection using transformers without edges,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 4257–4266. [88] X. Dai, H. Gong, S. Wu, X. Yuan, and Y. Ma, “Fully convolutional line parsing,” Neurocomputing, vol. 506, pp. 1–11, 2022. [89] H. Zhang, Y. Luo, F. Qin, Y. He, and X. Liu, “ELSD: Efficient line segment detector and descriptor,” in Proc. IEEE/CVF Int. Conf. Comput. Vis., 2021, pp. 2969–2978. [90] J. Lee and S.-Y. Park, “PLF-VINS: Real-time monocular visual-inertial slam with point-line fusion and parallel-line fusion,” IEEE Robot. Automat. Lett., vol. 6, no. 4, pp. 7033–7040, Oct. 2021. [91] S. Leutenegger, S. Lynen, M. Bosse, R. Siegwart, and P. Furgale, “Keyframe-based visual–inertial odometry using nonlinear optimization,” Int. J. Robot. Res., vol. 34, no. 3, pp. 314–334, 2015. [92] H. Lim, J. Jeon, and H. Myung, “UV-SLAM: Unconstrained line-based slam using vanishing points for structural mapping,” IEEE Robot. Automat. Lett., vol. 7, no. 2, pp. 1518–1525, Apr. 2022. [93] A. Rosinol et al., “Kimera: From SLAM to spatial perception with 3D dynamic scene graphs,” Int. J. Robot. Res., vol. 40, no. 12-14, pp. 1510–1546, 2021. [94] P. Geneva, K. Eckenhoff, W. Lee, Y. Yang, and G. Huang, “Openvins: A research platform for visual-inertial estimation,” in Proc. 2020 IEEE Int. Conf. Robot. Automat., IEEE, 2020, pp. 4666–4672. [95] F. Shu, J. Wang, A. Pagani, and D. Stricker, “Structure plp-slam: Efficient sparse mapping and localization using point, line and plane for monocular, RGB-D and stereo cameras,” in Proc. 2023 IEEE Int. Conf. Robot. Automat., IEEE, 2023, pp. 2105–2112. [96] A. Cramariuc et al., “maplab 2.0–a modular and multi-modal mapping framework,” IEEE Robot. Automat. Lett., vol. 8, no. 2, pp. 520–527, Feb. 2023. [97] Y. Wang, B. Xu, W. Fan, and C. Xiang, “A robust and efficient loop closure detection approach for hybrid ground/aerial vehicles,” Drones, vol. 7, no. 2, pp. 135–154, 2023. [98] X. Peng, Z. Liu, W. Li, P. Tan, S. Y. Cho, and Q. Wang, “Dvi-slam: A dual visual inertial SLAM network,” in Proc. 2024 IEEE Int. Conf. Robot. Automat., IEEE, 2024, pp. 12020–12026. [99] M. Burri et al., “The EuRoC micro aerial vehicle datasets,” Int. J. Robot. Res., vol. 35, no. 10, pp. 1157–1163, 2016. [100] M. Grupp, “evo: Python package for the evaluation of odometry and SLAM,” 2017. [Online]. Available: https://github.com/MichaelGrupp/ evo [101] E. Brachmann, T. Cavallari, and V. A. Prisacariu, “Accelerated coordinate encoding: Learning to relocalize in minutes using RGB and poses,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 5044– 5053. [102] J. Revaud, J. Almazán, R. S. Rezende, and C. R. d. Souza, “Learning with average precision: Training image retrieval with a listwise loss,” in Proc. IEEE/CVF Int. Conf. Comput. Vis., 2019, pp. 5107–5116. [103] Y. Ge, H. Wang, F. Zhu, R. Zhao, and H. Li, “Self-supervising fine-grained region similarities for large-scale image localization,” in Proc. Comput. Vis.–ECCV 2020: 16th Eur. Conf., Glasgow, U.K., Springer, Oct. 2020, pp. 369–386. [104] G. Berton, G. Trivigno, B. Caputo, and C. Masone, “Eigenplaces: Training viewpoint robust models for visual place recognition,” in Proc. IEEE/CVF Int. Conf. Comput. Vis., 2023, pp. 11080–11090. [105] D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis., vol. 60, pp. 91–110, 2004. [106] Y. Tian, X. Yu, B. Fan, F. Wu, H. Heijnen, and V. Balntas, “Sosnet: Second order similarity regularization for local descriptor learning,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2019, pp. 11016–11025. [107] L. Cavalli, V. Larsson, M. R. Oswald, T. Sattler, and M. Pollefeys, “Handcrafted outlier detection revisited,” in Proc. Comput. Vis.–ECCV: 16th Eur. Conf., Glasgow, U.K., Springer, Aug. 2020, pp. 770–787. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/694810000d3115b0dc3001395d0140c999491a1872812613b05f6e0b15fd9426.jpg) Kuan Xu received the B.E. and M.E. degrees in electrical engineering from the Harbin Institute of Technology, Harbin, China, in 2016 and 2018, respectively. He is currently working toward the Ph.D. degree in electrical and electronic engineering with the Department of Electrical and Electronic Engineering, Nanyang Technological University, Singapore. From 2018 to 2020, he worked as a Robot Algorithm Engineer with Tencent Holdings Ltd, Beijing, China. From 2020 to 2022, he served as a Senior Robot Algorithm Engineer with Geekplus Technology Company, Ltd., Beijing. His research interests include visual SLAM, robot localization, and perception. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/c77aba08919e0e842e1e439cafa2c627ec8e7b645aceb2cf72cce240e67e6a05.jpg) Yuefan Hao received the B.E. degree in communication engineering from the Kunming University of Science and Technology, Kunming, China, in 2017, and the M.E. degree in electrical and communication engineering from the University ofElectronic Science and Technology of China, Chengdu, China, in 2020. From 2020 to 2024, he served as a Robot Algorithm Engineer with Geekplus Technology Company, Ltd., Beijing, China. His research interests include computer vision, deep learning, and robotics. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/bf05506bc834faab4a1f5192b9ac388f795b9054b5e41942a76b38019fea7e34.jpg) Shenghai Yuan (Member, IEEE) received the B.S. and Ph.D. degrees in electrical and electronic engineering from Nanyang Technological University, Singapore, in 2013 and 2019, respectively. He is a Postdoctoral Senior Research Fellow with the Centre for Advanced Robotics Technology Innovation (CARTIN), Nanyang Technological University, Singapore. He has contributed over 70 papers to journals such as TRO, IJRR, TIE, and RAL, and to conferences including ICRA, CVPR, ICCV, NeurIPS, and IROS. He has also contributed over 10 technical disclosures and patents. His research interests include robotics perception and navigation. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/22dbf8c8cf19105009382bc5ce3c265555d8ac15080d0299ddd6f9e72c901e37.jpg) Dr. Yuan was the recipient of the second place in the academic track of the 2021 Hilti SLAM Challenge, third place in the visual-inertial track of the 2023 ICCV SLAM Challenge, the IROS 2023 Best Entertainment and Amusement Paper Award, and the Outstanding Reviewer Award at ICRA 2024. Currently, he serves as an Associate Editor for Unmanned Systems and as a Guest Editor for the Electronics Special Issue on Advanced Technologies of Navigation for Intelligent Vehicles. He served as the organizer of the CARIC UAV Swarm Challenge and Workshop at the 2023 CDC and the UG2 Anti-drone Challenge and Workshop at CVPR 2024. Currently, he is the organizer ofthe second CARIC UAV Swarm Challenge and Workshop at IROS 2024. ![](images/2025_AirSLAM__An_Efficient_and_Illumination-Robust_Point-Line/0f0f56ec004c22a079e0c4ccf5fc4847ea11b1b2e40d8b57825838eb4a38ba38.jpg) Chen Wang (Senior Member, IEEE) received the B.Eng. degree in electrical engineering from the Beijing Institute of Technology (BIT), Beijing, China, in 2014, and the Ph.D. degree in electrical engineering from Nanyang Technological University (NTU), Singapore, in 2019. He was a Postdoctoral Fellow with the Robotics Institute, Carnegie Mellon University (CMU), Pittsburgh, PA, USA. He is an Assistant Professor and leading the Spatial AI and Robotics (SAIR) Lab, Department of Computer Science and Engineering, University at Buffalo (UB), NY, USA. His research interests include spatial AI and robotics. Dr. Wang is an Associate Editor for the International Journal of Robotics Research (IJRR) and IEEE Robotics and Automation Letters (RA-L) and an Associate Co-chair for the IEEE RAS Technical Committee (TC) for Computer and Robot Vision. He served as an Area Chair for the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), the IEEE International Conference on Robotics and Automation (ICRA), and the Conference on Neural Information Processing Systems (NeurIPS). Since 1992, he has been with the School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, where he is currently a President’s Chair and Director, Center for Advanced Robotics Technology Innovation. He served as the Head of Division of Control and Instrumentation and Co-Director, Delta-NTU Corporate Lab for Cyber-Physical Systems. From 1986 to 1989, he held teach- Lihua Xie (Fellow, IEEE) received the Ph.D. degree in electrical engineering from the University of Newcastle, Australia, in 1992. ing appointments with the Department ofAutomatic Control, Nanjing University of Science and Technology, Nanjing, China. His research interests include robust control and estimation, networked control systems, multiagent networks, and unmanned systems. Dr. Xie is an Editor-in-Chief for Unmanned Systems and has served as Editor of IET Book Series in Control and Associate Editor of a number of journals including IEEE TRANSACTIONS ON AUTOMATIC CONTROL, Automatica, IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, IEEE TRANSACTIONS ON NETWORK CONTROL SYSTEMS, and IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS-II. He was an IEEE Distinguished Lecturer (January 2012–December 2014). He is a Fellow of Academy of Engineering Singapore, IEEE, IFAC, and CAA.