Tightly coupled GNSS-Visual-Inertial system for locally smooth and globally consistent state estimation in complex environment.

GVINS

GVINS: Tightly Coupled GNSS-Visual-Inertial Fusion for Smooth and Consistent State Estimation. paper link

Authors: Shaozu CAO, Xiuyuan LU and Shaojie SHEN

GVINS is a non-linear optimization based system that tightly fuses GNSS raw measurements with visual and inertial information for real-time and drift-free state estimation. By incorporating GNSS pseudorange and Doppler shift measurements, GVINS is capable to provide smooth and consistent 6-DoF global localization in complex environment. The system framework and VIO part are adapted from VINS-Mono. Our system contains the following features:

  • global 6-DoF estimation in ECEF frame;
  • multi-constellation support (GPS, GLONASS, Galileo, BeiDou);
  • online local-ENU frame alignment;
  • global pose recovery in GNSS-unfriendly or even GNSS-denied area.

1. Prerequisites

1.1 C++11 Compiler

This package requires some features of C++11.

1.2 ROS

This package is developed under ROS Kinetic environment.

1.3 Eigen

Our code uses Eigen 3.3.3 for matrix manipulation.

1.4 Ceres

We use ceres 1.12.0 to solve the non-linear optimization problem.

1.5 gnss_comm

This package also requires gnss_comm for ROS message definitions and some utility functions. Follow those instructions to build the gnss_comm package.

2. Build GVINS

Clone the repository to your catkin workspace (for example ~/catkin_ws/):

cd ~/catkin_ws/src/
git clone https://github.com/HKUST-Aerial-Robotics/GVINS.git

Then build the package with:

cd ~/catkin_ws/
catkin_make
source ~/catkin_ws/devel/setup.bash

If you encounter any problem during the building of GVINS, we recommend you to try docker first.

3. Docker Support

To simplify the building process, we add docker in our code. Docker is like a sandbox so it can isolate our code from your local environment. To run with docker, first make sure ros and docker are installed on your machine. Then add your account to docker group by sudo usermod -aG docker $USER. Logout and re-login to avoid the Permission denied error, then type:

cd ~/catkin_ws/src/GVINS/docker
make build

The docker image gvins:latest should be successfully built after a while. Then you can run GVINS with:

./run.sh LAUNCH_FILE

(for example ./run.sh visensor_f9p.launch). Open another terminal and play your rosbag file, then you should be able to see the result. If you modify the code, simply re-run ./run.sh LAUNCH_FILE to update.

4. Run GVINS with our dataset

Download our GVINS-Dataset and launch GVINS via:

roslaunch gvins visensor_f9p.launch

Open another terminal and launch the rviz by:

rviz -d ~/catkin_ws/src/GVINS/config/gvins_rviz_config.rviz

Then play the bag:

rosbag play sports_field.bag

5. Acknowledgements

The system framework and VIO part are adapted from VINS-Mono. We use camodocal to model the camera and ceres to solve the optimization problem.

6. Licence

The source code is released under GPLv3 license.

Owner
Comments
  • Running GVINS on ARM device

    Running GVINS on ARM device

    @shaozu Hi,thanks for your nice work.

    I just started working on GVINS and I'm trying to use this project for real-time drone navigation. That means, I need to run the program on ARM embedded computer.

    Firstly I run GVINS on my PC and laptop and get nice result(sports_field.bag). IMG_7646(20211215-171818)

    Then I run GVINS on NVIDIA Jetson TX2 and NVIDIA Jetson Xavier NX, but I got a problem:

    1. The gnss_enu_path seems to converge slowly and the results don't look as good as those on my PC/laptop;
    2. The path seems to start at a wrong start point and yaw angle.

    As these pictures show (Jetson TX2 with CUDA in feature_tracker): IMG_7635 IMG_7637 IMG_7638 IMG_7641

    And then I use CUDA in feature_tracker to speed up calculations to reduce CPU load, but it didn't seem to help.

    So is this result due to the lack of computing power on embedded computer?? But the CPU load is not too heavy...... Or is it processor architecture (ARM and x86)?

  •  time-synchronized logic with UTC

    time-synchronized logic with UTC

    Dear Officer,

    Thank you for sharing your great job. I wanna ask about the time-synchronized logic here. https://github.com/HKUST-Aerial-Robotics/GVINS/blob/c3885792c55f996f78807445fa2707f64af51d84/estimator/src/estimator_node.cpp#L274

    Here: time_diff_gnss_visensor = next_pulse_time - trigger_msg->header.stamp.toSec() + LEAP_SECONDS;

    next_pulse_time: pps GPS time in UTC format, remove the leap seconds in epoch2time() already? Correct me if I am wrong trigger_msg->header.stamp.toSec(): VI device's UTC time LEAP_SECONDS : 18seconds

    As the system is time_sync and gps_time = utc_time + LEAP_SECONDS according to here.http://leapsecond.com/java/gpsclock.htm Why do you need to add the LEAP_SECONDS to calculate the time_diff?

    Print the time difference of time_diff_gnss_visensor is quite large. I am running with data2, complex_environment.bag image

  • gnss_enable is enabled, can not get global path

    gnss_enable is enabled, can not get global path

    @shaozu Hello, disturb to again. I have a question about how to enable gnss_enable.

    as you can see #25. If I set gnss_enable:0, it can get the nice local position. however, when I set gnss_enable:1, it can not get the gloabl position. So i start to debug and get some questions. hope your reply. question: 1: I found that I have no topic /external_trigger is published. how does it work? or is it must? 2: what 's the function of the parameter gnss_local_time_diff ? 3: Is there some reference for explanation gnss_iono_default_parameters. sorry, I have not found any reference. Can it be changed? what it depends on? 4: In GNSSVIAlign() function, I found it always return false as the reason of gnss_meas_buf[i].empty() -> false && gnss_meas_buf[i].size() = 0. https://github.com/HKUST-Aerial-Robotics/GVINS/blob/d2cf40b49c6eb0e6ad3caa4f613713983be3fd74/estimator/src/estimator.cpp#L566-L570 But I can confirm that it has push the data in the buffer here https://github.com/HKUST-Aerial-Robotics/GVINS/blob/d2cf40b49c6eb0e6ad3caa4f613713983be3fd74/estimator/src/estimator.cpp#L326-L327. Very confused me.

    Thank you for sharing your great job.

  • Issues when enabling gnss

    Issues when enabling gnss

    Hi, thanks for the nice work. The demo sports_field really shows perfect performance.

    1. What's the necessary condition to generate the global path?

    The local gvins/path can be generated with RealSense D435i without gnss. But after piping the gnss topics correctly (also using ublox_driver from your repo), the altitude and longitude information do not show up and gvins/enu_path is not generated. The yaml setup for gnss part is the same as provided file.

    2. The following parameters do not show up in the code. What's the function of it?

    gnss_iono_default_parameters: !!opencv-matrix   # unused parameters
       rows: 1
       cols: 8
       dt: d
       data: [0.1118E-07,  0.2235E-07, -0.4172E-06,  0.6557E-06,
              0.1249E+06, -0.4424E+06,  0.1507E+07, -0.2621E+06]
    
  • Could not find the required component 'gnss_comm'.

    Could not find the required component 'gnss_comm'.

    system:Ubuntu 18.04.4 eigen:3.3.4 ceres:1.14 (but there is no error in build gnss_comm)

    I have build the gnss_comm package as instruction

    mkdir - p gnss_comm_ws/src
    cd gnss_comm_ws/src 
    git clone https://github.com/HKUST-Aerial-Robotics/gnss_comm.git
    cd ..
    catkin_make
    source ./devel/setup.bash
    

    there is no error then I build GVINS in another workspace as follows

    mkdir -p GVINS_ws/src
    cd GVINS_ws/src
    git clone https://github.com/HKUST-Aerial-Robotics/GVINS.git
    cd ..
    catkin_make
    

    the error is:

    Could not find the required component 'gnss_comm'. The following CMake error indicates that you either need to install the package with the same name or change your environment so that it can be found.
    CMake Error at /opt/ros/melodic/share/catkin/cmake/catkinConfig.cmake:83 (find_package):
      Could not find a package configuration file provided by "gnss_comm" with
      any of the following names:
    
        gnss_commConfig.cmake
        gnss_comm-config.cmake
    
      Add the installation prefix of "gnss_comm" to CMAKE_PREFIX_PATH or set
      "gnss_comm_DIR" to a directory containing one of the above files.  If
      "gnss_comm" provides a separate development package or SDK, be sure it has
      been installed.
    Call Stack (most recent call first):
      GVINS/estimator/CMakeLists.txt:8 (find_package)
    
    

    so if I have to give path in right path? or other method?

  • Question about UBLOX Config

    Question about UBLOX Config

    hello @shaozu I have follow your steps, do the VI slam, The result is below, looks good. Screenshot from 2021-12-17 16-26-39

    I have two question, hopes your suggestions: question 1: I have a f9p module, but I have not rtcm data, so it will never get rtk fix. Can this data fuse with the image&imu data? question 2: I use my f9p module, run the command, roslaunch ublox_driver ublox_driver.launch, I can see the topic show as below:

    /ublox_driver/ephem /ublox_driver/glo_ephem /ublox_driver/iono_params /ublox_driver/range_meas /ublox_driver/receiver_lla /ublox_driver/receiver_pvt /ublox_driver/time_pulse_info

    How ever, when I rostopic echo /ublox_driver/receiver_lla, no topic is published/display. Do I need some special config through u-center or some other tools?

  • How to dispaly the maps

    How to dispaly the maps

    @shaozu Hello, I am very like this project. it is my needed. I have run it in my pc. all works good. Thank you very much for you nice work.

    I have a simple question: I want know how to display this path trajectory/path in the maps as yours.what tools should i use. how to configure it? I am very appreciate that you can give some reference. thank you very much again. I can display the path in rviz without any map backgrounds. Screenshot from 2021-12-14 15-34-57

  • unit2rcv_pos in code

    unit2rcv_pos in code

    Hello I find you use dopp to constrain the position in code as follow Eigen::Matrix3d unit2rcv_pos; for (size_t i = 0; i < 3; ++i) { for (size_t j = 0; j < 3; ++j) { if (i == j) unit2rcv_pos(i, j) = (norm2-rcv2sat_ecef(i)*rcv2sat_ecef(i))/norm3; else unit2rcv_pos(i, j) = (-rcv2sat_ecef(i)*rcv2sat_ecef(j))/norm3; } } but this method don't mention in your paper, so can you give me some papers about this method

  • How can I get time of world frame?

    How can I get time of world frame?

    Hi, thanks for your great work. I want to evaluate accuracy of GVINS(without GNSS) which means I need time of reference frame (or world frame). How can I get this? Thanks in advance.

  • IMU noise unit

    IMU noise unit

    I am confused to the units of IMU parameters when I want to run GVINS with my own dataset. Would you mind if you could describe them thoroughly.

    #imu parameters       The more accurate parameters you provide, the better performance
    acc_n: 0.08          # accelerometer measurement noise standard deviation. #0.2   0.04
    gyr_n: 0.004         # gyroscope measurement noise standard deviation.     #0.05  0.004
    acc_w: 0.00004         # accelerometer bias random work noise standard deviation.  #0.02
    gyr_w: 2.0e-6       # gyroscope bias random work noise standard deviation.     #4.0e-5
    
  • Stereo Compatibility

    Stereo Compatibility

    Hello and thank you for a great piece of work! From what I gather, there is nothing stopping this method working with a pair of stereo cameras (such as the RS D455); I was just wondering what the changes needed to be done in the codebase would be and if you are looking forward to doing it

  • How to get standard deviation of gnss receiver clock bias and clock drift rate?

    How to get standard deviation of gnss receiver clock bias and clock drift rate?

    Thanks for your nice work. @shaozu

    I've got a new question about GNSS receiver dt and ddt factor.

    In your code, standard deviation of ddt is set to 0.1(unit: m/s ?) and the infomation matrix of dt is set to 50(unit: m ?). Could you please tell me why set them to these values?

  • Questions about running GVINS with my own device.

    Questions about running GVINS with my own device.

    Hi~bro. I ran the GVINS with my own device (including a realsense D455 as the VIS and a H-RTK-F9P as the GNSS module ). I set all the device right. And the GPS runs fine. But it shows "length error" in the Ublox-driver terminal windiow.But when I run the GVINS project with the estimate_extrinsic 2. But After I roslaunched the GVINS , the terminal windows shows only "init begins". And I opened the rviz , it had the original image and the feature point image but no pose. I echoed the enu_pose and other messages outputed by GVINS, they are all none. So why is this?

    image

    image

    Could you give me some advice about this? Thanks a lot.
    
  • IMU correction

    IMU correction

    Great work, code and paper!

    When looking at the datasheet of the ADIS16448 and your paper, your dataset should have X to the right of the helmet, Z axis to the front and Y axis down. This can be confirmed with the bag values (please correct me if I'm wrong) 20221024_gvins

    Similar, the camera matrix rotation need to be corrected to fit the IMU as this is the body frame (as indicated in the paper).

    Question: Where do you correct the IMU to real world coordinates system? I have not been able to found it. My coordinate system is not the same (X to the front, Y to the left and Z to the top), what make the algorithm fail

    Thanks!

  • Questions about running with my device

    Questions about running with my device

    Hi~ I used realsense D455 as the VIsensor and an H-RTK-F9P as the GNSS module. I set the estimate_extrinsic 2 and modified all the topic name But when I run the launch file. It showed this: "init begins" and image

    why is this? could you offer some advice? Thanks a lot.

  • Hardware Set-Up

    Hardware Set-Up

    Hey authors,

    Good work on the paper. I was wondering if you could provide details or a rough sketch of your experimental set-up so I may recreate and run it on my end. Thanks.

RRxIO - Robust Radar Visual/Thermal Inertial Odometry: Robust and accurate state estimation even in challenging visual conditions.
RRxIO - Robust Radar Visual/Thermal Inertial Odometry: Robust and accurate state estimation even in challenging visual conditions.

RRxIO - Robust Radar Visual/Thermal Inertial Odometry RRxIO offers robust and accurate state estimation even in challenging visual conditions. RRxIO c

Nov 30, 2022
A real-time, direct and tightly-coupled LiDAR-Inertial SLAM for high velocities with spinning LiDARs
A real-time, direct and tightly-coupled LiDAR-Inertial SLAM for high velocities with spinning LiDARs

LIMO-Velo [Alpha] ?? [16 February 2022] ?? The project is on alpha stage, so be sure to open Issues and Discussions and give all the feedback you can!

Nov 25, 2022
Iot-Surveillance-Car - This is a IOT Based Surveillance Car which can be controlled, tracked globally as well as its data can be accessed globally

Iot-Surveillance-Car - This is a IOT Based Surveillance Car which can be controlled, tracked globally as well as its data can be accessed globally. The camera on the front of the car can also be monitored globally. It can go anywhere where sim connection is available. 5th Sem Mini project

Nov 24, 2022
A dataset containing synchronized visual, inertial and GNSS raw measurements.
A dataset containing synchronized visual, inertial and GNSS raw measurements.

GVINS-Dataset Author/Maintainer: CAO Shaozu (shaozu.cao AT gmail.com), LU Xiuyuan (xluaj AT connect.ust.hk) This repository hosts dataset collected du

Nov 22, 2022
VID-Fusion: Robust Visual-Inertial-Dynamics Odometry for Accurate External Force Estimation
VID-Fusion: Robust Visual-Inertial-Dynamics Odometry for Accurate External Force Estimation

VID-Fusion VID-Fusion: Robust Visual-Inertial-Dynamics Odometry for Accurate External Force Estimation Authors: Ziming Ding , Tiankai Yang, Kunyi Zhan

Nov 18, 2022
ROS GNSS/INS driver for Inertial Labs positioning systems for the CARMA Platform
ROS GNSS/INS driver for Inertial Labs positioning systems for the CARMA Platform

CARMA Inertial Labs GNSS/INS Driver This is a fork of the Inertial Labs ROS package that is used for connecting to Inertial Labs GNSS/INS, IMU-P, AHRS

Dec 26, 2021
The movements of your RC vehicles are jerky and not smooth? This Arduino device will solve this issue by adding acceleration and deceleration ramps to the PWM signals!
The movements of your RC vehicles are jerky and not smooth? This Arduino device will solve this issue by adding acceleration and deceleration ramps to the PWM signals!

This is an Arduino Pro Mini 3.3V / 8MHz based RC servo ramp / delay generator Features: 4 RC servo PWM inputs and outputs (can be enhanced) Reads the

Apr 15, 2022
Non-blocking blinking patterns and smooth fade effects for your LEDs and buzzers.
Non-blocking blinking patterns and smooth fade effects for your LEDs and buzzers.

Arduino-Blinkenlight Featured on hackster.io! ?? Supercharge your status-LEDs ?? This library gives you non-blocking blinking patterns and smooth fade

Nov 6, 2022
Terrain generator with 5 visually distinct biomes, spread them in regions and smooth their borders
Terrain generator with 5 visually distinct biomes, spread them in regions and smooth their borders

termProject This is the repository of my term project: a terrain generator. Abstract The larger the scenario of a game, more time a player will spend

Oct 12, 2022
Continuous-Time Spline Visual-Inertial Odometry

Continuous-Time Spline Visual-Inertial Odometry Related Publications Direct Sparse Odometry, J. Engel, V. Koltun, D. Cremers, In IEEE Transactions on

Nov 18, 2022
Visual-inertial-wheel fusion odometry, better performance in scenes with drastic changes in light
Visual-inertial-wheel fusion odometry, better performance in scenes with drastic changes in light

VIW-Fusion An visual-inertial-wheel fusion odometry VIW-Fusion is an optimization-based viusla-inertial-wheel fusion odometry, which is developed as a

Nov 29, 2022
Best Method to get Globally Banned from the cfx.re community

Lua-Executor Best Method to get Globally Banned from the cfx.re community Since cheaters have been going crazy selling 'their' hacks, and often gets d

Jul 2, 2022
Wayfire plugin for handling touchpad gestures globally in a layer-shell surface

wf-globalgestures Global touchpad gestures plugin for Wayfire: implements a special protocol (also in this repo) that lets clients request that a part

Oct 3, 2022
RR4J is a tool that records java execution and later allows developers to replay locally.
RR4J is a tool that records java execution and later allows developers to replay locally.

RR4J [Record Replay 4 Java] RR4J is a tool that records java execution and later allows developers to replay locally. The tool solves one of the chall

Oct 31, 2022
Optimization-Based GNSS/INS Integrated Navigation System
Optimization-Based GNSS/INS Integrated Navigation System

OB_GINS Optimization-Based GNSS/INS Integrated Navigation System We open-source OB_GINS, an optimization-based GNSS/INS integrated navigation system.

Nov 24, 2022
C++ Library to Execute Leetcode Problems Locally

C++ Library to Execute Leetcode Problems Locally This repository contains a C++ library that helps to execute Leetcode problems in one line. header-on

Apr 7, 2022
Yet another matrix client. Click packaging for locally running on Ubuntu Touch

Cinny Click Packaging Cinny is a Matrix client focusing primarily on simple, elegant and secure interface. License Cinny source package licensed under

Nov 15, 2022
Visual Leak Detector for Visual C++ 2008-2015

Visual Leak Detector Introduction Visual C++ provides built-in memory leak detection, but its capabilities are minimal at best. This memory leak detec

Dec 1, 2022
fx is a workspace tool manager. It allows you to create consistent, discoverable, language-neutral and developer friendly command line tools.
fx is a workspace tool manager. It allows you to create consistent, discoverable, language-neutral and developer friendly command line tools.

fx is a workspace tool manager. It allows you to create consistent, discoverable, language-neutral and developer friendly command line tools.

Aug 27, 2022