Tsdf-plusplus - TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction

TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction

TSDF++ is a novel multi-object TSDF formulation that can encode multiple object surfaces at each voxel. In a multiple dynamic object tracking and reconstruction scenario, a TSDF++ map representation allows maintaining accurate reconstruction of surfaces even while they become temporarily occluded by other objects moving in their proximity. At the same time, the representation allows maintaining a single volume for the entire scene and all the objects therein, thus solving the fundamental challenge of scalability with respect to the number of objects in the scene and removing the need for an explicit occlusion handling strategy.

Citing

When using TSDF++ in your research, please cite the following publication:

Margarita Grinvald, Federico Tombari, Roland Siegwart, and Juan Nieto, TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction, in 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021. [Paper] [Video]

@INPROCEEDINGS{grinvald2021tsdf,
  author={Grinvald, Margarita and Tombari, Federico and Siegwart, Roland and Nieto, Juan},
  booktitle={2021 IEEE International Conference on Robotics and Automation (ICRA)}, 
  title={{TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction}}, 
  year={2021},
  volume={},
  number={},
  pages={14192-14198},
  doi={10.1109/ICRA48506.2021.9560923}}

Installation

The installation has been tested on Ubuntu 16.04 and Ubutnu 20.04.

Requirements

Install dependencies

Install ROS following the instructions at the ROS installation page. The full install (ros-kinetic-desktop-full, ros-melodic-desktop-full) are recommended.

Make sure to source your ROS setup.bash script by following the instructions on the ROS installation page.

Installation on Ubuntu

In your terminal, define the installed ROS version and name of the catkin workspace to use:

export ROS_VERSION=kinetic # (Ubuntu 16.04: kinetic, Ubuntu 18.04: melodic)
export CATKIN_WS=~/catkin_ws

If you don't have a catkin workspace yet, create a new one:

mkdir -p $CATKIN_WS/src && cd $CATKIN_WS
catkin init
catkin config --extend /opt/ros/$ROS_VERSION --merge-devel 
catkin config --cmake-args -DCMAKE_CXX_STANDARD=14 -DCMAKE_BUILD_TYPE=Release
wstool init src

Clone the tsdf-plusplus repository over HTTPS (no Github account required) and automatically fetch dependencies:

cd $CATKIN_WS/src
git clone https://github.com/ethz-asl/tsdf-plusplus.git
wstool merge -t . tsdf-plusplus/tsdf_plusplus_https.rosinstall
wstool update

Alternatively, clone over SSH (Github account required):

cd $CATKIN_WS/src
git clone [email protected]:ethz-asl/tsdf-plusplus.git
wstool merge -t . tsdf-plusplus/tsdf_plusplus_ssh.rosinstall
wstool update

Build and source the TSDF++ packages:

catkin build tsdf_plusplus_ros rgbd_segmentation mask_rcnn_ros cloud_segmentation
source ../devel/setup.bash # (bash shell: ../devel/setup.bash,  zsh shell: ../devel/setup.zsh)

Troubleshooting

Compilation freeze

By default catkin build on a computer with N CPU cores will run N make jobs simultaneously. If compilation seems to hang forever, it might be running low on RAM. Try limiting the number of maximum parallel build jobs through the -jN flag to a value way lower than your CPU count, i.e.

catkin build tsdf_plusplus_ros rgbd_segmentation mask_rcnn_ros cloud_segmentation -j4

If it still freezes at compilation time, you can go as far as limiting the maximum number of parallel build jobs and max load to 1 through the -lN flag:

catkin build tsdf_plusplus_ros rgbd_segmentation mask_rcnn_ros cloud_segmentation -j1 -l1

License

The code is available under the MIT license.

Comments
  • ROS2 Support

    ROS2 Support

    If I would add ROS2 support to your project in a forked repo, would you be open to review it as a PR into your repo (either master or special ros2 branch) ?

  • Compilation error: ‘struct std::atomic<bool>’ has no member named ‘set’

    Compilation error: ‘struct std::atomic’ has no member named ‘set’

    Hi, I am trying to build this seemingly wonderful package and can't seem to get over that error message... Does anybody knows what I am supposed to do to fix it ? Thanks in advance.

    Errors     << tsdf_plusplus:make /d/qserdel/ros_ws/voxbloxpp_ws/logs/tsdf_plusplus/build.make.004.log                                                                                                     
    /d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/core/map.cc: In member function ‘void Map::transformLayer(const ObjectID&, const Transformation&)’:
    /d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/core/map.cc:235:25: error: ‘struct std::atomic<bool>’ has no member named ‘set’
      235 |     mo_block->updated().set();
          |                         ^~~
    /d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/core/map.cc:322:27: error: ‘struct std::atomic<bool>’ has no member named ‘set’
      322 |       mo_block->updated().set();
          |                           ^~~
    /d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/core/map.cc: In member function ‘void Map::removeObject(const ObjectID&)’:
    /d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/core/map.cc:356:25: error: ‘struct std::atomic<bool>’ has no member named ‘set’
      356 |     mo_block->updated().set();
          |                         ^~~
    /d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/mesh/mesh_integrator.cc: In member function ‘bool MOMeshIntegrator::generateMesh(bool, bool)’:
    /d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/mesh/mesh_integrator.cc:58:49: error: ‘Update’ has not been declared
       58 |     map_->getMapLayerPtr()->getAllUpdatedBlocks(Update::kMesh, &all_map_blocks);
          |                                                 ^~~~~~
    /d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/mesh/mesh_integrator.cc: In member function ‘void MOMeshIntegrator::generateMeshBlocksFunction(const BlockIndexList&, bool, voxblox::ThreadSafeIndex*)’:
    /d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/mesh/mesh_integrator.cc:104:24: error: ‘struct std::atomic<bool>’ has no member named ‘reset’
      104 |       block->updated().reset(Update::kMesh);
          |                        ^~~~~
    /d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/mesh/mesh_integrator.cc:104:30: error: ‘Update’ has not been declared
      104 |       block->updated().reset(Update::kMesh);
          |                              ^~~~~~
    make[2]: *** [CMakeFiles/tsdf_plusplus.dir/build.make:89: CMakeFiles/tsdf_plusplus.dir/src/core/map.cc.o] Error 1
    make[2]: *** Waiting for unfinished jobs....
    /d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/integrator/integrator.cc: In member function ‘MOVoxel* Integrator::allocateStorageAndGetVoxelPtr(const GlobalIndex&, voxblox::Block<MOVoxel>::Ptr*, voxblox::BlockIndex*)’:
    /d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/integrator/integrator.cc:435:28: error: ‘struct std::atomic<bool>’ has no member named ‘set’
      435 |   (*last_block)->updated().set();
    
  • How to run

    How to run

    Hi, your work is amazing and thank you very much for the code! Could you please provide some instructions on how to run it? I would really like to test it on some datasets.

  • ROS Noetic compilation

    ROS Noetic compilation

    When compiling for ROS Noetic on Ubuntu 20.04, I encountered the error

    CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:117 (find_package): Could not find a package configuration file provided by "boost_signals" (requested version 1.71.0) with any of the following names:

    boost_signalsConfig.cmake
    boost_signals-config.cmake
    

    Add the installation prefix of "boost_signals" to CMAKE_PREFIX_PATH or set "boost_signals_DIR" to a directory containing one of the above files. If "boost_signals" provides a separate development package or SDK, be sure it has been installed. Call Stack (most recent call first): /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component) /usr/share/cmake-3.16/Modules/FindBoost.cmake:443 (find_package) CMakeLists.txt:6 (find_package)

    This was solved by simply removing signals from src/pcl_catkin/geometry2/tf2/CMakeLists.txt (see this for context).

    Thanks for the amazing work!

  • How to preprocess input data before passing it to tsdf_plusplus_node?

    How to preprocess input data before passing it to tsdf_plusplus_node?

    Hi there! Thank you for your great work. But how exactly do I need to preprocess the input data before passing it to the tsdf_plusplus_node on the /segment_pointcloud topic? Thanks in advance!

Blazingly fast multi-object tracker. Works on 1 (x, y) point per object.
Blazingly fast multi-object tracker. Works on 1 (x, y) point per object.

Norfair++ This is a C++ implementation of Techainer's Norfair, which originates from Norfair, a library for real-time 2D object tracking. Its function

Jan 26, 2022
Multi-dimensional dynamically distorted staggered multi-bandpass LV2 plugin
Multi-dimensional dynamically distorted staggered multi-bandpass LV2 plugin

B.Angr A multi-dimensional dynamicly distorted staggered multi-bandpass LV2 plugin, for extreme soundmangling. Based on Airwindows XRegion. Key featur

Nov 7, 2022
Unicorn is a lightweight, multi-platform, multi-architecture CPU emulator framework, based on QEMU.
Unicorn is a lightweight, multi-platform, multi-architecture CPU emulator framework, based on QEMU.

Unicorn Engine Unicorn is a lightweight, multi-platform, multi-architecture CPU emulator framework, based on QEMU. Unicorn offers some unparalleled fe

Nov 7, 2021
Path Tracking PID offers a tuneable PID control loop, decouling steering and forward velocity
Path Tracking PID offers a tuneable PID control loop, decouling steering and forward velocity

path_tracking_pid Overview Path Tracking PID offers a tuneable PID control loop decouling steerting and forward velocity. The forward velocity is gene

Nov 13, 2022
KDevelop plugin for automatic time tracking and metrics generated from your programming activity.

Wakatime KDevelop Plugin Installation instructions Make sure the project is configured to install to the directory of your choice: In KDevelop, select

Oct 13, 2021
Tiny blocker of Windows tracking and telemetry written in plain C++/Win32 API.

Tiny blocker of Windows tracking and telemetry written in plain C++/Win32 API. Just run once as admin and forget. No questions asked. No harmful actions performed like other Windows spying blockers try.

Mar 30, 2022
GPS tracking system
GPS tracking system

GPS_ASU_PROJECT_67 GPS Tracking System project for CSE211 - Introduction to Embedded Systems course. Provided by students of Team 67 (Junior level) in

Jun 13, 2021
A Keg tracking system that monitors weight

KegScale A Keg tracking system that monitors weight Right now the arduino zeros the scale when it turns on and subracts the weight of 5 gallon corny k

Mar 2, 2022
Arduino GPS Car Tracking with GPRS/HTTP
 Arduino GPS Car Tracking with GPRS/HTTP

Arduino GPS Car Tracking with GPRS/HTTP this is a simple car tracking source and module to start hacking around Overview Overview DIY Module Features

Nov 26, 2021
EDACS trunk tracking using RTL2832. rtl_fm, sdr++, gqrx, or gnuradio.
EDACS trunk tracking using RTL2832. rtl_fm, sdr++, gqrx, or gnuradio.

EDACS-FM Florida Man Edition EDACS 9600 Trunk Tracking software for Linux, *Nix, BSD systems (and maybe others) using two RTL2838 dongles(or anything

Jun 26, 2022
Environment Tracking for Smart Farm.

IOT-EnvironmentTracking This is my school project about IOT Fundamentals. Components included: • ESP32 DevKitC • DHT-11 • PIR HC-SR501 • LCD 1602 I2C

Dec 8, 2021
🎮 Plants vs. Zombies multiplayer battle, developed via reverse engineering, inline hook and dynamic-link library injection. Two online players defend and attack as the plant side and zombie side respectively.
🎮 Plants vs. Zombies multiplayer battle, developed via reverse engineering, inline hook and dynamic-link library injection. Two online players defend and attack as the plant side and zombie side respectively.

Plants vs. Zombies Online Battle This project has two original repositories: https://github.com/czs108/Plants-vs.-Zombies-Online-Battle https://github

Oct 14, 2021
MDE is a model extraction tool that converts Destiny 2 dynamic models into fbx files supporting textures, skeletons, and all provided vertex data.

MDE is a model extraction tool that converts Destiny 2 dynamic models into fbx files. A dynamic model is one that is animated or is spawned in during the game.

Sep 2, 2022
dwm is an extremely fast, small, and dynamic window manager for X.

dwm - dynamic window manager dwm is an extremely fast, small, and dynamic window manager for X. My Patches This is in the order that I patched everyth

Sep 22, 2022
An experimental dynamic malware unpacker based on Intel Pin and PE-sieve

Pin'n'Sieve A dynamic malware unpacker based on Intel Pin and PE-sieve (deploys PE-sieve scan on specific triggers). Caution: during the process the m

Nov 27, 2022
Improved Fractions Calculator using structures, user input parsing, and dynamic operator switching

Improved Fractions Calculator Program Structure: Main File: Runs fnctions from various header files. IO.h Header file containing IO functions, Interfa

Dec 5, 2021
The home for algorithms ranging from searching to search all the way to dynamic programming, branch and bound, etc.

Algorithms The home for algorithms ranging from searching and sorting all the way to dynamic programming algorithms, divide and conquer, etc. What are

Dec 6, 2021
Implementation and tutorial for a dynamic vector data-structure in C.

Vec: A Dynamic Vector in C The subject.md file contains exercises for this implementation if you want to do it yourself. In that case, don't peek at t

Oct 17, 2022
A model checker for the Dynamic Logic of Propositional Assignments (DL-PA) with solving and parameterized random formula generation functionalities.

A model checker for the Dynamic Logic of Propositional Assignments (DL-PA) with solving and parameterized random formula generation functionalities.

Dec 31, 2021