System Data Recorder (SDR)
A lifecycle node and executable for recording topic data to a rosbag2 bag, while simultaneously copying the split bag files to another location as each bag file is completed. This is useful, for example, to copy bag data files to an external disc during recording as each data file is completed, rather than waiting until all data is recorded before copying the entire bag at once (an operation that can take a significant time if the bag is large).
The copying function requires that a maximum file size for bag files be enabled. Otherwise no splitting will be performed and the files will not be copied until recording is terminated.
Compilation
The SDR requires two features not yet available in the rosbag2 main branch or binary releases. You will need to compile rosbag2 from source, applying the following two pull requests to the source prior to compiling it.
Create a colcon workspace with the SDR source code in it, and compile the workspace.
Use
Lifecycle management
Run the executable to start the node.
ros2 run system_data_recorder system_data_recorder
In a separate terminal, use the lifecycle manager to configure and activate the node.
ros2 lifecycle set sdr configure
ros2 lifecycle set sdr activate
This will enable recording of data to the bag. To pause recording, deactivate the node.
ros2 lifecycle set sdr deactivate
From here, recording can be resumed by re-activating the node, or recording can be terminated by cleaning up the node.
ros2 lifecycle set sdr cleanup
Once cleaned up, the node will copy the final files of the bag, as well as any metadata, to the backup destination.
Configuring
The SDR is configured by the arguments passed to the node's constructor. These are not currently exposed as command line arguments or ROS parameters, so they must be changed in the source code. See the constructor's documentation block for the possible parameters.
By default, the SDR will:
- Record from the
/chattertopic, expectingstd_msgs/msg/Stringdata. - Record to a bag named
test_bag. - Copy bag files to the directory
copied_bag/test_bag. - Split bag files every 100,000 bytes.
A simple test
In one terminal, start publishing data on the /chatter topic.
ros2 run demo_nodes_cpp talker
In another terminal, start the SDR node.
ros2 run system_data_recorder system_data_recorder
In a third terminal, configure and activate the SDR.
ros2 lifecycle set sdr configure
ros2 lifecycle set sdr activate
Wait a minute or two for 100,000 bytes of data to be recorded. Navigate to the directory copied_bag and observe that there is a test_bag directory containing the first data file of the bag.
Deactivate and cleanup the SDR.
ros2 lifecycle set sdr deactivate
ros2 lifecycle set sdr cleanup
Again navigate to the copied_bag directory, and observe that the test_bag directory now contains all the data files of the bag, and a metadata.yaml file.
This is a complete, usable bag. This can be demonstrated by stopping the talker node, then starting the listener node and playing the bag file.
ros2 run demo_nodes_cpp listener
ros2 bag play copied_bag/test_bag
The recorded data will be echoed by the listener node.
Lifecycle transition behaviours
on_configure
In the on_configure state, the node sets up the rosbag2 infrastructure for recording by creating a writer and opening the storage. It also sets up the worker thread that will copy files in parallel to the recording of data. A callback is registered with the writer so that the node will get informed each time a new file is created in the bag.
on_activate
In the on_activate transition, the node simply notifies the worker thread that it is recording. Data will be written to the bag automatically because the state changes to Active.
on_deactivate
In the on_deactivate transition, the node simply notifies the worker thread that it is paused. Data will not be written to the bag because the state changes to Inactive.
on_cleanup
When cleaning up, the node needs to stop recording, stop receiving data (i.e. unsubscribe from topics), and ensure that the final files of the bag (the last data file and the metadata file, which gets written when the writer object destructs) are copied to the destination directory.
on_shutdown
If not already performed, the shutdown transition performs the same functions as the cleanup transition.
