Code organization

Code organization

This section presents the code organization in the repository.

Tree repository

The tree repository is defined as followed:

.
├── contrib
├── data
├── doc
├── figs
├── msg
├── notebook
└── scripts

More details are available here:

• Contrib
• Data
  • ablation_tests
  • figures
  • filtered_prediction
  • gps_data
  • ground_truth
  • list_error
  • list_tf
  • raw_prediction
  • sensors_extrinsic_calibration
  • static_points
  • total_stations
• Doc
• Figs
• Msg
• Notebook
• Scripts

Contrib

The folder contrib/ is used to store unofficial code which are still been tested or used only for research purposes.

Data

The folder data/ is used to store the unprocessed data coming from different experiments, and also the data post-processed. All the experiment data are stored into a folder which follows a model to process the data easily. This model of folder can be copied from the folder Model_experiments. The tree of this folder is as following:

.
├── ablation_tests
├── figures
├── filtered_prediction
├── gps_data
├── ground_truth
├── list_error
├── list_tf
├── raw_prediction
├── sensors_extrinsic_calibration
├── static_points
└── total_stations

ablation_test

This folder contains the data processed by the pre-processing pipeline to do some ablations tests. The name of the file 1-2-3-4-5-6.txt refers to the parameters used by the pre-processing pipeline:

1. Tau_r, threshold applied on the prism speed [m/s]
2. Tau_a, threshold applied on the RTS azimuth [deg/s]
3. Tau_e, threshold applied on the RTS elevation [deg/s]
4. Tau_s, threshold applied on the split and merge protocol [s]
5. Delta_t, value to cut the edge of the sub-intervals computed after the split and merge protocol [s]
6. Tau_l, threshold applied on the sub-intervals time [s]

Six values are saved into these files, corresponding to the number of points kept for each ablation tests of the pipeline on the experiment dataset:

1. Number of input points
2. Number of points out after the module 1 of the pre-processing pipeline
3. Number of points out after the module 1 and 2 of the pre-processing pipeline
4. Number of points out after the module 2 only of the pre-processing pipeline
5. Number of points out after the module 1, 2 and 3 of the pre-processing pipeline
6. Number of points out after the module 2 and 3 of the pre-processing pipeline

figures

This folder contains figures about the experiments.

filtered_prediction

This folder contains the data processed and interpolated by the pre-processing pipeline. The name of the file f-1-2-3-4-5-6-7(-8)_9.csv refers to the parameters used by the pre-processing pipeline:

1. Tau_r, threshold applied on the prism speed [m/s]
2. Tau_a, threshold applied on the RTS azimuth [deg/s]
3. Tau_e, threshold applied on the RTS elevation [deg/s]
4. Tau_s, threshold applied on the split and merge protocol [s]
5. Tau_l, threshold applied on the sub-intervals time [s]
6. Delta_t, value to cut the edge of the sub-intervals computed after the split and merge protocol [s]
7. Type of interpolation used. Can be different type of interpolation:

• L: linear interpolation
• SGP: Gaussian Process computed with the Stheno library
• MGPO: Multiple Gaussian Process Output (not studied yet)

8. If the Gaussian Process is used, this number is the restart number of the Gaussian Process training
9. ID of the prism

These files contain the timestamp and positions of the prism line by line. If the Gaussian Process was used, the prediction is also stored for each axis.

gps_data

This folder contains GNSS data if the experiment were done outside. Files .pos contain post-processed GNSS data (rapid static method). Files gpsfront.csv, gpsback.csv and gpsmiddle.csv are the global positions of the GPS in the UTM frame, respectively for the GPS at the front, back and in the middle of the robot. Files with distance contain the inter-GPS distance of the experiment in meters. Distance 12 refers to the inter-GPS between the GPS in the front and the back. Distance 13 refers to the inter-GPS between the GPS in the front and the middle. Distance 23 refers to the inter-GPS between the GPS in the back and the middle one.

ground_truth

This folder contains the ground-truth trajectories of either the lidar on the robot, or the GPSs on the robot during the experiment. The ground-truth is generated in the frame of the RTS taken as reference.TUM file format are used: every row has 8 entries containing timestamp (in seconds), position and orientation (as quaternion) with each value separated by a space timestamp x y z q_x q_y q_z q_w

list_error

This folder contains files which recorded the error metrics used to evaluate the data after the pre-processing pipeline and the extrinsic calibration. All errors are in millimetres.

list_tf

This folder contains files which recorded the rigid transformations computed for the evaluation of the data after the pre-processing pipeline and the extrinsic calibration. The values in these files correspond to the pose matrix value stored line by line. The first three lines of the file correspond to the rigid transformation of the RTS taken as reference defined as 1, then the rigid transformation between the RTS 1 and 2, and then the RTS 2 and 3. If more lines are in the files, It will be always other three rigid transformations in the same order as the first three ones.

raw_prediction

This folder contains the data processed and interpolated by the pre-processing pipeline, without the module 1 used. The name of the file nf-1-2-3-5(-6)_7.csv refers to the parameters used by the pre-processing pipeline:

1. Tau_s, threshold applied on the split and merge protocol [s]
2. Tau_l, threshold applied on the sub-intervals time [s]
3. Delta_t, value to cut the edge of the sub-intervals computed after the split and merge protocol [s]
4. Type of interpolation used. Can be different type of interpolation:

• L: linear interpolation
• SGP: Gaussian Process computed with the Stheno library
• MGPO: Multiple Gaussian Process Output (not studied yet)

5. If the Gaussian Process is used, this number is the restart number of the Gaussian Process training
6. ID of the prism

These files contain the timestamp and positions of the prism line by line. If the Gaussian Process was used, the prediction is also stored for each axis.

sensors_extrinsic_calibration

This folder contains two files, one with the raw data of the sensor position taken by a RTS. The other contains the results of the sensor extrinsic calibration.

The first file calibration_raw.csv has this template: 1.1 1.2 1.3 2.1 2.2 2.3 3.1 3.2 3.3 4.1 4.2 4.3 5.1 5.2 6

1. First three values refer to the azimuth measured by the RTS. Each correspond respectively to the degrees, minutes degrees and second degrees values.
2. Second three values refer to the azimuth uncertainty measured by the RTS. Each correspond respectively to the degrees, minutes degrees and second degrees values.
3. Third three values refer to the elevation measured by the RTS. Each correspond respectively to the degrees, minutes degrees and second degrees values.
4. Fourth three values refer to the elevation uncertainty measured by the RTS. Each correspond respectively to the degrees, minutes degrees and second degrees values.
5. These two values are the range to the target measured by the RTS and its uncertainty in meters.
6. Value for the end of the line

The second file calibration_results.csv contains values about the inter-prim or inter-GPS distances. The first three values are always the inter-prism distance values in meters, respectively the distance between the prisms 1-2, 1-3 and 2-3. The next three values are the inter-GPS distance values in meters, respectively the distance between the GPS 1-2, 1-3 and 2-3.

static_points

This folder contains static prism positions in the same generic way defined in the folder filtered_prediction or raw_prediction.

total_stations

This folder contains data about the prism trajectories given by the RTS.

The file GCP.txt contains the static GCP positions measured by each RTS to compute the Static GCPs calibration. The legend of the data are explained in the file.

The files raw_data_n.csv contain the prism positions according to their own RTS frame they were taken.

The files trimble.csvcontain the prism positions in the RTS frame of reference taken. The method used to compute the rigid transformations to express these data into this common frame is the Static GCPs calibration.

Doc

The folder doc/ contains some documentation about package and command to install the repository.

Figs

The folder figs/ is used to store figures generated by the code.

Msg

The folder msg/ contains the message class of the Ros package used to collect the data coming from the RTS. Currently, the msg class is defined for ROS 2 implementation. Two messages are defined, one for the time correction and the other one for the RTS data.

Notebook

The folder notebook/ is used to store the notebooks to process the data coming from the RTS. Each notebook in this folder does a special processing, or can be used to see the data gathered. Description of the notebook are explained here:

• Analysis_outliers_drop.ipynb: Notebook which studies the impact of the pre-processing pipeline modules. specifically on the points removed by it.
• Analysis_sensibility_and_ablation_tests.ipynb: Notebook which analysis the sensibility of the different module parameters.
• Check_length_time_dataset.ipynb: Notebook used to check the length of the dataset.
• Check_predicted_data.ipynb: Notebook used to check the predicted data coming from the pre-processing pipeline.
• Check_raw_data_deployment.ipynb: Notebook used to check the raw data coming from one deployment and see if everything is correct before processing them and to use them.
• Comparison_calibrations: The following notebook compares the error obtained with the points used for the calibration, for all the methods.
• Dynamic_GCP_calibration.ipynb: Notebook which computes and saves results of the Dynamic GCP calibration.
• Dynamic_inter_prism_calibration.ipynb: Notebook which generates the results of the dynamic inter-prism calibration.
• Dynamic_inter_prism_calibration_detailed.ipynb: Notebook to test the dynamic inter-prism calibration with one set of data.
• Full_pipeline.ipynb: Notebook used to compute the pre-processing pipeline on the raw data coming from one deployment.
• Groundtruth_generation_ptp_gps.ipynb: Notebook used to generate the ground truth according to the position of GPS in post-process.
• Groundtruth_generation_ptp_prism.ipynb: Notebook used to generate the ground truth according to the position of prisms in post-process.
• Raw_data_saved.ipynb: Notebook to check the RTS data before applying the interpolation.
• Read_time_corrections.ipynb: Notebook to read and check the time corrections given by each of the RTS.
• Sensors_extrinsic_calibration.ipynb: Notebook used to compute the inter-prism and inter-GPS distance after an experiment, also their positions into a lidar frame if the measures were done.
• Static_GCP_calibration.ipynb: Notebook which computes and saves results of the static GCP calibration.
• Two_points_calibration.ipynb: Notebook which computes and saves results of the two-points calibration.
• Two_points_calibration_with_optimization.ipynb: Notebook which uses a cost function to do the two-points calibration.

Scripts

The folder scipts/ is used to store the Python code to process the data coming from the RTS. Each scripts in this folder contains functions which does special processing. The tree of this folder is as following:

.
├── GPs.py
├── liblie.py
├── prediction_utils.py
├── resection_functions.py
├── theodolite_function.py
├── theodolite_plot_function.py
├── theodolite_utils.py
└── theodolite_values.py

1. GPs.py: Contain code about different methods to predict positions with Gaussian Process.
2. liblie.py: Contain function to use Lie Algebra.
3. prediction_utils.py: Contain function related to the interpolation methods.
4. resection_functions.py: Contain function to compute the different extrinsic calibration methods.
5. theodolite_function.py: Contain high level functions.
6. theodolite_plot_function.py: Contain function to plot some results.
7. theodolite_utils.py: Contain low level functions.
8. theodolite_values.py: Contain static values about the experiments done.