Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need for speed - Speed up loading map time #1267

Open
wants to merge 14 commits into
base: master
Choose a base branch
from

Conversation

hellovuong
Copy link
Contributor

@hellovuong hellovuong commented Apr 23, 2024

This PR saves the FLANN index, and its data points and serializes the associated data.
Saving FLANN data points will take a long time to save data points, only do it once. The loading will be faster by x10 by using saved files.

@matlabbe
Copy link
Member

Thanks for the PR. Would it make more sense to save the flann index inside the database instead of creating new file(s)? I won't have much time in next 2 weeks to review it or try a database version, but you may check to fix the CI errors in the mean time.

cheers,
Mathieu

@hellovuong
Copy link
Contributor Author

Hi,
I fixed CI, when you have time can you try or write me suggestions on how to save in the database file? We then can work together on that.
Cheers,
Vuong

@hellovuong
Copy link
Contributor Author

Hmm, I am not sure why the last step AppVeyor failed. All build on Ubuntu succeed

@matlabbe
Copy link
Member

For the windows build, maybe it is because the Eigen version used is not CMAKE_CXX_STANDARD 17 compatible. We may have to update windows dependencies to support it.

For database usage, the dictionary could be loaded from a database field in Admin table (with a new field flann_index_state BLOB). That could be done when Memory is initialized:

_dbDriver->load(_vwd, _vwd->isIncremental());

We may add a new parameter (e.g., Mem/DictionaryStateSaved) as a new flag to this function to tell DBDriver to look into Admin table for the serialized state of the flann index.

_dbDriver->load(_vwd, _vwd->isIncremental(), _dictionaryStateSaved);

If the field in database exists, inside load, there could be a call to a function like dictionary->restoreFlannState(blob) (which will deserialize the flann data and restore flann to right state) after adding the words.

When closing rtabmap, the flann index state could be saved before removing the visual words from the dictionary... before this line:

cleanUnusedWords();

like:

// Setting 0 or null would set null to field in database, otherwise, it will update it
_dbDriver->saveFlannState(_dictionaryStateSaved?_vwd:0);
cleanUnusedWords(); 

This would overwrite the last flann_index_state blob (with serialized data of the flann index) in the database if exists.

Note that by default I would keep the new parameter Mem/DictionaryStateSaved set to false to continue the old behavior, till it is tested more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants