habitat_vc

Habitat: ImageNav and ObjectNav

ImageNav

1. To run experiments in Habitat, first we need to get access to the necessary scene dataset. We are using Gibson scene datasets for our ImageNav experiment. You can find instructions for downloading the dataset here.

2. Next we need the episode dataset for ImageNav. You can get the training and validation dataset from here and place it in the ./data folder under the path : data/datasets/pointnav/gibson/v1/.

3. Now we are ready to start training the agent. Checkout the run_habitat_vc.py script, which allows running an experiment on the cluster. The script can be used as follows:

   python run_habitat_vc.py --config-name=config_imagenav -m 

   This will start a run on the slurm with the default folder name imagenav_run.

4. If you want to start a local run, add hydra/launcher=slurm at the end of the command listed in the previous point.

5. Once you have trained a model, it is time for evaluation. We evaluate every 5th saved checkpoint. To run an evaluation, do the following:

   python run_habitat_vc.py --config-name=eval_config_imagenav hydra/launcher=slurm_eval NUM_ENVIRONMENTS=14 -m

ObjectNav

1. For our ObjectNav IL experiments we will be using the HM3DSem v0.1 scene dataset and the corresponding HM3DSem v0.1 ObjectNav episode dataset. Currently the data is available by following these instructions. Download the Scene dataset HM3D

2. The Task training dataset linked here which contents you need to place in data/datasets/objectnav/hm3d/objectnav_hm3d_77k/

3. The Task validation dataset linked here which contents you need to place in data/datasets/objectnav/hm3d/v1/

4. To start a training run with the vc1_vitl model use the following command:

      python run_habitat_vc.py --config-name=config_objectnav_il_frozen WANDB.name=Objectnav_first_experiment model=vc1_vitl -m

5. Once you have trained a model, it is time for evaluation. We evaluate every 5th saved checkpoint. To run an evaluation, do the following:

   python run_habitat_vc.py --config-name=eval_config_objectnav_il_frozen WANDB.name=Objectnav_first_experiment model=vc1_vitl  hydra/launcher=slurm_eval NUM_ENVIRONMENTS=30 -m