CORTEXBENCH is a collection of 17 EAI tasks drawn from 7 existing benchmarks, designed to assess the efficacy of pre-trained visual representations (PVRs) in supporting a variety of EAI applications. Each task includes a downstream policy learning paradigm and evaluation protocol adhering to community standards. By using fixed tasks and learning methods, CORTEXBENCH evaluates the contribution of PVRs and measures progress towards developing an artificial visual cortex for embodied intelligence. This benchmark conducts the largest and most comprehensive study of PVRs to-date.
CORTEXBENCH includes tasks from 7 diverse benchmarks with different combinations of observations, actions, and goals as well as different standard policy learning paradigms:
| Benchmark Suite | Observation Space | Action Space | Goal Specification | Policy Learning |
|---|---|---|---|---|
| Adroit | RGB + proprio. | Continuous | - | IL |
| Metaworld | RGB + proprio. | Continuous | - | IL |
| DMControl | RGB + proprio. | Continuous | - | IL |
| Trifinger | RGB + proprio. | Continuous | Goal Image/Position | IL |
| ObjectNav | RGB + proprio. | Discrete | Object Category | IL |
| ImageNav | RGB | Discrete | Goal Image | RL |
| MobilePick | RGB + proprio. | Continuous | Goal Position | RL |
To install CortexBench, please follow the instructions in ../INSTALLATION.md.
Benchmarks from CortexBench use datasets described in DATASETS.md.