SLAM Competition

Benchmark

Competition Results: V-SLAM

Click here to download the detailed result.
Configuration of the benchmark PC:
- CPU : i7-9700K 3.60GHz
- Memory : 32G
- GPU : Nvidia RTX 2070-8G
- Hard Disk : Samsung SSD 850EVO 500G

Rank	Participants	Affiliation	System Name	APE	RPE	ARE	RRE	Badness	Initialization Quality	Robustness	Relocalization Time	Benchmark Score	Average FrameRate	Speed Penalty	Final Score	System Description
1	Zike Yan, Pijian Sun, Xin Wang, Shunkai Li, Sheng Zhang, Hongbin Zha	Key Laboratory of Machine Perception(MOE), School of EECS, Peking University	LF-SLAM	0.7155	0.4037	0.8115	0.2078	0.4554	0.7166	0.9972	0.9965	0.8331	30.0687	1.0000	83.31	Robust line tracking for monocular visual system (Doc would be uploaded after the acceptance of the paper)
2	Darius Rueckert	University of Erlangen-Nuremberg	Snake-SLAM	0.7543	0.5700	0.8229	0.2709	0.5183	0.7818	0.6685*	0.9974	0.8273*	106.1527	1.0000	82.73*	Doc
3	Neo Yuan Rong Dexter, Toh Yu Heng	Pensees	AR-ORB-SLAM2	0.5511	0.3850	0.5280	0.2028	0.6020	0.7080	0.9947	0.9890	0.7778	31.0803	1.0000	77.78
4	Xinyu Wei, Zengming Tang, Huiyan Wu, Jun Huang	Shanghai Advanced Research Institute, Chinese Academy of Sciences	PL-SLAM	0.6036	0.4754	0.7354	0.2376	0.4551	0.8062	0.9985	0.9916	0.8245	27.5107	0.9170	75.61	Doc Slides
5	Ao Li, Yue Ni	University of Science and Technology of China	Dy-SLAM	0.7588	0.4390	0.7953	0.2002	0.5776	0.6018	0.9981	0.8578	0.8182	3.8851	0.1295	10.60	Doc

Note: the output trajectory files by Snake-SLAM (the final submitted executable program during the competition) denote the invalid poses by an identity one, i.e. (0 0 0 -0 -0 -0 1). Unfortunately, it does not fit the invalid pose format that we define, so our evaluation tool still regarded them as valid poses. As a result, the computed robustness score was affected. If we remove these invalid poses, the new robustness score of Snake-SLAM becomes 0.9348, and the final score is 87.17. Since this format issue was discovered after the competition, and did not appear in the results of other teams, the competition ranking can no longer be changed.

Competition Results: VI-SLAM

Click here to download the detailed result.
Configuration of the benchmark PC:
- CPU : i7-9700K 3.60GHz
- Memory : 32G
- GPU : Nvidia RTX 2070-8G
- Hard Disk : Samsung SSD 850EVO 500G

Rank	Participants	Affiliation	System Name	APE	RPE	ARE	RRE	Badness	Initialization Quality	Robustness	Relocalization Time	Benchmark Score	Average FrameRate	Speed Penalty	Final Score (%)	System Description
1	Shaozu Cao, Jie Pan, Jieqi Shi, Shaojie Shen	Hong Kong University of Science and Technology	VINS-Mono	0.6341	0.4225	0.4945	0.2429	0.8175	0.5678	0.8037	0.8572	0.7513	30.1062	1.0000	75.13	Doc Slides
2	Xinyu Wei, Zengming Tang, Huiyan Wu, Jun Huang	Shanghai Advanced Research Institute, Chinese Academy of Sciences	PLVI-SLAM	0.2767	0.0994	0.3383	0.1675	0.0097	0.2813	0.8183	0.6654	0.4205	18.3380	0.6113	25.71	Doc
3	Jianhua Zhang, Shengyong Chen, Mengping Gui, Jialing Liu, Luzhen Ma, Kaiqi Chen	Zhejiang University of Technology	MMF-SLAM	0.1203	0.0834	0.1760	0.1407	0.0010	0.3214	0.0102	0.0000	0.1235	29.9620	0.9987	12.33	Doc Slides