method: update submodule on all our methods.

* metric discussion attached.
* octomap more friendly dependencies
* beautymap bug fixed on over range.

Author: Kin-Zhang

README.md

@@ -7,7 +7,8 @@ A Dynamic Points Removal Benchmark in Point Cloud Maps
 [![video](https://img.shields.io/badge/video-YouTube-FF0000?logo=youtube&logoColor=white)](https://youtu.be/pCHsNKXDJQM?si=nhbAnPrbaZJEqbjx)
 [![poster](https://img.shields.io/badge/Poster-6495ed?style=flat&logo=Shotcut&logoColor=wihte)](https://hkustconnect-my.sharepoint.com/:b:/g/personal/qzhangcb_connect_ust_hk/EQvNHf9JNEtNpyPg1kkNLNABk0v1TgGyaM_OyCEVuID4RQ?e=TdWzAq)
 [![Built with Material for MkDocs](https://img.shields.io/badge/Material_for_MkDocs-526CFE?logo=MaterialForMkDocs&logoColor=white)](https://kth-rpl.github.io/DynamicMap_Benchmark/)
-    
+[![blog](https://img.shields.io/badge/Blog%7C%E7%9F%A5%E4%B9%8E%E4%B8%AD%E6%96%87-1772f6?style=flat&logo=Shotcut)](https://zhuanlan.zhihu.com/p/662096274) 
+
 Here is a preview of the readme in codes. Task detects dynamic points in maps and removes them, enhancing the maps:
 
 <center>
@@ -41,7 +42,7 @@ Here is a preview of the readme in codes. Task detects dynamic points in maps an
 
 Feel free to pull a request if you want to add more methods or datasets. Welcome! We will try our best to update methods and datasets in this benchmark. Please give us a star 🌟 and cite our work 📖 if you find this useful for your research. Thanks!
 
-- **2024/04/29** [BeautyMap](https://arxiv.org/abs/2405.07283) is accepted by RA-L'24. Updated benchmark: BeautyMap and DeFlow submodule instruction in the benchmark. Added the first data-driven method [DeFlow](https://github.com/KTH-RPL/DeFlow/tree/feature/dynamicmap) into our benchmark. Feel free to check.
+- **2024/04/29** [BeautyMap](https://arxiv.org/abs/2405.07283) is accepted by RA-L'24. Updated benchmark: BeautyMap and DeFlow submodule instruction in the benchmark. Added the first data-driven method [DeFlow](https://github.com/KTH-RPL/DeFlow/tree/feature/dynamicmap) into our benchmark. Feel free to check. And [a discussion](https://github.com/KTH-RPL/DynamicMap_Benchmark/discussions/15) for why metric different than previous works.
 - **2024/04/18** [DUFOMap](https://arxiv.org/abs/2403.01449) is accepted by RA-L'24. Updated benchmark: DUFOMap and dynablox submodule instruction in the benchmark. Two datasets w/o gt for demo are added in the download link. Feel free to check.
 - **2024/03/08** **Fix statements** on our ITSC'23 paper: KITTI sequences pose are also from SemanticKITTI which used SuMa. In the DUFOMap paper Section V-C, Table III, we present the dynamic removal result on different pose sources. Check discussion in [DUFOMap](https://arxiv.org/abs/2403.01449) paper if you are interested.
 - **2023/06/13** The [benchmark paper](https://arxiv.org/abs/2307.07260) Accepted by ITSC 2023 and release five methods (Octomap, Octomap w GF, ERASOR, Removert) and three datasets (01, 05, av2, semindoor) in [benchmark paper](https://arxiv.org/abs/2307.07260).
@@ -124,11 +125,11 @@ Please cite our works if you find these useful for your research:
 @article{jia2024beautymap,
   author={Jia, Mingkai and Zhang, Qingwen and Yang, Bowen and Wu, Jin and Liu, Ming and Jensfelt, Patric},
   journal={IEEE Robotics and Automation Letters}, 
-  title={BeautyMap: Binary-Encoded Adaptable Ground Matrix for Dynamic Points Removal in Global Maps}, 
+  title={{BeautyMap}: Binary-Encoded Adaptable Ground Matrix for Dynamic Points Removal in Global Maps}, 
   year={2024},
-  volume={},
-  number={},
-  pages={1-8},
+  volume={9},
+  number={7},
+  pages={6256-6263},
   doi={10.1109/LRA.2024.3402625}
 }
 @article{daniel2024dufomap,

methods/BeautyMap

@@ -6,6 +6,7 @@ Demo Image, results you can have after reading this README:
 <img src="../assets/imgs/octomap_gf_00.png" style="width: 70%;" alt="Octomap Image">
 
 All of them have same dependencies [PCL, Glog, yaml-cpp], we will show how to install and build:
+
 ## Install & Build
 
 Test computer and System:
@@ -14,6 +15,7 @@ Test computer and System:
 - System setting: Ubuntu 20.04 [ROS noetic-full installed in system]
 
 Dependencies listed following if you want to install them manually, or you can use docker to build and run if you don't like trash your env.
+
 ### Docker
 If you want to use docker, please check [Dockerfile](../Dockerfile) for more details.  This can also be a reference for you to install the dependencies.
 ```
@@ -25,36 +27,22 @@ docker run -it --rm --name dynamicmap -v /home/kin/data:/home/kin/data zhangkin/
 - If it's hard to build, you can always use `docker pull zhangkin/dynamic_map` to pull the image from docker hub.
 
 
-### PCL / OpenCV
-Normally, you will directly have PCL and OpenCV library if you installed ROS-full in your computer. 
-OpenCV is required by Removert only, PCL is required by Benchmark.
-
-Reminder for ubuntu 20.04 may occur this error:
-```bash
-fatal error: opencv2/cv.h: No such file or directory
-```
-ln from opencv4 to opencv2
-```bash
-sudo ln -s /usr/include/opencv4/opencv2 /usr/include/opencv2
-```
+### PCL
+Normally, you will directly have PCL and OpenCV library if you installed ROS-full in your computer. Otherwise
 
-### glog gflag [for print]
-or you can install through `sudo apt install`
 ```sh
-sh -c "$(wget -O- https://raw.githubusercontent.com/Kin-Zhang/Kin-Zhang/main/Dockerfiles/latest_glog_gflag.sh)"
+sudo apt update && sudo apt install -y libpcl-dev 
 ```
 
-### yaml-cpp [for config]
-Please set the FLAG, check this issue if you want to know more: https://github.com/jbeder/yaml-cpp/issues/682, [TOOD inside the CMakeLists.txt](https://github.com/jbeder/yaml-cpp/issues/566)
-
-If you install in Ubuntu 22.04, please check this commit: https://github.com/jbeder/yaml-cpp/commit/c86a9e424c5ee48e04e0412e9edf44f758e38fb9 which is the version could build in 22.04
+### glog gflag [for print]
 
+You can install them manually or use the script to install them.
 ```sh
-cd ${Tmp_folder}
-git clone https://github.com/jbeder/yaml-cpp.git && cd yaml-cpp
-env CFLAGS='-fPIC' CXXFLAGS='-fPIC' cmake -Bbuild
-cmake --build build --config Release
-sudo cmake --build build --config Release --target install
+# option 1:
+sudo apt install -y libgoogle-glog-dev libgflags-dev
+
+# option 2:
+sh -c "$(wget -O- https://raw.githubusercontent.com/Kin-Zhang/Kin-Zhang/main/Dockerfiles/latest_glog_gflag.sh)"
 ```
 
 ### Build
@@ -68,35 +56,38 @@ cmake -B build && cmake --build build
 
 Check each methods config file in their own folder `config/*.yaml` or `assets/*.yaml`
 ```bash
-./build/${methods_name}_run ${data_path} ${config.yaml} -1
+./build/${methods_name}_run ${data_path} ${config.yaml/toml} -1
 ```
 
 For example, if you want to run octomap with GF
 
 ```bash
-./build/octomap_run /home/kin/data/00 assets/config_fg.yaml -1
+./build/octomap_run /home/kin/data/00 assets/config_fg.toml -1
 ```
 
-Then you can get a time table with clean map result from Octomap w GF like top image shows. Or ERASOR on semindoor dataset:
+Then you can get a time table with clean map result from Octomap w GF like top image shows.
+
+ <!-- Or ERASOR on semindoor dataset:
 
 ```bash
 ./build/erasor_run /home/kin/data/semindoor config/seq_semindoor.yaml -1
 ```
 
-![ERASOR Image](../assets/imgs/erasor_semindoor.png)
-
-All Running commands, `-1` means all pcd files in the `pcd` folder, if you only want to run `10` frames change to `10`.
+![ERASOR Image](../assets/imgs/erasor_semindoor.png) -->
+All Running commands, `-1` means all pcd files in the `pcd` folder, otherwise the default is `10`/`1` frames.
 
 ```
-./build/removert_run ${data_path} ${config.yaml} -1
-./build/octomap_run ${data_path} ${config.yaml} -1
-./build/erasor_run ${data_path} ${config.yaml} -1
+./build/octomap_run ${data_path} ${config.toml} -1
+python BeautyMap/main.py --data_dir ${data_path} --run_file_num -1
+./build/dufomap_run ${data_path} ${config.toml}
 ```
 
-Then maybe you would like to have quantitative and qualitative result, check [scripts/py/eval](../scripts/py/eval).
+<!-- ./build/removert_run ${data_path} ${config.yaml} -1
+./build/erasor_run ${data_path} ${config.yaml} -1 -->
 
 ## Quick view on results
 
 Both with quantitative table and qualitative result
 
-TODO
+- Quantitative result print out the table: [scripts/py/eval/evaluate_all.py](../scripts/py/eval/evaluate_all.py)
+- Qualitative result automatically compare and merge: [scripts/py/eval/figure_plot.ipynb](../scripts/py/eval/figure_plot.ipynb)

methods/DeFlow

@@ -21,7 +21,7 @@
 
 # TODO: Change the parameters below to your own settings ====>>
 Result_Folder = "/home/kin/data/Dynamic_Papers_assets/BenchmarkPaper"
-algorithms = ["removert", "erasor", "octomap", "octomapg", "octomapfg"]
+algorithms = ["octomap", "octomapg", "octomapfg", "beautymap", "dufomap"]
 all_seqs = ["00", "05", "av2", "semindoor"]
 # TODO: Change the parameters below to your own settings <<===