2D Body Keypoint Datasets¶
It is recommended to symlink the dataset root to $MMPOSE/data.
If your folder structure is different, you may need to change the corresponding paths in config files.
MMPose supported datasets:
Images
Videos
COCO¶
COCO (ECCV'2014)
@inproceedings{lin2014microsoft,
title={Microsoft coco: Common objects in context},
author={Lin, Tsung-Yi and Maire, Michael and Belongie, Serge and Hays, James and Perona, Pietro and Ramanan, Deva and Doll{\'a}r, Piotr and Zitnick, C Lawrence},
booktitle={European conference on computer vision},
pages={740--755},
year={2014},
organization={Springer}
}
For COCO data, please download from COCO download, 2017 Train/Val is needed for COCO keypoints training and validation. HRNet-Human-Pose-Estimation provides person detection result of COCO val2017 to reproduce our multi-person pose estimation results. Please download from OneDrive or GoogleDrive. Optionally, to evaluate on COCO’2017 test-dev, please download the image-info. Download and extract them under $MMPOSE/data, and make them look like this:
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── coco
│-- annotations
│ │-- person_keypoints_train2017.json
│ |-- person_keypoints_val2017.json
│ |-- person_keypoints_test-dev-2017.json
|-- person_detection_results
| |-- COCO_val2017_detections_AP_H_56_person.json
| |-- COCO_test-dev2017_detections_AP_H_609_person.json
│-- train2017
│ │-- 000000000009.jpg
│ │-- 000000000025.jpg
│ │-- 000000000030.jpg
│ │-- ...
`-- val2017
│-- 000000000139.jpg
│-- 000000000285.jpg
│-- 000000000632.jpg
│-- ...
MPII¶
MPII (CVPR'2014)
@inproceedings{andriluka14cvpr,
author = {Mykhaylo Andriluka and Leonid Pishchulin and Peter Gehler and Schiele, Bernt},
title = {2D Human Pose Estimation: New Benchmark and State of the Art Analysis},
booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2014},
month = {June}
}
For MPII data, please download from MPII Human Pose Dataset. We have converted the original annotation files into json format, please download them from mpii_annotations. Extract them under {MMPose}/data, and make them look like this:
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── mpii
|── annotations
| |── mpii_gt_val.mat
| |── mpii_test.json
| |── mpii_train.json
| |── mpii_trainval.json
| `── mpii_val.json
`── images
|── 000001163.jpg
|── 000003072.jpg
During training and inference, the prediction result will be saved as ‘.mat’ format by default. We also provide a tool to convert this ‘.mat’ to more readable ‘.json’ format.
python tools/dataset/mat2json ${PRED_MAT_FILE} ${GT_JSON_FILE} ${OUTPUT_PRED_JSON_FILE}
For example,
python tools/dataset/mat2json work_dirs/res50_mpii_256x256/pred.mat data/mpii/annotations/mpii_val.json pred.json
MPII-TRB¶
MPII-TRB (ICCV'2019)
@inproceedings{duan2019trb,
title={TRB: A Novel Triplet Representation for Understanding 2D Human Body},
author={Duan, Haodong and Lin, Kwan-Yee and Jin, Sheng and Liu, Wentao and Qian, Chen and Ouyang, Wanli},
booktitle={Proceedings of the IEEE International Conference on Computer Vision},
pages={9479--9488},
year={2019}
}
For MPII-TRB data, please download from MPII Human Pose Dataset. Please download the annotation files from mpii_trb_annotations. Extract them under {MMPose}/data, and make them look like this:
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── mpii
|── annotations
| |── mpii_trb_train.json
| |── mpii_trb_val.json
`── images
|── 000001163.jpg
|── 000003072.jpg
AIC¶
AI Challenger (ArXiv'2017)
@article{wu2017ai,
title={Ai challenger: A large-scale dataset for going deeper in image understanding},
author={Wu, Jiahong and Zheng, He and Zhao, Bo and Li, Yixin and Yan, Baoming and Liang, Rui and Wang, Wenjia and Zhou, Shipei and Lin, Guosen and Fu, Yanwei and others},
journal={arXiv preprint arXiv:1711.06475},
year={2017}
}
For AIC data, please download from AI Challenger 2017, 2017 Train/Val is needed for keypoints training and validation. Please download the annotation files from aic_annotations. Download and extract them under $MMPOSE/data, and make them look like this:
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── aic
│-- annotations
│ │-- aic_train.json
│ |-- aic_val.json
│-- ai_challenger_keypoint_train_20170902
│ │-- keypoint_train_images_20170902
│ │ │-- 0000252aea98840a550dac9a78c476ecb9f47ffa.jpg
│ │ │-- 000050f770985ac9653198495ef9b5c82435d49c.jpg
│ │ │-- ...
`-- ai_challenger_keypoint_validation_20170911
│-- keypoint_validation_images_20170911
│-- 0002605c53fb92109a3f2de4fc3ce06425c3b61f.jpg
│-- 0003b55a2c991223e6d8b4b820045bd49507bf6d.jpg
│-- ...
CrowdPose¶
CrowdPose (CVPR'2019)
@article{li2018crowdpose,
title={CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark},
author={Li, Jiefeng and Wang, Can and Zhu, Hao and Mao, Yihuan and Fang, Hao-Shu and Lu, Cewu},
journal={arXiv preprint arXiv:1812.00324},
year={2018}
}
For CrowdPose data, please download from CrowdPose. Please download the annotation files and human detection results from crowdpose_annotations. For top-down approaches, we follow CrowdPose to use the pre-trained weights of YOLOv3 to generate the detected human bounding boxes. For model training, we follow HigherHRNet to train models on CrowdPose train/val dataset, and evaluate models on CrowdPose test dataset. Download and extract them under $MMPOSE/data, and make them look like this:
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── crowdpose
│-- annotations
│ │-- mmpose_crowdpose_train.json
│ │-- mmpose_crowdpose_val.json
│ │-- mmpose_crowdpose_trainval.json
│ │-- mmpose_crowdpose_test.json
│ │-- det_for_crowd_test_0.1_0.5.json
│-- images
│-- 100000.jpg
│-- 100001.jpg
│-- 100002.jpg
│-- ...
OCHuman¶
OCHuman (CVPR'2019)
@inproceedings{zhang2019pose2seg,
title={Pose2seg: Detection free human instance segmentation},
author={Zhang, Song-Hai and Li, Ruilong and Dong, Xin and Rosin, Paul and Cai, Zixi and Han, Xi and Yang, Dingcheng and Huang, Haozhi and Hu, Shi-Min},
booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
pages={889--898},
year={2019}
}
For OCHuman data, please download the images and annotations from OCHuman, Move them under $MMPOSE/data, and make them look like this:
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── ochuman
│-- annotations
│ │-- ochuman_coco_format_val_range_0.00_1.00.json
│ |-- ochuman_coco_format_test_range_0.00_1.00.json
|-- images
│-- 000001.jpg
│-- 000002.jpg
│-- 000003.jpg
│-- ...
MHP¶
MHP (ACM MM'2018)
@inproceedings{zhao2018understanding,
title={Understanding humans in crowded scenes: Deep nested adversarial learning and a new benchmark for multi-human parsing},
author={Zhao, Jian and Li, Jianshu and Cheng, Yu and Sim, Terence and Yan, Shuicheng and Feng, Jiashi},
booktitle={Proceedings of the 26th ACM international conference on Multimedia},
pages={792--800},
year={2018}
}
For MHP data, please download from MHP. Please download the annotation files from mhp_annotations. Please download and extract them under $MMPOSE/data, and make them look like this:
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── mhp
│-- annotations
│ │-- mhp_train.json
│ │-- mhp_val.json
│
`-- train
│ │-- images
│ │ │-- 1004.jpg
│ │ │-- 10050.jpg
│ │ │-- ...
│
`-- val
│ │-- images
│ │ │-- 10059.jpg
│ │ │-- 10068.jpg
│ │ │-- ...
│
`-- test
│ │-- images
│ │ │-- 1005.jpg
│ │ │-- 10052.jpg
│ │ │-- ...~~~~
Human-Art dataset¶
Human-Art (CVPR'2023)
@inproceedings{ju2023humanart,
title={Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenes},
author={Ju, Xuan and Zeng, Ailing and Jianan, Wang and Qiang, Xu and Lei, Zhang},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),
year={2023}}
For Human-Art data, please download the images and annotation files from its website. You need to fill in the data form to get access to the data. Move them under $MMPOSE/data, and make them look like this:
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
|── data
│── HumanArt
│-- images
│ │-- 2D_virtual_human
│ │ |-- cartoon
│ │ | |-- 000000000000.jpg
│ │ | |-- ...
│ │ |-- digital_art
│ │ |-- ...
│ |-- 3D_virtual_human
│ |-- real_human
|-- annotations
│ │-- validation_humanart.json
│ │-- training_humanart_coco.json
|-- person_detection_results
│ │-- HumanArt_validation_detections_AP_H_56_person.json
You can choose whether to download other annotation files in Human-Art. If you want to use additional annotation files (e.g. validation set of cartoon), you need to edit the corresponding code in config file.
ExLPose dataset¶
ExLPose (2023)
@inproceedings{ExLPose_2023_CVPR,
title={Human Pose Estimation in Extremely Low-Light Conditions},
author={Sohyun Lee, Jaesung Rim, Boseung Jeong, Geonu Kim, ByungJu Woo, Haechan Lee, Sunghyun Cho, Suha Kwak},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2023}
}
For ExLPose data, please download from ExLPose, Move them under $MMPOSE/data, and make them look like this:
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── ExLPose
│-- annotations
| |-- ExLPose
│ |-- ExLPose_test_LL-A.json
│ |-- ExLPose_test_LL-E.json
│ |-- ExLPose_test_LL-H.json
│ |-- ExLPose_test_LL-N.json
│ |-- ExLPose_test_WL.json
│ |-- ExLPose_train_LL.json
│ |-- ExLPose_train_WL.json
|-- bright
| |-- imgs_0119_3_vid000002_exp100_bright_000052__gain_0.00_exposure_1000.png
| |-- imgs_0119_3_vid000002_exp200_bright_000052__gain_6.02_exposure_500.png
| |-- ...
|-- dark
|-- imgs_0119_3_vid000002_exp100_dark_000052__gain_0.00_exposure_1000.png
|-- imgs_0119_3_vid000005_exp200_dark_000002__gain_1.60_exposure_500.png
|-- ...
PoseTrack18¶
PoseTrack18 (CVPR'2018)
@inproceedings{andriluka2018posetrack,
title={Posetrack: A benchmark for human pose estimation and tracking},
author={Andriluka, Mykhaylo and Iqbal, Umar and Insafutdinov, Eldar and Pishchulin, Leonid and Milan, Anton and Gall, Juergen and Schiele, Bernt},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
pages={5167--5176},
year={2018}
}
For PoseTrack18 data, please download from PoseTrack18. Please download the annotation files from posetrack18_annotations. We have merged the video-wise separated official annotation files into two json files (posetrack18_train & posetrack18_val.json). We also generate the mask files to speed up training. For top-down approaches, we use MMDetection pre-trained Cascade R-CNN (X-101-64x4d-FPN) to generate the detected human bounding boxes. Please download and extract them under $MMPOSE/data, and make them look like this:
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── posetrack18
│-- annotations
│ │-- posetrack18_train.json
│ │-- posetrack18_val.json
│ │-- posetrack18_val_human_detections.json
│ │-- train
│ │ │-- 000001_bonn_train.json
│ │ │-- 000002_bonn_train.json
│ │ │-- ...
│ │-- val
│ │ │-- 000342_mpii_test.json
│ │ │-- 000522_mpii_test.json
│ │ │-- ...
│ `-- test
│ │-- 000001_mpiinew_test.json
│ │-- 000002_mpiinew_test.json
│ │-- ...
│
`-- images
│ │-- train
│ │ │-- 000001_bonn_train
│ │ │ │-- 000000.jpg
│ │ │ │-- 000001.jpg
│ │ │ │-- ...
│ │ │-- ...
│ │-- val
│ │ │-- 000342_mpii_test
│ │ │ │-- 000000.jpg
│ │ │ │-- 000001.jpg
│ │ │ │-- ...
│ │ │-- ...
│ `-- test
│ │-- 000001_mpiinew_test
│ │ │-- 000000.jpg
│ │ │-- 000001.jpg
│ │ │-- ...
│ │-- ...
`-- mask
│-- train
│ │-- 000002_bonn_train
│ │ │-- 000000.jpg
│ │ │-- 000001.jpg
│ │ │-- ...
│ │-- ...
`-- val
│-- 000522_mpii_test
│ │-- 000000.jpg
│ │-- 000001.jpg
│ │-- ...
│-- ...
The official evaluation tool for PoseTrack should be installed from GitHub.
pip install git+https://github.com/svenkreiss/poseval.git
sub-JHMDB dataset¶
RSN (ECCV'2020)
@misc{cai2020learning,
title={Learning Delicate Local Representations for Multi-Person Pose Estimation},
author={Yuanhao Cai and Zhicheng Wang and Zhengxiong Luo and Binyi Yin and Angang Du and Haoqian Wang and Xinyu Zhou and Erjin Zhou and Xiangyu Zhang and Jian Sun},
year={2020},
eprint={2003.04030},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
For sub-JHMDB data, please download the images from JHMDB, Please download the annotation files from jhmdb_annotations. Move them under $MMPOSE/data, and make them look like this:
mmpose
├── mmpose
├── docs
├── tests
├── tools
├── configs
`── data
│── jhmdb
│-- annotations
│ │-- Sub1_train.json
│ |-- Sub1_test.json
│ │-- Sub2_train.json
│ |-- Sub2_test.json
│ │-- Sub3_train.json
│ |-- Sub3_test.json
|-- Rename_Images
│-- brush_hair
│ │--April_09_brush_hair_u_nm_np1_ba_goo_0
| │ │--00001.png
| │ │--00002.png
│-- catch
│-- ...