Body 3D Keypoint¶

H36m Dataset¶

Motionbert + Motionbert on H36m¶

MotionBERT (2022)

 @misc{Zhu_Ma_Liu_Liu_Wu_Wang_2022,
 title={Learning Human Motion Representations: A Unified Perspective},
 author={Zhu, Wentao and Ma, Xiaoxuan and Liu, Zhaoyang and Liu, Libin and Wu, Wayne and Wang, Yizhou},
 year={2022},
 month={Oct},
 language={en-US}
 }

Human3.6M (TPAMI'2014)

@article{h36m_pami,
author = {Ionescu, Catalin and Papava, Dragos and Olaru, Vlad and Sminchisescu, Cristian},
title = {Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
publisher = {IEEE Computer Society},
volume = {36},
number = {7},
pages = {1325-1339},
month = {jul},
year = {2014}
}

Results on Human3.6M dataset with ground truth 2D detections

Arch	MPJPE	average MPJPE	P-MPJPE	ckpt
MotionBERT*	34.5	34.6	27.1	ckpt
MotionBERT-finetuned*	26.9	26.8	21.0	ckpt

Results on Human3.6M dataset converted from the official repo¹ with ground truth 2D detections

Arch	MPJPE	average MPJPE	P-MPJPE	ckpt	log
MotionBERT*	39.8	39.2	33.4	ckpt	/
MotionBERT-finetuned*	37.7	37.2	32.2	ckpt	/

¹ By default, we test models with Human 3.6m dataset processed by MMPose. The official repo’s dataset includes more data and applies a different pre-processing technique. To achieve the same result with the official repo, please download the test annotation file, train annotation file and factors under $MMPOSE/data/h36m/annotation_body3d/fps50 and test with the configs we provided.

Models with * are converted from the official repo. The config files of these models are only for validation. We don’t ensure these config files’ training accuracy and welcome you to contribute your reproduction results.

Image Pose Lift + Simplebaseline3d on H36m¶

SimpleBaseline3D (ICCV'2017)

@inproceedings{martinez_2017_3dbaseline,
  title={A simple yet effective baseline for 3d human pose estimation},
  author={Martinez, Julieta and Hossain, Rayat and Romero, Javier and Little, James J.},
  booktitle={ICCV},
  year={2017}
}

Human3.6M (TPAMI'2014)

@article{h36m_pami,
  author = {Ionescu, Catalin and Papava, Dragos and Olaru, Vlad and Sminchisescu,  Cristian},
  title = {Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  publisher = {IEEE Computer Society},
  volume = {36},
  number = {7},
  pages = {1325-1339},
  month = {jul},
  year = {2014}
}

Results on Human3.6M dataset with ground truth 2D detections

Arch	MPJPE	P-MPJPE	ckpt	log
SimpleBaseline3D¹	43.4	34.3	ckpt	log

¹ Differing from the original paper, we didn’t apply the max-norm constraint because we found this led to a better convergence and performance.

Video Pose Lift + Videopose3d on H36m¶

VideoPose3D (CVPR'2019)

@inproceedings{pavllo20193d,
title={3d human pose estimation in video with temporal convolutions and semi-supervised training},
author={Pavllo, Dario and Feichtenhofer, Christoph and Grangier, David and Auli, Michael},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={7753--7762},
year={2019}
}

Human3.6M (TPAMI'2014)

@article{h36m_pami,
author = {Ionescu, Catalin and Papava, Dragos and Olaru, Vlad and Sminchisescu, Cristian},
title = {Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
publisher = {IEEE Computer Society},
volume = {36},
number = {7},
pages = {1325-1339},
month = {jul},
year = {2014}
}

Testing results on Human3.6M dataset with ground truth 2D detections, supervised training

Arch	Receptive Field	MPJPE	P-MPJPE	ckpt	log
VideoPose3D-supervised-27frm	27	40.1	30.1	ckpt	log
VideoPose3D-supervised-81frm	81	39.1	29.3	ckpt	log
VideoPose3D-supervised-243frm	243	37.6	28.3	ckpt	log

Testing results on Human3.6M dataset with CPN 2D detections¹, supervised training

Arch	Receptive Field	MPJPE	P-MPJPE	ckpt	log
VideoPose3D-supervised-CPN-1frm	1	53.0	41.3	ckpt	log
VideoPose3D-supervised-CPN-243frm	243	47.9	38.0	ckpt	log

Testing results on Human3.6M dataset with ground truth 2D detections, semi-supervised training

Training Data	Arch	Receptive Field	MPJPE	P-MPJPE	N-MPJPE	ckpt	log
10% S1	VideoPose3D-semi-supervised-27frm	27	57.2	42.4	54.2	ckpt	log

Testing results on Human3.6M dataset with CPN 2D detections¹, semi-supervised training

Training Data	Arch	Receptive Field	MPJPE	P-MPJPE	N-MPJPE	ckpt	log
10% S1	VideoPose3D-semi-supervised-CPN-27frm	27	67.3	50.4	63.6	ckpt	log

¹ CPN 2D detections are provided by official repo. The reformatted version used in this repository can be downloaded from train_detection and test_detection.