Skip to yearly menu bar Skip to main content

In-Person Poster presentation / poster accept

Proactive Multi-Camera Collaboration for 3D Human Pose Estimation

Hai Ci · Mickel Liu · Xuehai Pan · Fangwei Zhong · Yizhou Wang

MH1-2-3-4 #49

Keywords: [ Applications ] [ Multi-Agent Credit Assignment ] [ Active Vision ] [ Multi-Cameras Collaboration ] [ Human Pose Estimation ]


This paper presents a multi-agent reinforcement learning (MARL) scheme for proactive Multi-Camera Collaboration in 3D Human Pose Estimation in dynamic human crowds. Traditional fixed-viewpoint multi-camera solutions for human motion capture (MoCap) are limited in capture space and susceptible to dynamic occlusions. Active camera approaches proactively control camera poses to find optimal viewpoints for 3D reconstruction. However, current methods still face challenges with credit assignment and environment dynamics. To address these issues, our proposed method introduces a novel Collaborative Triangulation Contribution Reward (CTCR) that improves convergence and alleviates multi-agent credit assignment issues resulting from using 3D reconstruction accuracy as the shared reward. Additionally, we jointly train our model with multiple world dynamics learning tasks to better capture environment dynamics and encourage anticipatory behaviors for occlusion avoidance. We evaluate our proposed method in four photo-realistic UE4 environments to ensure validity and generalizability. Empirical results show that our method outperforms fixed and active baselines in various scenarios with different numbers of cameras and humans.

Chat is not available.