Pose estimation in 4D video

Title: Pose estimation in 4D video (Undergrad final project)

Supervisor: Dr John Collomosse, University of Surrey, UK

Languages/Technologies: C/C++, matlab, opengl, computer vision algorithms, machine learning, kinect, microsoft visual studio

Role: Wrote algorithms for sampling, classification, analysis of a 4d model


Image


The main aim of the project was to explore how computer vision techniques can be applied to a 4D video in order to determine the pose of the model. The 3d model in this project was a dancer, continuously moving and changing poses (as seen below).

Image

The dancer was captured using motion capture program built in house at the university using a number of cameras and this was then made into a 3d model. By using different algorithms to sample the model, a unique id was assigned to each frame which were then classified into different categories. The system was then trained using manually marked up data (i.e. the joint positions in my body)which I captured using kinect. This was used to identify the right data and false positives. The algorithm was then designed to identify and retrieve a similar pose based on the input.

Image

The same algorithm was also applied to test different videos with occlusion (i.e. the model wearing a scarf). Throughout the process a number of parameters were changed and analysed to improve pose retrieval. The analysis was used to get feedback of the performance and to make changes to the algorithms.

Image