Human Action Recognition using Random Forest

TNT members involved in this project:
Prof. Dr.-Ing. Bodo Rosenhahn
Show all

Human action recognition is a complex area of computer vision since static object characteristics, motion and time information have to be taken into account. Furthermore, actions are divided into human actions (walking, running, jogging), human-human interactions (handshaking, kissing, punching), human-object interactions (calling, writing, driving car) and group activities (football, soccer, group stealing). Due to environment variations such as moving backgrounds, different view points or occlusions the detection and classification of actions is even more difficult. Additionally, each actor has its own style of performing an action, leading to many variations in the subject's movement and a large intra-class variation.

  • Environment variations such as moving background, occlusions, different view points.
  • Variations of actors movement, each person has own style of executing an action.
  • Various types of activity: gestures, actions, interactions, group activities.
  • Insufficient amount of training videos.

A Random Forest consists of CART-like decision trees that are independently constructed on a bootstrap sample. Compared to other ensemble learning algorithms, i.e. boosting that build a flat tree structure of decision stumps, a Random Forest uses an ensemble of decision trees and is multi-class capable.
A tree is grown using the following algorithm:

  • Choose n samples with m variables from N training samples at random.
  • The remaining samples are used to calculate the out-of-bag error (OOB-error).
  • At each node specify m_try << M variables at random based on best split.
  • Completely grow the tree without pruning.
A completed classifier consists of several trees in which the class probabilities, estimated by majority voting, are used to calculate the sample's label.

Hollywood dataset:

UCF Sports dataset:

KTH dataset:

TNT Action Recognition using Motion Binary Patterns:

Supervised theses:

  • Evaluation of different LBP Computation Strategies for Action Recognition
  • Kinect-based Action Recognition
  • Evaluation of HOG3D and Random Forest for Action Recognition
  • Acceleration-based Action Recognition
  • An Augmented Reality Game for Mobile Devices
  • Slow-Feature Analysis for Action Recognition
  • A Spacetime Feature for Action Recognition
  • Action Bank for Human Action Recognition

Open theses:

  • XSens accelerations sensors
  • Eye tracking on mobile devices (iOS and Android)
  • Kinect Action Recognition
  • ToF Action Recognition (with Time of Flight camera)
  • Modify existing framework for Action Recognition with Volume Local Binary Patterns
  • Implement interactive games on mobile devices (iOS and Android) using object detection frameworks

Projects: Action Recognition in Thermal Images:

Projects: Action Recognition with Inertial Sensors:

Acceleration-based with inertial sensor by

Projects: Recognize head movements and implement interactive games:

ERC Starting Grants

This project has been partially funded by the ERC within the starting grant Dynamic MinVIP.

Show all publications
  • Christoph Reinders, Florian Baumann, Björn Scheuermann, Arne Ehlers, Nicole Mühlpforte, Alfred O. Effenberg, Bodo Rosenhahn
    On-The-Fly Handwriting Recognition using a High-Level Representation
    The 16th International Conference on Computer Analysis of Images and Patterns (CAIP), Lecture Notes in Computer Science (LNCS), Valetta, Malta, September 2015
  • Florian Baumann, Jinghui Chen, Karsten Vogt, Bodo Rosenhahn
    Improved Threshold Selection by using Calibrated Probabilities for Random Forest Classifiers
    12th Conference on Computer and Robot Vision (CRV), Halifax, Nova Scotia, Canada, June 2015
  • Florian Baumann, Karsten Vogt, Arne Ehlers, Bodo Rosenhahn
    Probabilistic Nodes for Modelling Classification Uncertainty for Random Forest
    14th IAPR International Conference on Machine Vision Applications (MVA), Tokio, Japan, May 2015