Research Focus: Success through the analysis of video data

This year, 20 different activities had to be identified in 200 videos, each 15 minutes long. These activities included, amongst others:

• person_closes_vehicle_door (person closes a vehicle door)
• person_reads_document (person reads a document)
• person_enters_scene_through_structure (person enters the scene through a structure)
• person_sits_down (person sits down)
• person_enters_vehicle (person gets into a vehicle)
• person_stands_up (Person stands up)
• person_exits_scene_through_structure (Person leaves the scene via a structure)
• person_talks_to_person (Person speaks to another person)
• person_exits_vehicle (Person gets out of a vehicle)
• person_texts_on_phone (Person is texting on their mobile phone)
• person_interacts_with_laptop (Person interacts with a laptop)
• person_transfers_object (Person hands over an object)
• person_opens_facility_door (Person opens a building door)
• vehicle_starts (Vehicle starts)
• person_opens_vehicle_door (Person opens a vehicle door)
• vehicle_stops (Vehicle stops)
• person_picks_up_object (Person picks up an object)
• vehicle_turns_left (Vehicle turns left)
• person_puts_down_object (Person puts down an object)
• vehicle_turns_right (Vehicle turns right)

With extensive support from the CSMRT (Computer Science and Media in Research and Transfer) and Faculty CB, without which this work would not have been possible, Rico Thomanek developed a modular framework that efficiently searches through large video datasets, identifies relevant human activities and determines their temporal position within the video. A particular strength of the system is its flexibility: new algorithms for computer-aided image analysis can be easily integrated into the processing workflow, enabling continuous improvements.

In the ActEV task, Rico Thomanek achieved impressive results and came second with an AOD Mean Pmiss@0.1RFA-Wert of 0.8330, just behind the winning team, which scored 0.8232. But what exactly does this figure mean? The AOD Mean Pmiss@0.1RFA-Wert is a measure of how well a system can detect activities and objects in videos. Put simply:

• AOD stands for Activity and Object Detection.
• Pmiss is the probability that the system will fail to detect a relevant activity.
• 0.1RFA means that, on average, the system generates 0.1 false alarms per minute—in other words, roughly one false alarm every ten minutes.

A lower Pmiss value is better, as it means the system misses fewer important activities. The evaluation therefore shows how reliably the system detects relevant events without generating too many false alarms. The aim is to identify as many relevant activities as possible whilst issuing as few false alerts as possible.

It is particularly noteworthy that Thomanek’s system achieved significantly better results than the top-ranked team in certain activities, such as ‘person_closes_vehicle_door’ or ‘person_texts_on_phone’. This underlines the high quality and efficiency of the framework developed.

Such frameworks are used, for example, in security surveillance to detect suspicious activities, as well as in traffic planning to analyse traffic flows and optimise public spaces.

An example of such a video, including the evaluation, is available at the following link: https://www.staff.hs-mittweida.de/~rthomane/trecvid/resultsExample/results.html

In recognition of these achievements, Professor Christian Roschke and Rico Thomanek have been invited by the organisers of the TRECVID conference to present their research findings in a scientific paper at this year’s conference in mid-November in Maryland, USA. This invitation provides an opportunity to present the developments and findings to an international specialist audience.

This success demonstrates Mittweida University of Applied Sciences’ expertise in the field of video and image data analysis and contributes to the international visibility of the university and its research projects.

Text and images: Rico Thomanek