Videogame DB
Dataset Description
1. Video Games context:
Video games have been popular for decades, and in recent years their popularity only continues to grow. Today, online and live streaming of people playing and sharing their video gaming experience has become a major part of the video games industry. Large technology companies are investing heavily in live video game streaming services, such as Twitch.tv, YouTube Gaming, and Facebook Gaming reflecting the strong expectation for continued growth in this type of entertainment. The popularity of such streaming platforms has accelerated the transition of video games from a hobby or individual pastime to a more social and interactive experience, and has proven there is a large market for people to watch other people playing video games.
In video games, there are usually visual or audio cues that indicate the occurrence of important events, such as attaining a goal or a kill. An algorithm that can recognize such cues can thus be very useful. For instance, the presence of such detected cues can indicate an interesting section of gameplay (a ‘highlight’). We thus might add a clip of that highlight to an automatic compilation, or suggest the user shares this highlight with their friends.
However the development of a fully supervised models to detect highlights is very costly in terms of the labeling effort. Hence, self-supervised learning methods are a good candidate in that situation in order to lower the need for labeled images. Moreover, until now, most work in this space has focused on academic benchmarks, rather than real-world datasets. For that, we created a new dataset for self-supervised learning applied to video games highlighting called Videogame DB.
2. Dataset Information:
Videogame DB consists of data gathered from Twitch.tv and YouTube Gaming. In order to constrain the domain of the video data, videos focused on gameplay of one specific game - Apex Legends - were selected. Apex Legends is a first person shooter game that can be played on PlayStation, Xbox, and PC. Since its release, Apex Legends has remained one of the highest watched gaming categories on Twitch.tv.
Dataset creation consisted of the several interrelated phases of content acquisition, data organisation and data processing. A number of relevant YouTube and Twitch URLs were collated by searching for Apex Legends videos. These were then downloaded using the tool youtube-dl.
For the most part, these downloaded videos show gameplay footage of people playing Apex Legends. Some other types of content are mixed in, most frequently footage of the player or `streamer' talking to their audience. Two to five random five minute non-overlapping clips were taken from each of these longer videos to reduce the dataset size while retaining diversity. In total, there are 6500 unique five minute video clips in the dataset, which are sampled from a total of 1132 unique longer videos.
In the final stage of the dataset creation process, RGB frames were extracted from each of the videos at two frames per second. Around 10000 of these frames were manually annotated by human labellers. The labels specified whether the frame was in-game (showed gameplay footage) or out-game (showed something else). For the 1400 frames labeled as in-game, additional annotations were collected for 6 binary characteristics, each referring to the presence of some visual cue that occasionally appears during gameplay footage. These include:
Visualization of some samples of the human labeled frames for different visual cues
- damage marker
- damage fx
- hit marker
- health bar depleting
- knocking-down banner
- killing banner.
For any questions on this dataset and/or to get access, please contact us by filling the form below