With the spread of high-performance sensors and social network services (SNS) and the remarkable advances in machine learning technologies, fake media such as fake videos, spoofed voices, and fake reviews that are generated using high-quality learning data and are very close to the real thing are causing serious social problems. We launched a research project, the Media Clone (MC) project, to protect receivers of replicas of real media called media clones (MCs) skillfully fabricated by means of media processing technologies. Our aim is to achieve a communication system that can defend against MC attacks and help ensure safe and reliable communication. This paper describes the results of research in two of the five themes in the MC project: 1) verification of the capability of generating various types of media clones such as audio, visual, and text derived from fake information and 2) realization of a protection shield for media clones' attacks by recognizing them.

Spoofing attacks are one of the biggest concerns for most biometric recognition systems. This will be also the case with silhouette-based gait recognition in the near future. So far, gait recognition has been fortunately out of the scope of spoofing attacks. However, it is becoming a real threat with the rapid growth and spread of deep neural network-based multimedia generation techniques, which will allow attackers to generate a fake video of gait silhouettes resembling a target person's walking motion. We refer to such computer-generated fake silhouettes as gait silhouette clones (GSCs). To deal with the future threat caused by GSCs, in this paper, we propose a supervised method for discriminating GSCs from genuine gait silhouettes (GGSs) that are observed from actual walking people. For training a good discriminator, it is important to collect training datasets of both GGSs and GSCs which do not differ from each other in any aspect other than genuineness. To this end, we propose to generate a training set of GSCs from GGSs by transforming them using multiple autoencoders. The generated GSCs are used together with their original GGSs for training the discriminator. In our experiments, the proposed method achieved the recognition accuracy of up to 94% for several test datasets, which demonstrates the effectiveness and the generality of the proposed method.

Fake media has been spreading due to remarkable advances in media processing and machine leaning technologies, causing serious problems in society. We are conducting a research project called Media Clone aimed at developing methods for protecting people from fake but skillfully fabricated replicas of real media called media clones. Such media can be created from fake information about a specific person. Our goal is to develop a trusted communication system that can defend against attacks of media clones. This paper describes some research results of the Media Clone project, in particular, various methods for protecting personal information against generating fake information. We focus on 1) fake information generation in the physical world, 2) anonymization and abstraction in the cyber world, and 3) modeling of media clone attacks.

Video summarization is defined as creating a video summary which includes only important scenes in the original video streams. In order to realize automatic video summarization, the significance of each scene needs to be determined. When targeted especially on broadcast sports videos, a play scene, which corresponds to a play, can be considered as a scene unit. The significance of every play scene can generally be determined based on the importance of the play in the game. Furthermore, the following two issues should be considered: 1) what is important depends on each user's preferences, and 2) the summaries should be tailored for media devices that each user has. Considering the above issues, this paper proposes a unified framework for user and device adaptation in summarizing broadcast sports videos. The proposed framework summarizes sports videos by selecting play scenes based on not only the importance of each play itself but also the users' preferences by using the metadata, which describes the semantic content of videos with keywords, and user profiles, which describe users' preference degrees for the keywords. The selected scenes are then presented in a proper way using various types of media such as video, image, or text according to device profiles which describe the device type. We experimentally verified the effectiveness of user adaptation by examining how the generated summaries are changed by different preference degrees and by comparing our results with/without using user profiles. The validity of device adaptation is also evaluated by conducting questionnaires using PCs and mobile phones as the media devices.