Objective Evaluation Metric for Motion Generative Models: Validating Frechet Motion Distance on Foot Skating and Over-smoothing Artifacts

Nowadays, Deep Learning-powered generative models are able to generate new synthetic samples nearly indistinguishable from natural data. The development of such systems necessarily involves the design of evaluation protocols to assess their performance. Quantitative objective metrics, such as Frechet distance, in addition to human-centered subjective surveys, have become a standard for evaluating generative algorithms. Although motion generation is a popular research field, only a few works addressed the problem of the design and validation of a robust objective evaluation metric for motion-generative models. These previous works proposed to degrade ground truth motion samples with synthetic noises (e.g., Gaussian, Salt& Pepper) and studied the behavior of the proposed metric. However, this degradation does not mimic common motion artifacts produced by generative models. In this work, we propose (1) to validate Frechet distance-based objective metrics on motion datasets degraded by two realistic motion artifacts, foot skating and over-smoothing, often found in motion synthesis results, and (2) a Frechet Motion Distance (FMD), using Transformer-based feature extractor, able to capture the motion artifacts and also robust towards the variation of motion length.