A Balanced Relation Prediction Framework for Scene Graph Generation

It has become a consensus that regular scene graph generation (SGG) is limited in actual applications due to the overfitting of head predicates. A series of debiasing methods, i.e. unbiased SGG, have been proposed to solve the problem. However, existing unbiased SGG methods have a tendency to fit the tail predicates, which is another type of bias. This paper aims to eliminate the one-way overfitting of head or tail predicates. In order to provide more balanced relationship prediction, we propose a new framework DCL (Dual-branch Cumulative Learning) which integrates regular relation prediction process and debiasing relation prediction process by employing cumulative learning mechanism. The learning process of DCL enhances the discrimination of tail predicates without reducing the discrimination performance of the model on head predicates. DCL is model-agnostic and compatible with existed different type of debiasing methods. Experiments on Visual Genome dataset show that, among all the model-agnostic methods, DCL achieves the best comprehensive performance while considering both R@K and mR@K.