Hybrid Multi-Modal Deep Learning using Collaborative Concat Layer in Health Bigdata

A health model based on data has various missing values depending on the user situation, and the accuracy of a health model requiring variables that the user cannot collect appears low. A deep learning health model is fitted by learning weights to increase accuracy. In the process of applying a deep-learning-based health model to the user situation, accuracy may be degraded if learning is omitted. In this paper, we propose hybrid multimodal deep learning using a collaborative concat layer in health big data. The proposed method uses a machine learning technique to alleviate the issue caused by the change in the data observation range according to a change in the user situation, and occurring in multimodal health deep learning. It is a layer composed of the connection, input, and output of the model of the collaborative node (CN). A CN is a node that predicts absent variables through filtering using the similarity of input values. With CN, a collaborative concat layer (CCL) that handles missing values from the input of the health model can be configured, and the issue related to missing values occurring in the health model can be resolved. With the proposed CCL, it is possible to reuse existing models or construct new models through the concatenation of several single-modal deep learning models. By evaluating the effect on the input and output of the model according to the structural position of the CCL, various networks can be configured, and the performance of the single-modal model can be maintained. In particular, the accuracy of a deep learning model is more stable when the CCL is used, suggesting the experiment progress based on the assumption that a specific variable is absent depending on the user situation.