Efficient Machine Reading Comprehension for Health CareApplications:Algorithm Development and Validation of a ContextExtraction Approach

Background: Extractive methods for machine reading comprehension (MRC) tasks have achieved comparable or better accuracythan human performance on benchmark data sets. However, such models are not as successful when adapted to complex domainssuch as health care. One of the main reasons is that the context that the MRC model needs to process when operating in a complexdomain can be much larger compared with an average open-domain context. This causes the MRC model to make less accurateand slower predictions. A potential solution to this problem is to reduce the input context of the MRC model by extracting onlythe necessary parts from the original context.Objective: This study aims to develop a method for extracting useful contexts from long articles as an additional componentto the question answering task, enabling the MRC model to work more efficiently and accurately.Methods: Existing approaches to context extraction in MRC are based on sentence selection strategies, in which the modelsare trained to find the sentences containing the answer. We found that using only the sentences containing the answer wasinsufficient for the MRC model to predict correctly. We conducted a series of empirical studies and observed a strong relationshipbetween the usefulness of the context and the confidence score output of the MRC model. Our investigation showed that a preciseinput context can boost the prediction correctness of the MRC and greatly reduce inference time. We proposed a method toestimate the utility of each sentence in a context in answering the question and then extract a new, shorter context according tothese estimations. We generated a data set to train 2 models for estimating sentence utility, based on which we selected moreprecise contexts that improved the MRC model's performance.Results: We demonstrated our approach on the Question Answering Data Set for COVID-19 and Biomedical Semantic Indexingand Question Answering data sets and showed that the approach benefits the downstream MRC model. First, the methodsubstantially reduced the inference time of the entire question answering system by 6 to 7 times. Second, our approach helpedthe MRC model predict the answer more correctly compared with using the original context (F1-score increased from 0.724 to0.744 for the Question Answering Data Set for COVID-19 and from 0.651 to 0.704 for the Biomedical Semantic Indexing andQuestion Answering). We also found a potential problem where extractive transformer MRC models predict poorly despite beinggiven a more precise context in some cases. Conclusions: The proposed context extraction method allows the MRC model to achieve improved prediction correctness anda significantly reduced MRC inference time. This approach works technically with any MRC model and has potential in tasksinvolving processing long texts