Natural language based analysis of SQuAD: An analytical approach for BERT

•The NLP-based Question Answering System (NLP-QAS) is proposed for answer detection.•Remove and compare, Search with NER and POS (RNP) methods has been developed.•It’s the first one that extends the BERT model’s capability with RNP methods.•The accuracy of BERT has increased by approximately 1.1% to 2.4% with RNP methods.•It is proved that BERT models don’t use NLP-based techniques sufficiently. In recent years, deep learning models have been used in the implementation of question answering systems. In this study, the performance of the question answering system was evaluated from the perspective of natural language processing using SQuAD, which was developed to measure the performance of deep learning language models. In line with the evaluations, in order to increase the performance, 3 natural language based methods, namely RNP, that can be used with pre-trained BERT language models have been proposed and they have increased the performance of the question answering system in which the pre-trained BERT models are used by 1.1% to 2.4%. As a result of the application of RNP methods with sentence selection, an increase in accuracy between 6.6% and 8.76% was achieved in answer detection. Since these methods don’t require any training process, it has been shown that they can be used in question answering systems to increase the performance of any deep learning model.