Intelligent image-based colourimetric tests using machine learning framework for lateral flow assays

•An investigation into an automatic computational framework for lateral flow assays.•Critical examination of the data structure, features and algorithms.•Pseudo control colours were proposed as a new feature within the optimal feature set.•Proposed scheme offers high accuracy and fulfils ASSURED criteria. This paper aims to deliberately examine the scope of an intelligent colourimetric test that fulfils ASSURED criteria (Affordable, Sensitive, Specific, User-friendly, Rapid and robust, Equipment-free, and Deliverable) and demonstrate the claim as well. This paper presents an investigation into an intelligent image-based system to perform automatic paper-based colourimetric tests in real-time to provide a proof-of-concept for a dry-chemical based or microfluidic, stable and semi-quantitative assay using a larger dataset with diverse conditions. The universal pH indicator papers were utilised as a case study. Unlike the works done in the literature, this work performs multiclass colourimetric tests using histogram-based image processing and machine learning algorithm without any user intervention. The proposed image processing framework is based on colour channel separation, global thresholding, morphological operation and object detection. We have also deployed aserver-based convolutional neural network framework for image classification using inductive transfer learning on a mobile platform. The results obtained by both traditional machine learning and pre-trained model-based deep learning were critically analysed with the set evaluation criteria (ASSURED criteria). The features were optimised using univariate analysis and exploratory data analysis to improve the performance. The image processing algorithm showed >98% accuracy while the classification accuracy by Least Squares Support Vector Machine (LS-SVM) was 100%. On the other hand, the deep learning technique provided >86% accuracy, which could be further improved with a large amount of data. The k-fold cross-validated LS-SVM based final system, examined on different datasets, confirmed the robustness and reliability of the presented approach, which was further validated using statistical analysis. The understaffed and resource-limited healthcare system can benefit from such an easy-to-use technology to support remote aid workers, assist in elderly care and promote personalised healthcare by eliminating the subjectivity of interpretation.