Abstraction and decision fusion architecture for resource-aware image understanding with application on handwriting character classification

Resource-aware image understanding aims to achieve a soft equilibrium by effectively managing the constraints imposed by computational resources while improving the inferential capabilities of image processing systems. It serves a broad range of applications, spanning from embedded systems and IoT devices to budget-friendly smartphones and tablets. The proposed Abstraction and Decision Fusion Architecture (ADFA) addresses this problem through three tiers: abstraction, computation, and decision fusion. The first tier contains various views to generate different abstractions from the original data. These views are processed independently by an array of lightweight models forming the computation tier. They make independent decisions, where the final output is produced using a decision fusion tool in the last tier. To assess the capability of the proposed architecture, we have developed several ADFA-based models for the classification of handwriting data. In this regard, we first defined three data abstractions. Then, we trained support vector machines and fully connected neural networks according to the abstractions, which led to a set of independent basic models. Finally, an adaptive neuro-fuzzy inference system is employed as their hub to increase the accuracy by performing the decision fusion. Our experiments on the EMNIST dataset verify the efficiency and high accuracy of the proposed architecture in recognizing handwritten data, where the model size and the number of Multiply-Accumulate (MAC) operations are significantly smaller than state-of-the-art computational models. [Display omitted] •A three-layer Abstraction and Decision Fusion Architecture (ADFA) is introduced.•ADFA enables balancing resource-aware computing with system inferential capabilities.•ADFA includes multiple data abstractions and fusing the different lightweight models.•Several ADFA-based models for classifying handwriting characters are realized.