An Evaluation Of A Linguistically Motivated Conversational Software Agent Framework

[EN] This paper presents a critical evaluation framework for a linguistically motivated conversational software agent (CSA). The CSA prototype investigates the integration, intersection and interface of the language, knowledge, and speech act constructions (SAC) based on a grammatical object, and the sub-model of belief, desires and intention (BDI) and dialogue management (DM) for natural language processing (NLP). A long-standing issue within NLP CSA systems is refining the accuracy of interpretation to provide realistic dialogue to support human-to-computer communication. This prototype constitutes three phase models: (1) a linguistic model based on a functional linguistic theory – Role and Reference Grammar (RRG), (2) an Agent Cognitive Model with two inner models: (a) a knowledge representation model, (b) a planning model underpinned by BDI concepts, intentionality and rational interaction, and (3) a dialogue model. The evaluation strategy for this Java-based prototype is multi-approach driven by grammatical testing (English language utterances), software engineering and agent practice. A set of evaluation criteria are grouped per phase model, and the testing framework aims to test the interface, intersection and integration of all phase models. The empirical evaluations demonstrate that the CSA is a proof-of-concept, demonstrating RRG’s fitness for purpose for describing, and explaining phenomena, language processing and knowledge, and computational adequacy. Contrastingly, evaluations identify the complexity of lower level computational mappings of NL – agent to ontology with semantic gaps, and further addressed by a lexical bridging solution. Panesar, K. (2019). An Evaluation Of A Linguistically Motivated Conversational Software Agent Framework. Journal of Computer-Assisted Linguistic Research. 3(3):41-66. https://doi.org/10.4995/jclr.2019.11118 Abran, A., A. Khelifi, W. Suryn, and A. Seffah. 2003. "Consolidating the ISO usability models. ." In proceedings of 11th International Software Quality Management Conference 2003:pp. 23-25. https://doi.org/10.1023/A:1025869312943 Bimson, Kent D., and Richard D. Hull. 2016. "Unnatural Language Processing: Characterizing the Challenges in Translating Natural Language Semantics into Ontology Semantics." In Semantic Web: Implications for Technologies and Business Practices, edited by Michael Workman, 119-135. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-16658-2_6 Bimson, Kent D., Ri