Conversational artificial intelligence - demystifying statistical vs linguistic NLP solutions
Keywords:conversational artificial intelligence, knowledge representation, machine/deep learning, natural language understanding/processing, role and reference grammar
AbstractThis paper aims to demystify the hype and attention on chatbots and its association with conversational artificial intelligence. Both are slowly emerging as a real presence in our lives from the impressive technological developments in machine learning, deep learning and natural language understanding solutions. However, what is under the hood, and how far and to what extent can chatbots/conversational artificial intelligence solutions work – is our question. Natural language is the most easily understood knowledge representation for people, but certainly not the best for computers because of its inherent ambiguous, complex and dynamic nature. We will critique the knowledge representation of heavy statistical chatbot solutions against linguistics alternatives. In order to react intelligently to the user, natural language solutions must critically consider other factors such as context, memory, intelligent understanding, previous experience, and personalized knowledge of the user. We will delve into the spectrum of conversational interfaces and focus on a strong artificial intelligence concept. This is explored via a text based conversational software agents with a deep strategic role to hold a conversation and enable the mechanisms need to plan, and to decide what to do next, and manage the dialogue to achieve a goal. To demonstrate this, a deep linguistically aware and knowledge aware text based conversational agent (LING-CSA) presents a proof-of-concept of a non-statistical conversational AI solution.
AbuShawar, Bayan, and Eric Atwell. 2015. "ALICE chatbot: Trials and outputs." Computación y Sistemas 19 (4):625-632. https://doi.org/10.13053/cys-19-4-2326
Alammar, Jay 2018. "Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention)." accessed 12 December 2019. https://jalammar.github.io/visualizing-neural-machine-translation-mechanics-of-seq2seq-models-with-attention/.
Bahdanau, Dzmitry, Jan Chorowski, Dmitriy Serdyuk, Philemon Brakel, and Yoshua Bengio. 2016. "End-to-end attention-based large vocabulary speech recognition." 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP). https://doi.org/10.1109/ICASSP.2016.7472618
Ball, John. 2019. "The Problem with AI State of the Art Methodology." Pat Inc., accessed 1 June 2019. https://medium.com/datadriveninvestor/the-problem-with-ai-state-of-the-art-methodology-db30762d4b84.
Banerjee, Satanjeev, and Alon Lavie. 2005. "METEOR: An automatic metric for MT evaluation with improved correlation with human judgments." Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization.
Bhashkar, Kunal. 2019. "Conversational AI Chatbot using Deep Learning: How Bi-directional LSTM, Machine Reading Comprehension, Transfer Learning, Sequence to Sequence Model with multi-headed attention mechanism, Generative Adversarial Network, Self Learning based Sentiment Analysis and Deep Reinforcement Learning can help in Dialog Management for Conversational AI chatbot." accessed 12 November 2019. https://medium.com/@BhashkarKunal/conversational-ai-chatbot-using-deep-learning-how-bi-directional-lstm-machine-reading-38dc5cf5a5a3.
Bird, Steven, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python: O'Reilly Media, Inc.
Biswas, Sourav. 2019. "The Basics Of Conversation Designing in Chatbots." https://medium.com/datadriveninvestor/the-basics-of-conversation-designing-in-chatbots-c15491bd3ec4.
Bourke, Daneil. 2018. "An In-depth Review of Andrew Ng's deeplearning.ai Specialization." Towards Data Science, accessed 12 October 2019. https://towardsdatascience.com/an-in-depth-review-of-andrew-ngs-deeplearning-ai-speciliazation-99612991eb61.
Bratman, Michael. 1987. Intention, plans, and practical reason. Center for the Study of Language and Information: Stanford University.
Cho, Kyunghyun, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. "Learning phrase representations using RNN encoder-decoder for statistical machine translation." arXiv preprint arXiv:1406.1078. https://doi.org/10.3115/v1/D14-1179
Cohen, Philip R, and Hector J Levesque. 1988. Rational interaction as the basis for communication. DTIC Document.
Cohen, Philip R, and Hector J Levesque. 1990. "Intention is choice with commitment." Artificial Intelligence 42 (2):213-261. https://doi.org/10.1016/0004-3702(90)90055-5
Collobert, Ronan, and Jason Weston. 2008. "A unified architecture for natural language processing: Deep neural networks with multitask learning." Proceedings of the 25th international conference on Machine learning. https://doi.org/10.1145/1390156.1390177
Denil, Misha, Alban Demiraj, Nal Kalchbrenner, Phil Blunsom, and Nando de Freitas. 2014. "Modelling, visualising and summarising documents with a single convolutional neural network." arXiv preprint arXiv:1406.3830.
Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. "Bert: Pre-training of deep bidirectional transformers for language understanding." arXiv preprint arXiv:1810.04805.
Ekzarian, Daniil 2018. "Hack4th0n & RASA chatbot." 16 May 2019. https://medium.com/reflash-programming-adventures/hack4th0n-rasa-chatbot-19c0778b4da9.
Feldman, Dave. 2018. "Chatbots: What happened?". https://chatbotslife.com/chatbots-what-happened-dcc3f91a512c.
Gao, Jianfeng, Michel Galley, and Lihong Li. 2019. "Neural approaches to conversational AI." Foundations and Trends® in Information Retrieval 13 (2-3):127-298. https://doi.org/10.1561/1500000074
Gartner. 2018. "Emerging Technology Trends in 2018." accessed 6 May 2019. https://www.gartner.com/smarterwithgartner/5-trends-emerge-in-gartner-hype-cycle-for-emerging-technologies-2018/.
Grunwitz, Kai. 2017. "Chatbot: the good, the bad and the ugly." 2 February 2018. https://twitter.com/KaiGrunwitz/status/1010025618220544000.
Joulin, Armand, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hérve Jégou, and Tomas Mikolov. 2016. "Fasttext. zip: Compressing text classification models." arXiv preprint arXiv:1612.03651.
Kamp, Hans, Josef Van Genabith, and Uwe Reyle. 2011. "Discourse representation theory." In Handbook of philosophical logic, 125-394. Springer. https://doi.org/10.1007/978-94-007-0485-5_3
Kim, Yoon. 2014. "Convolutional neural networks for sentence classification." arXiv preprint arXiv:1408.5882. https://doi.org/10.3115/v1/D14-1181
Lester, James, Karl Branting, and Bradford Mott. 2004. "Conversational agents." The Practical Handbook of Internet Computing, CRC Press, UK.
Li, Jiwei, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2015. "A diversity-promoting objective function for neural conversation models." arXiv preprint arXiv:1510.03055. https://doi.org/10.18653/v1/N16-1014
Li, Xiang, Lili Mou, Rui Yan, and Ming Zhang. 2016. "Stalematebreaker: A proactive content-introducing approach to automatic human-computer conversation." arXiv preprint arXiv:1604.04358.
Liddy, Elizabeth D. 2001. "Natural language processing." In Encyclopedia of Library and Information Science. NY: Marcel Decker, Inc.
Lin, Chin-Yew, and FJ Och. 2004. "Looking for a few good metrics: ROUGE and its evaluation." Ntcir Workshop.
Liu, Chia-Wei, Ryan Lowe, Iulian V Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. "How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation." arXiv preprint arXiv:1603.08023. https://doi.org/10.18653/v1/D16-1230
Lowe, Ryan, Michael Noseworthy, Iulian V Serban, Nicolas Angelard-Gontier, Yoshua Bengio, and Joelle Pineau. 2017. "Towards an automatic turing test: Learning to evaluate dialogue responses." arXiv preprint arXiv:1708.07149. https://doi.org/10.18653/v1/P17-1103
Mao, Xuetao, Jean-Paul Sansonnet, and Lin Li. 2012. "Textual Conversation Agent for Enhancing Attraction in E-Learning." Proceedings of International Conference on Computer Science and Information Technology.
McTear, Michael Frederick, Zoraida Callejas, and David Griol. 2016. The conversational interface. Vol. 6: Springer. https://doi.org/10.1007/978-3-319-32967-3
Mikolov, Tomas, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. "Distributed representations of words and phrases and their compositionality." Advances in neural information processing systems.
Mou, Lili, Zhengdong Lu, Hang Li, and Zhi Jin. 2017. "Coupling distributed and symbolic execution for natural language queries." Proceedings of the 34th International Conference on Machine Learning-Volume 70.
Mou, Lili, Yiping Song, Rui Yan, Ge Li, Lu Zhang, and Zhi Jin. 2016. "Sequence to backward and forward sequences: A content-introducing approach to generative short-text conversation." arXiv preprint arXiv:1607.00970.
Nolan, Brian. 2014. "Constructions as grammatical objects : A case study of prepositional ditransitive construction in Modern Irish." In Linking Constructions into Functional Linguistics: The role of constructions in grammar, edited by Brian Nolan and Elke Diedrichsen, 143-178. Amsterdam/Philadelphia: John Benjamins Publishing Company. https://doi.org/10.1075/slcs.145.06nol
O'Shea, J. 2010. "A Framework for Applying Short Text Semantic Similarity in Goal Orientated Conversational Agents." PhD thesis PhD thesis, School of Computing, Mathematics and Digital Technology, Manchester Metropolitan University.
O'Shea, Karen, Zuhair Bandar, and Keeley Crockett. 2010. "A conversational agent framework using semantic analysis." International Journal of Intelligent Computing Research (IJICR) 1 (1/2). https://doi.org/10.20533/ijicr.2042.4655.2010.0002
Omarsar, Elvis. 2018. "Deep Learning for NLP: An Overview of Recent Trends." accessed 1 May 2019. https://medium.com/dair-ai/deep-learning-for-nlp-an-overview-of-recent-trends-d0d8f40a776d.
Pan, Sinno Jialin, and Qiang Yang. 2009. "A survey on transfer learning." IEEE Transactions on knowledge and data engineering 22 (10):1345-1359. https://doi.org/10.1109/TKDE.2009.191
Panesar, Kulvinder. 2017. "A linguistically centred text-based conversational software agent." Unpublished PhD thesis, School of Computing, Creative Technologies and Engineering, Leeds Beckett University.
Panesar, Kulvinder. 2019a. "CHAPTER FIFTEEN FUNCTIONAL LINGUISTIC BASED MOTIVATIONS FOR A CONVERSATIONAL SOFTWARE AGENT KULVINDER PANESAR." In Linguistic Perspectives on the Construction of Meaning and Knowledge, edited by Brian Nolan and Elke Diedrichsen, 340. Cambridge Scholars Publishing. https://doi.org/10.4995/jclr.2019.11118
Panesar, Kulvinder. 2019b. "An Evaluation Of A Linguistically Motivated Conversational Software Agent Framework." Journal of Computer-Assisted Linguistic Research 3 (3):41-66. https://doi.org/10.4995/jclr.2019.11118
Peebles, D. 2012. "Artificial Intelligence and Cognitive Systems." The Newsletter of the Society for the Study of Artifical Intelligence and Simulation of Behaviour, January 2012, 16.
Pennington, Jeffrey, Richard Socher, and Christopher Manning. 2014. "Glove: Global vectors for word representation." Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). https://doi.org/10.3115/v1/D14-1162
Radziwill, Nicole M, and Morgan C Benton. 2017. "Evaluating Quality of Chatbots and Intelligent Conversational Agents." arXiv preprint arXiv:1704.04579.
Ritter, Alan, Sam Clark, and Oren Etzioni. 2011. "Named entity recognition in tweets: an experimental study." Proceedings of the conference on empirical methods in natural language processing.
Ruder, Sebastian. 2016. "Highlights of NIPS 2016: Adversarial learning, Meta-learning, and more." accessed 12 December 2019. https://ruder.io/highlights-nips-2016/index.html#thenutsandboltsofmachinelearning.
Saifee, M. 2019a. "Recent Advancements in NLP (1/2)." The Startup, 1st December 2019. https://medium.com/swlh/recent-advancements-in-nlp-1-2-192ac7eefe3c.
Saifee, M. 2019b. "Recent Advancements in NLP (2/2)." The Startup, 1st December 2019. https://medium.com/swlh/recent-advancements-in-nlp-1-2-192ac7eefe3c.
Salem, Y., A. Hensman, and B. Nolan. 2008. "Implementing Arabic-to-English machine translation using the Role and Reference Grammar linguistic model." the 8th Annual Information Technology and Telecommunication Conference (IT&T 2008), Galway, Ireland.
Searle, John R. 1969. Speech acts: An essay in the philosophy of language. Vol. 626. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9781139173438
Searle, John R. 1983. Intentionality: An essay in the philosophy of mind. Cambridge Cambridge University Press. https://doi.org/10.1017/CBO9781139173452
Sharda, R, Dursun Delen, and E Turban. 2019. Analytics, Data Science, & Artificial Intelligence: Systems for Decision Support. Eleventh ed: Pearson.
Shewan, Dan. 2020. "10 of the Most Innovative Chatbots on the Web." WordStream. https://www.wordstream.com/blog/ws/2017/10/04/chatbots.
Singh, Seema. 2018. "Cousins of AI." Medium - Towards Data Science. https://towardsdatascience.com/cousins-of-artificial-intelligence-dda4edc27b55.
Stalnaker, Robert. 2002. "Common ground." Linguistics and philosophy 25 (5):701-721. https://doi.org/10.1023/A:1020867916902
Sundermeyer, Martin, Ralf Schlüter, and Hermann Ney. 2012. "LSTM neural networks for language modeling." Thirteenth annual conference of the international speech communication association.
Sutskever, I, O Vinyals, and QV Le. 2014. "Sequence to sequence learning with neural networks." Advances in NIPS.
Synced. 2019. "AI - Interview:Yoshua Bengio On AI Priors and Challenges." https://syncedreview.com/2019/02/16/yoshua-bengio-on-ai-priors-and-challenges/.
Tao, Chongyang, Lili Mou, Dongyan Zhao, and Rui Yan. 2018. "Ruber: An unsupervised method for automatic evaluation of open-domain dialog systems." Thirty-Second AAAI Conference on Artificial Intelligence.
Uszkoreit, Jakob. 2017. "Transformer: A Novel Neural Network Architecture for Language Understanding." Google AI Blog. https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html.
Van Valin Jr, Robert D. 2005. Exploring the syntax-semantics interface. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511610578.001
Van Valin, Robert D, and William A Foley. 1980. "Role and reference grammar." In Moravcik and Wirth:329-352. https://doi.org/10.1163/9789004373105_014
Van Valin, Robert D, and Randy J LaPolla. 1997. Syntax: Structure, meaning, and function. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9781139166799
Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. "Attention is all you need." Advances in neural information processing systems.
Wallace, Richard. 2018. "Chatbots - a personal perspective." Society for the study of Artifical Intelligence and Simulation of Behaviour (AISB) quarterly (No.149):6.
Wilson, H. J., Daugherty, P. R., & Morini-Bianzino, N. . 2017. "Will AI Create as Many Jobs as it Eliminates?". MIT Sloan Management Review, accessed 13 March 2017. http://sloanreview.mit.edu/article/will-ai-create-as-many-jobs-as-it-eliminates/.
Wooldridge, Michael. 2013. "Intelligent Agents." In Multiagent Systems, edited by Gerhard Weiss, 3-50. USA: Massachusetts Institute of Technology.
Woolf, Joseph. 2019. "An introdcution to Chatbots." accessed 12 March 2019. https://josephwoolf.com/an-introduction-to-chatbots/.
Yan, Rui. 2018. "" Chitty-Chitty-Chat Bot": Deep Learning for Conversational AI." IJCAI. https://doi.org/10.24963/ijcai.2018/778
Yan, Rui, Yiping Song, and Hua Wu. 2016. "Learning to respond with deep neural networks for retrieval-based human-computer conversation system." Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. https://doi.org/10.1145/2911451.2911542
Yao, Lili, Yaoyuan Zhang, Yansong Feng, Dongyan Zhao, and Rui Yan. 2017. "Towards implicit content-introducing for generative short-text conversation systems." Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.18653/v1/D17-1233
Zhang, Ye, and Byron Wallace. 2015. "A sensitivity analysis of (and practitioners' guide to) convolutional neural networks for sentence classification." arXiv preprint arXiv:1510.03820.
Zumstein, Darius, and Sophie Hundertmark. 2017. "Chatbots - an interactive technology for personalized communication, transactions and services." IADIS International Journal on WWW/Internet 15 (1).
This journal is licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License