How to Assess the Quality of a Chatbot

Log in
Menu
English | Dutch | German | Spanish | French

    Dear Community Members

    soon many contents will be available only for registered members, please start to make the registration ( Register here ) for keeping updated with all future contents, it's free.

    Regards

How to Assess the Quality of a Chatbot

By Discovery Lean Six Sigma

0/5 stars (0 votes)

https://qualityandinnovation.files.wordpress.com/2013/01/doug-jan-e.jpg?w=600&h=434 600w, https://qualityandinnovation.files.wordpress.com/2013/01/doug-jan-e.jpg?w=150&h=108 150w" sizes="(max-width: 300px) 100vw, 300px">

Image Credit: Doug Buckley of http://hyperactive.to

Quality is the “totality of characteristics of an entity that bear upon its ability to meet stated and implied needs.” (ISO 9001:2015, p.3.1.5) Quality assurance is the practice of assessing whether a particular product or service has the characteristics to meet needs, and through continuous improvement efforts, we use data to tell us whether or not we are adjusting those characteristics to more effectively meet the needs of our stakeholders.

But what if the entity is a chatbot?

In June 2017, we published a paper that explored that question. We mined the academic and industry literature to determine 1) what quality attributes have been used by others to determine chatbot quality, we 2) organized them according to the efficiency, effectiveness, and satisfaction (using guidance from the ISO 9241 definition of usability), and 3) we explored the utility of Saaty’s Analytic Hierarchy Process (AHP) to help organizations select between one or more versions of chatbots based on quality considerations. (It’s sort of like A/B testing for chatbots.)

“There are many ways for practitioners to apply the material in this article:

  • The quality attributes in Table 1 can be used as a checklist for a chatbot implementation team to make sure they have addressed key issues.
  • Two or more conversational systems can be compared by selecting the most significant quality attributes.
  • Systems can be compared at two points in time to see if quality has improved, which may be particularly useful for adaptive systems that learn as they as exposed to additional participants and topics.”





Original: https://qualityandinnovation.com/2017/12/31/how-to-assess-the-quality-of-a-chatbot/
By: Nicole Radziwill
Posted: December 31, 2017, 10:40 pm

comments powered by Disqus

Discovery Lean Six Sigma

Dummy user for scooping articles

I'm a dummy user created for scooping  great articles in the network for the community.