Artificial Intelligence (AI) concocts an image of robots taking over the world, machines doing seemingly impossible things with absolute precision and devices powered by software doing unimaginable things! A crucial element in all of this is what we as humans experience and are either “wowed” or have a feeling of “eh”. There is a lot of thought, hard work and research that goes into producing the “wowed”experience that needs to constantly evolve.
Start with a “People-Focused” design. Most Artificial Intelligence systems are used by people and their experience is critical in evaluating the impact of these systems. Having a design that provides three options with Good / Better / Best answers will help users make an impactful decision. Even technically, it is easier to achieve good precision at a few answers than a single answer and limiting it to 3 makes it user friendly. Get early feedback and validation on the design from your actual users (preferably a diverse set of users) and ask them to test the live system in an iterative mode for better results.
Focus on Key Metrics. Work with your stakeholders to define multiple metrics that you would like to measure “what success looks like”. This will help you understand the tradeoffs between different types of issues and user experiences. Incorporate feedback from user surveys, false positive and false negative results and ensure that the key metrics are relevant to your system goals and objectives.
Understand the Raw Data. Losing focus on the raw data can lead to skewed results and loss of confidence on the machine learning (ML) models. If you do not understand the data, get the help from business experts to grasp a full understanding of what story the raw data is telling you. Analyze it to ensure that there are no missing values, incorrect labels, typos and check that the sample contains the full spectrum of all users that you wish to analyze, consider the relationship between data labels and values that you are trying to predict based on dependent data and ensure that there is no biased data (data favoring a particular result). While analyzing the raw data, you will get an understanding of the limitations of your data set and the model. This will help you communicate the scope and limitations of your predictions based on the pattern of the data to your stakeholders.
Test it to the end of the world. Testing the model and its predictions thoroughly will help you clean up the chinks in the armor. Keep a clean data set as a master that can be reused for every round of testing and incorporate new and changing use cases on this data set. Engage users for iterative user testing during the development cycle and don’t forget to continue to monitor the model for issues and factor in time for addressing the issues, both in the short term and long term.
I hope that you find these best practices useful in your journey with AI and ML. Please share with your comments on what other best practices worked for you!
Reposted from my original post on LinkedIn dated 4/22/19.