Link to Black Knights post on their website

Link to where you can download the paper from Black Knight

Black Knight recently released an outstanding white paper on AI and Machine Learning. While focused on the mortgage industry, they do a great job of addressing questions on the use of AI in general.

Understanding how to apply probabilistic software (AI/ML) to your business is akin to learning how to use a search engine in the early 2000’s. This will soon become second nature to us. Importantly, even minor improvements in accuracy or efficiency can have major positive effects on the bottom line.

It’s great to see a company like Black Knight helping build the framework for how others in the industry should think about AI/ML in their businesses. I’ve pulled out a few snippets that I thought were particularly poignant:

• “The principal regulatory concern centers on fair lending: proving that no consumer is harmed by bias in an application of AI/ML in the lending process.” Black Knight defines “the regulatory risk associated with the possible introduction of bias that can result in unfair treatment of a consumer” as “Regulatory Bias Risk”.

Black Knight gets the reader to understand that since AI works differently than traditional software, we must think about it in a different way.

• “The output of an AI/ML model is a “probably,” not a consistently predictable result. No matter how good the model is, it will occasionally be wrong….”
• “High performance is not necessarily required for a model to be useful. Business value can accrue from the presence of a prediction, even if the prediction has relatively low [performance].”
• “Managers need to correctly weigh the quest for rightness as they set performance goals for a model. This has always been true of human processes: cost-effective quality management programs are central to managing any business, and such programs rarely expect to get it right 100% of the time. The same is true of AI/ML models.”
• “Unlike traditional software, AI/ML is not a set-and-forget technology.” Models must be continuously updated with evolving data and model architectures.
• “AI/ML is an emerging technology, and it does not stand alone.” “The core infrastructure and human capital that makes your traditional software work effectively is also needed for AI/ML. The secret is to gain an understanding of how the traditional and the disruptive differ, and how they work together”
• “Providing stakeholders across the organization with educational materials and regular briefings” is particularly important.
• “Developing quality AI/ML is not cheap. Investment in the full set of capabilities needed to manage AI/ML solutions is considerable. The pursuit of transparency cannot overwhelm the protection of intellectual property.” “Explainability demand cannot overcome a software owner’s right to protect their investment by requiring disclosure of trade secrets.”

As the demand for AI/ML grows in business applications, frameworks on how to think about and evaluate AI/ML will become a valuable strategic tool in adoption.