“Whenever you’re engaging vendors, what’s their training data?” Idziak said. “From a fair lending perspective, we have ECOA, so you can’t discriminate based on sex, race, or national origin. But if you have a vendor from outside the space that’s come in saying, ‘Hey, I’m going to help you underwrite your loans,’ one, what data do they train on? Two, how does it do its thinking, and how is it producing the result? Because for ECOA adverse action notices, you need to have a reason. You can’t just say ‘The AI said so.’ Well, why did it say so? ‘I don’t know. It’s a black box.'”
What regulators are looking for
A central concern is whether AI tools are staying in their lane. Regulators are asking whether systems can access or pull in data they were never meant to see, a risk that gets serious fast when tools are designed to connect information across multiple platforms. Vendor chains are getting the same treatment, with supervisors pressing banks on whether third-party AI providers and their subcontractors are held to the same standards as the banks themselves.
Michelle Bowman, the Fed’s vice chair for supervision, signaled in an April speech that the existing toolkit may not be enough.
“Today, banks are relying on existing risk-management frameworks to guide their use of AI,” Bowman said. “While these supervisory tools are intended to support banks in applying sound governance and risk management, we should assess whether our supervisory guidance is fit for the future.”
Experts are concerned that formal guidance, when it does arrive, risks being outdated before the ink is dry. The technology is moving faster than the regulatory process was built to handle.
