AI models, e.g., neural networks, are notoriously difficult to evaluate exhaustively, especially regarding so-called shortcuts and other failures in decision logic. Shortcuts describe ways in which the model – via its learned representations – circumvents the intended decision logic and decides based on spurious information, especially in images. Such models might perform well on the test data, but fail in deployment contexts. Especially for high-risk systems, such errors need to be identified before deployment.
Shortcuts and related defects in AI models can be detected via Explainable AI (XAI) methods. The Fraunhofer IPA developed an XAI method (RENTT – Runtime Efficient Network to Tree Transformation) with a guarantee on the correctness/fidelity of explanations, which other methods do not yield. In this service, this and other XAI methods will be used to find shortcuts in your AI model and evaluate its general decision logic, to make it fit for deployment.
Report with all identified shortcuts and errors in the decision logic and related countermeasures.