DFKI researchers warn against X-hacking in explainable AI
Researchers from the German Research Center for Artificial Intelligence (DFKI) have presented a new risk in the field of explainable artificial intelligence (XAI) at the International Conference on Machine Learning (ICML) 2025: “X-Hacking”. The study by Prof. Sebastian Vollmer’s team from the Data Science and its Applications research department shows how automated model selection by AutoML tools can jeopardize the trustworthiness of AI.
X-hacking describes two mechanisms: cherry-picking, in which models with desired explanations are specifically selected, and directed search, in which AutoML systems prefer models with specific explanation patterns. It is particularly problematic that different models can provide contradictory explanations despite similar predictive performance. This poses risks in sensitive areas such as medicine or the social sciences, where explainable models form the basis for decision-making.
AutoML tools that automate model development and optimization lower the entry barriers, but make it more difficult to understand decisions. The DFKI team recommends transparency measures such as explanation histograms, complete documentation of the model search space and interdisciplinary training to ensure methodological accuracy.
The study underscores DFKI’s focus on “Trustworthy AI” and advocates a scientific culture that prioritizes not only accuracy, but also honesty in explainability.
Original Paper:
X-Hacking: The Threat of Misguided AutoML | OpenReview
Editorial office: X-Press Journalistenbüro GbR
Gender note. The personal designations used in this text always refer equally to female, male and diverse persons. Double/triple references and gendered designations are avoided for the sake of better readability ected.
Most read
Hamburg becomes a pioneer: PROVIREX enables local CAR-T cell production CAR T-cell therapy has been revolutionizing the treatment of certain blood cancers in Germany since...- Virtual shock room at Ulm University digitally enhances medical studies The Faculty of Medicine at Ulm University is introducing an innovative training method for medical s...
- New approach to rapid sepsis diagnosis using AI and hyperspectral imaging Researchers at the German Cancer Research Center (DKFZ ) and the...
