Designing Trust-Aware AI Systems: Measuring and Modeling Human Trust in AI Decisions

Abstract

Trust is crucial for successful human-AI collaboration, but humans’ confidence in AI decisions is often miscalibrated, leading to overconfidence or disuse. In this work, we propose a paradigm for the development of trust-aware AI systems, by quantifying human trust with well-established scales such as the Short Trust in AI Scale (S-TIAS) and modelling it with dynamic calibration models. We propose a methodology that integrates behavioral monitoring and contextual bandits to achieve adaptive trust adjustment. Simulations suggest a 12% improvement in team performance with confidence-based delegation. Our approach emphasizes key pillars including explainability and robustness for enabling safer AI deployment in high-stakes domains.