Seminar M.Sc Students

Join us with Zoom

Predicting User Decisions in DSS-Assisted Signal Detection 

Omri Bouchnick,  M.Sc. student in the School of Industrial & Intelligent Systems Engineering

Advisor: Prof. Joachim Meyer &Mr. Yoav Ben Yaakov

Abstract: 
Can single decisions be predicted in a Decision Support System (DSS) context? This thesis addresses this question through a controlled signal-detection experiment. A large-scale online study was conducted (N=569 participants, 3x11x11 factorial design) where users classified noisy stimuli as "Signal" or "Noise" while receiving DSS recommendations of varying reliability. Machine learning models were then trained to predict individual decisions using information available at decision time, including current-trial features and behavioral history.
The models achieved strong predictive performance: an XGBoost classifier reached 86.3% accuracy (AUC=0.937) for within-user prediction and 83.3% accuracy (AUC=0.910) when generalizing to entirely new users - substantially outperforming baseline heuristics. Feature importance analysis (SHAP) revealed that the DSS recommendation was the dominant predictor, followed by stimulus evidence and the known sensitivities of the DSS and human. Notably, removing behavioral history features reduced accuracy by only 2%, indicating that decisions are driven primarily by immediate context rather than past behavior.
Behavioral analyses provided additional insight: users followed the DSS on 76.6% of trials, and override decisions were significantly slower than follows, suggesting that disagreement requires cognitive effort rather than reflecting snap judgments. Despite reasonable performance (75.6% accuracy), participants left substantial value on the table: 60% would have done better by simply always following the DSS, and all fell far short of the optimal 86.2% achievable by integrating both DSS and stimulus evidence. These findings demonstrate that human-DSS decisions are highly predictable, supporting the design of adaptive interfaces that intervene at moments of likely conflict while highlighting persistent inefficiencies in human reliance strategies.

Bio:
Omri Bouchnick is an M.Sc. candidate in Industrial Engineering at Tel Aviv University and a Business-Data Science Analyst. His research focuses on human-automation interaction, trust calibration, and decision-making under uncertainty. 
Contact: 
E-Mail: bouchnick@mail.tau.ac.il
LinkedIn: https://www.linkedin.com/in/omri-bo/