Fuzzy-MAP EM: Addressing Data Scarcity in Sequential Decision-Making through Expert Knowledge Modeling

Real-world problems are often characterized by uncertainty and scarcity of observational data, requiring human experts to rely on qualitative heuristics and domain knowledge that are rarely made explicit in standard artificial intelligence pipelines. This issue is evident in sequential decision-making problems, which can be naturally modeled using Partially Observable Markov Decision Processes (POMDPs), offering a probabilistic framework to represent latent dynamics, observations, and treatment effects. POMDPs' parameters are typically learned by using the data-driven Expectation-Maximization (EM) algorithm. However, in real-world domains characterized by data scarcity, EM frequently overfits or fails to converge on meaningful parameters. While Maximum A Posteriori (MAP) estimation offers a method for regularizing learning using prior knowledge, standard conjugate priors are mathematically rigid and notoriously difficult to define accurately. We address this challenge by reformulating POMDP learning as a MAP EM problem, and integrating domain knowledge through an expert-defined fuzzy model that generates fuzzy pseudo-counts serving as informative priors. This novel approach, named Fuzzy-MAP EM, ensures convergence within the generalized EM setting and offers principled regularization in data-scarce and uncertain environments by means of adaptive hyperparameters and an Empirical Bayes framework. In synthetic simulations, Fuzzy-MAP EM consistently outperforms the standard EM and weakly informative MAP algorithms across both low-data and high-noise regimes. A case study on Myasthenia Gravis, a neuromuscular autoimmune rare disease characterized by data scarcity, further demonstrates the ability of Fuzzy-MAP EM to recover a clinically interpretable POMDP, underscoring its promise as a practical and data-efficient modeling tool for complex, real-world applications.