Safe Exogenous State Reinforcement Learning for Water Tank System

Safety constraints are essential in real-world sequential decision-making problems, especially when violations may lead to irreversible system disruptions. We study this problem in water tank control, where stochastic water demand influences safety but is unknown at decision time. We extend Feasibility-guided Safe Offline Reinforcement Learning (FISOR) to Exogenous Markov Decision Processes by explicitly accounting for exogenous uncertainty, with the aim of learning safe policies that minimize operational costs. Experiments on a realistic water tank system show that augmenting offline data with synthetic trajectories significantly reduces operational costs while preserving safety guarantees.