Flexible Beta: A Novel Policy Distribution for Proximal Policy Optimization

Continuous control problems in reinforcement learning often involve bounded action spaces and may require multimodal action distributions to represent multiple valid strategies. Standard policy parameterizations based on Gaussian distributions are ill-suited for these settings, as their unbounded support can introduce bias in constrained environments. We propose the integration of the Flexible Beta distribution, a structured mixture of beta distributions, into the Proximal Policy Optimization. The Flexible Beta naturally enforces bounded support while providing increased flexibility through its inherent multimodality, making it well-suited for continuous control tasks with complex action preferences. We evaluate the proposed approach in autonomous driving scenarios. Experimental results show that PPO with Flexible Beta achieves efficient learning and robust performance, highlighting its potential as an effective alternative for policy optimization.