Reinforcement learning has the potential to assist clinical experts with decisions that may have long-term consequences.  However, training these models is challenging, and there are many reasons why a model may not behave as expected.  Statistical approaches to validation can help identify areas of concern, but many kinds of issues cannot be identified via statistical methods alone.  In this talk, I will discuss how we build models that are designed for expert validation from the start, as well as other ways in which one might work with experts to validate models.