One of the key challenges of personalized medicine is to identify which patients will respond positively to a given treatment .The first step towards this is to identify the baseline variables (e.g. biomarkers) that influence the treatment effect, which are known as predictive biomarkers.  When we discover predictive biomarkers it is crucial to have control over the false-positives to avoid waste of resources, as well as provide guarantees over the replicability of our findings. With our work we introduce a set of methods for controlled predictive biomarker discovery, and we use them to explore heterogeneity in psoriatic arthritis trials.