Recent studies have shown that both biologic and non-biologic disease-based characteristics can be predicted from medical imaging. For instance, self-reported race, sex, and age can be predicted from chest X-rays, retinal scans and other imaging modalities, as well as disease conditions, such as ICD code diagnoses. Additional research has indicated that disease risk, such as the risk of breast cancer, can be predicted from medical imaging with better performance than clinical and traditional scoring systems like the Tyrer-Cuzick and Gail breast cancer risk prediction. These models prove to be powerful even when they lack high precision labels from radiologists.   

However, these image models face challenges due to the inadequacy of existing explanatory techniques. Furthermore, given the known issue of shortcut learning causing bias, there is increased concern over the use of image-only models. Conversely, if properly leveraged, image-only models can be successful, particularly for opportunistic screenings and mining information for population health, even if their initial intent was not for subsequent use.

I will discuss image-only models and their methodologies that have demonstrated superior performance over non-traditional imaging-only models, as well as the challenges and limitations of scaling imaging for precision medicine, specifically shortcut learning and limited explainability techniques.