A common issue when analyzing real-world complex systems is that the interactions between the elements often change over time. Here we propose a new modeling approach for time-varying interactions generalising the well-known Kinetic Ising Model, a minimalistic pairwise constant interactions model which has found applications in several scientific disciplines. Keeping arbitrary choices of dynamics to a minimum and seeking information theoretical optimality, the Score-Driven methodology significantly increases the knowledge that can be dynamically extracted from data. We identify a parameter whose value at a given time can be directly associated with the local predictability of the dynamics and we introduce a method to dynamically learn its value from the data, without specifying parametrically the system’s dynamics. We extend our framework to disentangle different sources (e.g. endogenous vs exogenous) of predictability in real time. We show several financial applications including the estimation of time-varying correlations, volatility dynamics, and interactions between real agents.