Accurate forecasting of solar power generation with fine temporal and spatial resolution is vital for energy trading and energy management to support the integration of more renewable energy sources into the power grid. To address the limited resolution and computational costs of methods based on numerical weather forecasts, we take PV production data as main input for forecasting. Since PV power is affected by weather conditions and cloud dynamics, spatio-temporal correlations between production data of a network of PV systems change dynamically. We model them by representing PV systems as nodes in a graph and embedding production data as signals on that graph. We have modeled PV production time-series as signals on a graph with the intuition that for a sufficiently dense network of PV systems, graph-based models can exploit the spatio-temporal dependencies of PV production data to infer part of the cloud dynamics and forecast production more accurately. We present two graph neural network models for deterministic multi-site PV forecasting dubbed the graph-convolutional long short term memory (GCLSTM) and temporal-spatial multi-windows graph attention network (TSM-GAT) for predicting future PV power production. The former uses graph convolution with recurrent structures, while latter relies on graph attention network. The proposed models outperform state-of-the-art methods for deterministic multi-site PV forecasting, for a forecasting horizon of six hours ahead, over an entire year in one dataset distributed over Switzerland.