With the help of modern technology in telemedicine, significant progress has been achieved in the remote diagnosis and treatment of patients. These solutions rely on telecommunication technology to transmit data; however, their applications have been confined to measurements that can be taken mechanically such as monitoring blood pressure, breathing rate and heart rate. Imaging solutions are rarely used for Remote Patient Monitoring (RPM) because they require skills and present high costs. In marginalized areas that are in dire need of RPM, these challenges present serious barriers. This research presents a prototype that automates RPM through computer vision. The solution makes use of object tracking and object detection techniques to process images and videos taken by ordinary digital cameras found in mobile phones to monitor infants' physical and cognitive developmental milestones remotely. Experimental results from simulated lab tests show that computer vision can automate the process used by paediticians to monitor a baby's ability to follow objects visually with their eyes or move their head. Limited resources and access to health care in developing countries has reduced the frequency of postnatal care visits and subsequently increased infant mortality rates. In this way the proposed solution supports the global sustainable development goals of promiting healthcare for all.