We propose a novel approach for real-time image registration for image sequences of organs subject to repetitive movement, such as breathing. The method exploits the redundancy within the images and consists of a training and an application phase. During training, the images are registered and then the relationship between the image appearance and the spatial transformation is learned by employing dimensionality reduction to the images and storage of the corresponding displacements. For each image in the application phase, the most similar images in the training set are selected for predicting the associated displacements. Registration and update of the training data is only performed for outliers. The method is assessed on 2D sequences (4 MRI, 1 ultrasound) of the liver during free breathing. The performance is evaluated on manually selected landmarks, such as vessel centers and the distal point of the inferior segment. The proposed algorithm is real-time (9 ms per frame) and the prediction error is on average 1.2 mm for both MRI and ultrasound.