There are several papers which show the use of principle component analysis to reduce noise from a discretely sampled data. This method consists of embedding the data in higher dimensions and then using the singular value decomposition of the resulting matrix, retaining only the dominant components in the data. This method does involve throwing away some information and is not very effective if the sampling interval is not very small. If we know the underlying differential equation, we can find expressions for the neglected components and restore them. We first develop a theoretical basis for the procedure and then illustrate it in the case of some data generated using the Duffing's equation.