Generalisation remains a fundamental problem in the implementation of modern machine learning techniques and in particular the problem of embedding domain dependent information in a principled fashion. This problem has inspired my interest in hybrid models which aim to combine the flexibility of modern machine learning with the ability to embed a certain amount of mechanistic structure at the modelling stage.
This then is a quick announcement of the release of my package
PydyGP: a Python package for Dynamic systems with latent Gaussian Processes
The aim is to provide a usable and robust implementation of a class of hybrid machine-learning methods for systems with time dependent features by combining simple dynamic equations with flexible Gaussian processes in an attempt to achieve the best of both worlds. While I am still in the process of releasing the full code from my hard drive in to the light over on github it has got to the point where there is now enough code moved across to be of interest to a larger audience.
Where can I find pydygp?
The package is hosted over on PyPi and so should be installable with
pip install pydygp
However in the short term the package is likely to grow quite rapidly and so cloning the the repository on github is probably advisable.
Features
As a near term goal I will be primarily focused on:
- (Additive) Linear latent force models: These models are described in [1], they combine simple linear ODEs with a flexible Gaussian process term
- Multiplicative latent force models: This is an extension of the latent force model to allow interactions between the state variables and the latent forces
Where in both cases $\mathbf{g}$ and ${ g_r }$ represent latent Gaussian processes.
One approach to fitting the second class of models is the adaptive gradient matching methods introduced in [2] and so at a future date I may adapt the existing code to handle the Metropolis-Hastings fitting of more general nonlinear ODE models using the adaptive gradient approach. However is worth noting that these models are philisophically quite different, often being very well specified mechanistic models with a small set of random parameters, and unlike the two evolution equations introduced above where the GP forces drive the equation, the use of the GP terms in the adaptive gradient matching approach is as an interpolater of the latent states used to avoid the explicit solution of the ODE.
Further Info
I will be continuously updating the documentation and in particular would point interested parties in the direction of the user guide and examples contained within.
Contribution
The project is very much in its infancy but if you are interested in contributing anything from documentation, tutorials and examples or any relevant modules do get in touch at <package name>@gmail.com.