Implementing other pre-packaged kernels in Regression Tutorial

[frankiezoo]

In the GPyTorch Regression tutorial, you use the ExactGPModel, which I've copy pasted here:

class ExactGPModel(gpytorch.models.ExactGP):
def init(self, train_x, train_y, likelihood):
super(ExactGPModel, self).init(train_x, train_y, likelihood)
self.mean_module = gpytorch.means.ConstantMean()
self.covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.RBFKernel())

def forward(self, x):
    mean_x = self.mean_module(x)
    covar_x = self.covar_module(x)
    return gpytorch.distributions.MultivariateNormal(mean_x, covar_x)

I'm trying to use other kernels, like Cosine, Cylindrical, Linear, etc., in the GPyTorch kernel documentation.
For the Matern, Periodic, and Piecewise Polynomial kernels, I can just replace the following line of code in the sample code with the kernel name, like so:

    self.covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.MaternKernel())

But using Polynomial, Cosine, Cylindrical, Linear, Spectral Mixture kernels is not as straightforward.
Do you have any tips as to how to define a GP model with these kernels I'm having trouble incorporating or any sample code that worked for you that used these kernels?

[gpleiss]

But using Polynomial, Cosine, Cylindrical, Linear, Spectral Mixture kernels is not as straightforward.

Is the challenge with fitting these kernels to data? The spectral mixture and cosine kernels are challenging to optimize. The linear and polynomial kernels should be relatively easy to optimize, but they aren't as expressive as RBF/Matern kernels, so they may not fit your data as well. I'm not as familiar with the cylindrical kernel.

[frankiezoo]

The challenge is indeed to fit these kernels to data, but I think it has to do with the inputs and structure of the code capturing the kernel transformation. You're right that the linear and polynomial kernels are not as expressive but I'm running a kernel study on some planetary science data and need to show the improvement of my GP model depending on the kernel. I also have a benchmark parabolic surface that the polynomial kernel might fit perfectly but want to prove that. Also, hi Geoff!

…

On Tue, Nov 9, 2021 at 2:25 AM Geoff Pleiss ***@***.***> wrote: But using Polynomial, Cosine, Cylindrical, Linear, Spectral Mixture kernels is not as straightforward. Is the challenge with fitting these kernels to data? The spectral mixture and cosine kernels are challenging to optimize. The linear and polynomial kernels should be relatively easy to optimize, but they aren't as expressive as RBF/Matern kernels, so they may not fit your data as well. I'm not as familiar with the cylindrical kernel. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1814 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACD66AEHNJ7HAE6QUVQG3ITULEHMNANCNFSM5HT55HGA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

-- Frances Zhu PhD, Aerospace Engineering franceszhu.space

[gpleiss]

Hi Frankie!!! Would you be able to share any code/plots that show the kernel fit?