Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updating Flux 0.36 -> 0.37 gives error: CannotUpdate #17

Open
WillNilges opened this issue Nov 28, 2022 · 3 comments
Open

Updating Flux 0.36 -> 0.37 gives error: CannotUpdate #17

WillNilges opened this issue Nov 28, 2022 · 3 comments

Comments

@WillNilges
Copy link

A few days ago, Flux started trying to update itself from 0.36 to 0.37. It then began failing. The 0.37 version is stuck in a Pending state due to a RequirementsNotMet: one or more requirements couldn't be found error. 0.36 was in a Failing state.

image

OLM Logs:

I1128 02:12:22.717658       1 event.go:282] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"", Name:"flux-system", UID:"203c94e7-1df1-4a48-a2e2-d882dd13c68d", APIVersion:"v1", ResourceVersion:"539695358", FieldPath:""}): type: 'Warning' reason: 'ResolutionFailed' constraints not satisfiable: clusterserviceversion flux.v0.36.0 exists and is not referenced by a subscription, subscription flux exists, subscription flux requires @existing/flux-system//flux.v0.37.0, @existing/flux-system//flux.v0.37.0 and @existing/flux-system//flux.v0.36.0 provide HelmRelease (helm.toolkit.fluxcd.io/v2beta1)
time="2022-11-28T02:12:23Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=
I1128 02:12:23.716878       1 event.go:282] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"", Name:"flux-system", UID:"203c94e7-1df1-4a48-a2e2-d882dd13c68d", APIVersion:"v1", ResourceVersion:"539695358", FieldPath:""}): type: 'Warning' reason: 'ResolutionFailed' constraints not satisfiable: @existing/flux-system//flux.v0.37.0 and @existing/flux-system//flux.v0.36.0 originate from package flux, clusterserviceversion flux.v0.36.0 exists and is not referenced by a subscription, subscription flux exists, subscription flux requires @existing/flux-system//flux.v0.37.0
time="2022-11-28T02:12:24Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=
I1128 02:12:24.717529       1 event.go:282] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"", Name:"flux-system", UID:"203c94e7-1df1-4a48-a2e2-d882dd13c68d", APIVersion:"v1", ResourceVersion:"539695358", FieldPath:""}): type: 'Warning' reason: 'ResolutionFailed' constraints not satisfiable: subscription flux requires @existing/flux-system//flux.v0.37.0, @existing/flux-system//flux.v0.37.0 and @existing/flux-system//flux.v0.36.0 provide Kustomization (kustomize.toolkit.fluxcd.io/v1beta2), clusterserviceversion flux.v0.36.0 exists and is not referenced by a subscription, subscription flux exists
time="2022-11-28T02:12:25Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=
I1128 02:12:25.716471       1 event.go:282] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"", Name:"flux-system", UID:"203c94e7-1df1-4a48-a2e2-d882dd13c68d", APIVersion:"v1", ResourceVersion:"539695358", FieldPath:""}): type: 'Warning' reason: 'ResolutionFailed' constraints not satisfiable: subscription flux exists, clusterserviceversion flux.v0.36.0 exists and is not referenced by a subscription, @existing/flux-system//flux.v0.37.0 and @existing/flux-system//flux.v0.36.0 provide Bucket (source.toolkit.fluxcd.io/v1beta1), subscription flux requires @existing/flux-system//flux.v0.37.0
time="2022-11-28T02:12:26Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=
I1128 02:12:26.717348       1 event.go:282] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"", Name:"flux-system", UID:"203c94e7-1df1-4a48-a2e2-d882dd13c68d", APIVersion:"v1", ResourceVersion:"539695358", FieldPath:""}): type: 'Warning' reason: 'ResolutionFailed' constraints not satisfiable: @existing/flux-system//flux.v0.37.0 and @existing/flux-system//flux.v0.36.0 provide ImagePolicy (image.toolkit.fluxcd.io/v1beta1), clusterserviceversion flux.v0.36.0 exists and is not referenced by a subscription, subscription flux exists, subscription flux requires @existing/flux-system//flux.v0.37.0
time="2022-11-28T02:12:27Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=
I1128 02:12:27.718395       1 event.go:282] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"", Name:"flux-system", UID:"203c94e7-1df1-4a48-a2e2-d882dd13c68d", APIVersion:"v1", ResourceVersion:"539695358", FieldPath:""}): type: 'Warning' reason: 'ResolutionFailed' constraints not satisfiable: subscription flux exists, @existing/flux-system//flux.v0.36.0 and @existing/flux-system//flux.v0.37.0 originate from package flux, clusterserviceversion flux.v0.36.0 exists and is not referenced by a subscription, subscription flux requires @existing/flux-system//flux.v0.37.0
time="2022-11-28T02:12:28Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=
I1128 02:12:28.718722       1 event.go:282] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"", Name:"flux-system", UID:"203c94e7-1df1-4a48-a2e2-d882dd13c68d", APIVersion:"v1", ResourceVersion:"539695358", FieldPath:""}): type: 'Warning' reason: 'ResolutionFailed' constraints not satisfiable: subscription flux exists, clusterserviceversion flux.v0.36.0 exists and is not referenced by a subscription, @existing/flux-system//flux.v0.36.0 and @existing/flux-system//flux.v0.37.0 provide ImageRepository (image.toolkit.fluxcd.io/v1beta1), subscription flux requires @existing/flux-system//flux.v0.37.0
time="2022-11-28T02:12:29Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=
I1128 02:12:29.718087       1 event.go:282] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"", Name:"flux-system", UID:"203c94e7-1df1-4a48-a2e2-d882dd13c68d", APIVersion:"v1", ResourceVersion:"539695358", FieldPath:""}): type: 'Warning' reason: 'ResolutionFailed' constraints not satisfiable: subscription flux exists, clusterserviceversion flux.v0.36.0 exists and is not referenced by a subscription, @existing/flux-system//flux.v0.36.0 and @existing/flux-system//flux.v0.37.0 provide Provider (notification.toolkit.fluxcd.io/v1beta1), subscription flux requires @existing/flux-system//flux.v0.37.0
time="2022-11-28T02:12:30Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=
I1128 02:12:30.717538       1 event.go:282] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"", Name:"flux-system", UID:"203c94e7-1df1-4a48-a2e2-d882dd13c68d", APIVersion:"v1", ResourceVersion:"539695358", FieldPath:""}): type: 'Warning' reason: 'ResolutionFailed' constraints not satisfiable: subscription flux requires @existing/flux-system//flux.v0.37.0, subscription flux exists, @existing/flux-system//flux.v0.37.0 and @existing/flux-system//flux.v0.36.0 provide HelmChart (source.toolkit.fluxcd.io/v1beta2), clusterserviceversion flux.v0.36.0 exists and is not referenced by a subscription
time="2022-11-28T02:12:31Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=
I1128 02:12:31.717771       1 event.go:282] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"", Name:"flux-system", UID:"203c94e7-1df1-4a48-a2e2-d882dd13c68d", APIVersion:"v1", ResourceVersion:"539695358", FieldPath:""}): type: 'Warning' reason: 'ResolutionFailed' constraints not satisfiable: subscription flux exists, subscription flux requires @existing/flux-system//flux.v0.37.0, clusterserviceversion flux.v0.36.0 exists and is not referenced by a subscription, @existing/flux-system//flux.v0.37.0 and @existing/flux-system//flux.v0.36.0 provide HelmChart (source.toolkit.fluxcd.io/v1beta1)

I attempted to perform the steps in this RedHat article, but they didn't work (Delete the Subscription and CSVs, re-install).

Logs post re-install

time="2022-11-28T02:24:22Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=
time="2022-11-28T02:24:22Z" level=info msg=syncing id=b/zWa ip=install-k8vlf namespace=flux-system phase=Installing
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=ocirepository.crd.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=Service" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=webhook-receiver.service.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=ClusterRole" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=crd-controller-flux-system.clusterrole.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=kustomization.crd.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=ClusterRoleBinding" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=cluster-reconciler-flux-system.clusterrolebinding.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=receiver.crd.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=provider.crd.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=Service" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=source-controller.service.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=helmrelease.crd.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=imageupdateautomation.crd.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=imagepolicy.crd.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=gitrepository.crd.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=helmrepository.crd.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=helmchart.crd.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=imagerepository.crd.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=bucket.crd.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=ClusterServiceVersion" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=flux.v0.37.0.clusterserviceversion.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=alert.crd.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=ClusterRoleBinding" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=crd-controller-flux-system.clusterrolebinding.yaml
time="2022-11-28T02:24:23Z" level=info msg="added to bundle, Kind=Service" configmap=openshift-marketplace/117acbb8c3636cc0e69f7d4db4f352bb74df05c0154698c1603646f69a7a6d8 key=notification-controller.service.yaml
time="2022-11-28T02:24:23Z" level=error msg="risk of data loss updating \"imagepolicies.image.toolkit.fluxcd.io\": new CRD removes version v1alpha1 that is listed as a stored version on the existing CRD"

Anyone seen this issue before? I must admit, I'm not all that familiar with how operators work, so any help or advice would be appreciated.
@WillNilges
Copy link
Author

Upon attempting to re-install from the OperatorHub, I get this error on the Subscription page

risk of data loss updating "imagepolicies.image.toolkit.fluxcd.io": new CRD removes version v1alpha1 that is listed as a stored version on the existing CRD

@WillNilges
Copy link
Author

Uninstalled Flux via FluxCLI with flux uninstall and re-installed via the OperatorHub. That seems to have worked :)

@kingdonb
Copy link

Thanks @WillNilges for helping us trace down this issue!

The upgrade should be seamless, I'm not sure how we can address this issue in a future release to prevent this error, or if it's possible to bypass that error somehow else without uninstalling. (Maybe we can add upgrade testing to our validation plan?)

I'm making a lot of assumptions, but I believe that this is caused by the final deprecation of the v1alpha APIs that came in Flux v0.37, and that it won't happen again. The v1alpha APIs required a manual translation into v1beta in a much earlier version of Flux, such that there was no translator requirement to ensure that v1alpha APIs are automatically upgraded into beta when the new API version comes out.

But beta APIs have a higher guarantee level for production readiness in Kubernetes and in Flux. So the deprecation of alpha APIs is a signal that the General Availability release is coming soon 🎉 which means I think this will not happen again.

I think a manual upgrade of the CRDs might have also worked around this issue. I don't think OLM is being too conservative here, I think that it's probably unusual for operators that are still based on prerelease APIs to land in the catalog ahead of the API stability guarantees being hardened. I would be speculating about a lot of that stuff though, just giving some color to the report that hopefully helps explain a bit how we got here, on the way to Flux GA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kingdonb @WillNilges