Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support flagger canary resources #173

Open
dudicoco opened this issue Jul 6, 2020 · 5 comments
Open

Support flagger canary resources #173

dudicoco opened this issue Jul 6, 2020 · 5 comments

Comments

@dudicoco
Copy link

dudicoco commented Jul 6, 2020

Hi,

Flagger is a great tool for performing automated canary deployments in k8s: https://github.com/weaveworks/flagger/

Is it possible to support the flagger canary custom resource so that kubedog will output the events from the resource?

example:

kubectl -n test describe canary/podinfo

Status:
  Canary Weight:         0
  Failed Checks:         10
  Phase:                 Failed
Events:
  Type     Reason  Age   From     Message
  ----     ------  ----  ----     -------
  Normal   Synced  3m    flagger  Starting canary deployment for podinfo.test
  Normal   Synced  3m    flagger  Advance podinfo.test canary weight 5
  Normal   Synced  3m    flagger  Advance podinfo.test canary weight 10
  Normal   Synced  3m    flagger  Advance podinfo.test canary weight 15
  Normal   Synced  3m    flagger  Halt podinfo.test advancement success rate 69.17% < 99%
  Normal   Synced  2m    flagger  Halt podinfo.test advancement success rate 61.39% < 99%
  Normal   Synced  2m    flagger  Halt podinfo.test advancement success rate 55.06% < 99%
  Normal   Synced  2m    flagger  Halt podinfo.test advancement success rate 47.00% < 99%
  Normal   Synced  2m    flagger  (combined from similar events): Halt podinfo.test advancement success rate 38.08% < 99%
  Warning  Synced  1m    flagger  Rolling back podinfo.test failed checks threshold reached 10
  Warning  Synced  1m    flagger  Canary failed! Scaling down podinfo.test

Thanks!

@daviddelucca
Copy link

@dudicoco we will try to implement it. I will let you know as soon we have tested in our pipeline.

@daviddelucca
Copy link

@dudicoco #218

echo '{ "Canaries": [ { "ResourceName": "mycanary", "Namespace": "myns" } ] }' | go run main.go multitrack

@dudicoco
Copy link
Author

dudicoco commented Sep 2, 2021

thanks @daviddelucca!

@distorhead
Copy link
Member

@dudicoco @daviddelucca Check this out, you can now use helm built into werf, which uses kubedog, to automatically watch all flagger canaries found in the helm chart (werf installation):

. $(trdl use werf 1.2 alpha)
werf helm upgrade --install --wait myrelease CHART 

@sly-roar
Copy link

@daviddelucca Thank you for your contribution into the project and for your implementation of supporting canary releases!
I have been looking for a tool that would be able to provide the status of canary releases performed by Flagger for a long time.

However, I faced with strange behavior of it. For example, I have a test canary release named podinfo in test namespace:

% kubectl -n test get canary
NAME      STATUS      WEIGHT   LASTTRANSITIONTIME
podinfo   Succeeded   0        2021-09-17T11:46:06Z
% kubectl -n test describe canary podinfo
..........
Status:
  Canary Weight:  0
  Conditions:
    Last Transition Time:  2021-09-17T11:46:06Z
    Last Update Time:      2021-09-17T11:46:06Z
    Message:               Canary analysis completed successfully, promotion finished.
    Reason:                Succeeded
    Status:                True
    Type:                  Promoted
  Failed Checks:           0
  Iterations:              0
  Last Applied Spec:       5b7578bd5f
  Last Transition Time:    2021-09-17T11:46:06Z
  Phase:                   Succeeded
  Tracked Configs:
Events:
  Type    Reason  Age   From     Message
  ----    ------  ----  ----     -------
  Normal  Synced  33m   flagger  New revision detected! Scaling up podinfo.test
  Normal  Synced  32m   flagger  Starting canary analysis for podinfo.test
  Normal  Synced  32m   flagger  Pre-rollout check acceptance-test passed
  Normal  Synced  32m   flagger  Advance podinfo.test canary weight 25
  Normal  Synced  31m   flagger  Advance podinfo.test canary weight 50
  Normal  Synced  30m   flagger  Copying podinfo.test template spec to podinfo-primary.test
  Normal  Synced  29m   flagger  Routing all traffic to primary
  Normal  Synced  28m   flagger  Promotion completed! Scaling down podinfo.test

According to the output above canary releases with the help of Flagger are working properly. But kubedog v0.6.2, which should contain your solution already, shows the following:

$ echo '{"Canaries":[{"ResourceName":"podinfo","Namespace":"test"}]}' | kubedog multitrack
┌ Status progress
│ CANARY                            STATUS                WEIGHT                 LASTUPDATE                                
│ podinfo                                                 0                      0001-01-01 00:00:00 +0000 UTC             
└ Status progress

┌ Status progress
│ CANARY                            STATUS                WEIGHT                  LASTUPDATE                                
│ podinfo                                                 0                      0001-01-01 00:00:00 +0000 UTC             
└ Status progress

So kubedog does not show status of canary release and the time of its last update.
Besides that it does not monitor the status because it does not get changed when canary deploy is going.

Could you point me to a misconfiguration if it is somewhere or point the direction that I should dig in for figuring out the cause of the wrong behavior?
Thank you in advance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants