Releases: nokia/danm
4.3.0 - Keeping up with upstream!
K8s. K8s always changes
The only constant in Kubernetes is change, and boy 1.21 had a LOT of them!
DANM 4.3 is a release dedicated to take care of these adjustments, and thus officially declares K8s 1.21 compability!
But as is customary by now, 4.3 also has a little cherry on top: Netwatcher Operator can now fully interwork with IPVLAN/MACVLAN NetworkAttachmentDefinitions. enabling the value added services it provides in any Multus environment!
Please find the detailed changelist below:
- #240 : DANM value-added features now correctly work with CNIs which do not create any kernel interfaces
Dev note: this practically means the userspace CNI, or the SR-IOV CNI in the rare cases the VF is bound to a non-bifurcated kernel driver - #245: CNI_ADD / DEL timeout events are now correctly logged in all cases
- #247: Netwatcher now supports the NAD API!
- #253: SR-IOV CNI dynamic integration needed fixing due to backward incompatible upstream API change
Dev note: DANM does not depend on the checkpoint Kubelet API anymore, so incompatibility issues related to this API should not happen in the future. Credit to Multus devs for the creative solution, based on the Pod Device monitoring API! - #255: PodIPs are now correctly recognized even when CRI is used instead of dockershim
Dev note: it is amazing how much everyone disregards the CNI spec - both the CNIs, and the CRIs. DANM is caught in the middle, and needs to actively massage the results into a format which is compatible with the additional massaging containerd will do on its own...
BEWARE: even more changes are on the horizon
Users should not forget about #242, and what it entails. With K8s 1.22 CRD v1beta API path support officially ends, which will force DANM to migrate to GA API.
Once that happens DANM API group must be changed to comply with the upstream policy mentioned in the ticket.
This means migration. Prepare yourself!
Releasing fixes on top of 4.2.0
DANM 4.2 - New feature release arrives with a bag of networking goodies!
New features and enhancements
DANM 4.2 is a full-fledged feature release, containing lots of enhancements long sought after by our community!
Thoroughest IPv6 support imaginable
IPv6 was always something we generally supported, albeit with some restrictions. With DANM 4.2 all such restrictions are things of the past, and we are confident our users will appreciate all the thought that went into creating the most seamless V6 network management experience available for Kubernetes.
These are:
- DANM IPAM now supports defining allocation pools for V6 subnets as well: #177 , #178
- IP allocation algorithm optimized to work in constant time even with big (~4 million) addresses: #179
- V6 usage is now enabled on interfaces even when "none" allocation scheme is used: #174
- V4 and V6 IP routes now work independently from each other, as intended: #205
- D(uplicate)A(ddress)D(etection) is automatically disabled on DANM managed V6 interfaces to avoid unnecessarily delaying, or failing startup of V6 using Pods: #215
- ndisc_notify is set to 1 on all DANM managed V6 interfaces to automatically force neighbour update in a V6 fabric: #222
Svcwatcher becomes production-grade
Svcwatcher implements a very handy functionality, but it was always more of a showcase of what use-cases suddenly become possible with the DANM platform, rather than a utility used in production. But after the following enhancements we now confidently propose taking it into production use as well:
- support added for V6 endpoints, endpoint update stability GREATLY increased: #187
- performance greatly improved by limiting processing to only the relevant API objects: #200
- svcwatcher created EndPoints are now fully compatible with latest Kubernetes DNS backends, enabling Pod name based DNS queries: #204
CI and deployment enhancements
New features are nice, but development and operator experience are equally important! Thanks to @carstenkoester and @eMGabriel DANM made great strides in this regard as well:
- build process was refactored and as a result became more convinient, and way faster: #196
- dependency management is now based on standard Go modules: #190
- DANM can now be auto-installed on existing Kubernetes environments: #212
Some call it "chaos testing", we call it... testing
DANM is deployed in production by various TelCo products, and as such is subject to rigorous E2E system testing processes. Unfortunetaly Kubernetes, and the CNI architecture in general is not entirely without shortcomings when facing node, cluster, or network outage events, but we do our best to maintain cluster state even when the unexpected happens by:
- retrying network access dependent execution paths to protect against network outages, and tying IP reservations and releases to the success of these operations: #188
- employ the fresh new error handling Informer API to protect against long-term network outages: #207
- automatically hunting for dangling resources during every CNI_ADD operation: #216
Dummy interfaces just become smarter
One of our recent hit feature was the creation of dummy interfaces in place of non-kernel managed DPDK devices to convey network allocations to unprivileged user space apps. However, due to the simplicity of the SR-IOV architecture users were still having trouble identifying which interface belongs to which device in complex setups, containing multiple allocations.
With the recent enhancement #214 DANM created dummy interfaces now convey their DPDK device's PCI ID, VLAN tag, and even their original MAC address!
DANM goes utils
From 4.2 and onward the DANM project now consists of two repositories!
Newly created danm-utils houses all the new, optional components / controllers / operators built on top of the DANM core platform, but being only loosely related to the core project. The main idea is to implement all higher-level, value adding features as optional add-ons, so you can individually decide if you would like to add their related complexity to your cluster.
So if you like DANM core, you might be interested in danm-utils too: https://github.com/nokia/danm-utils.
Together with this release we are also publicly announcing the generic availability of the first two such utilities:
https://github.com/nokia/danm-utils/releases/tag/cleaner-v1.0
If the outage scenarios described in this release notes were hampering you in the past, make sure to check out our optional Cleaner utility!
DANM 4.1: Focus on stability and generalization
Stability improvements
After every major release one should spend some time polishing the rough edges, and this is exactly what we have done in 4.1.
One good way of doing just that is to cover the entirety of the new implementation with Unit Tests:
confman: #104
admit1: #105
admit2: #114
admit3: #116
Another is to correct bugs found either during white box, or black box testing, such as:
- correcting regression related to "none" IP allocation scheme: #110
- solving a nasty race condition resulting in network resources possibly leaking: #123
- fixing a possible core in metacni: #119
- and another in confman: #159
- enforcing the maximum size of the usable IPv4 allocation pool to be inline with the current ETCD limitation: #137
- decreasing pollution by limiting the log output of the svcwatcher component: #134
Supporting new use-cases
Polishing is all nice and dandy, but what about new features?
I can happily say that release 4.1 also continues our great tradition of making more features generally available, and supported through the same network management API:
- DANM IPAM can be used with all backends, including static: #111
- DANM is now compatible with the latest CNI release, and spec: #121
- Netwatcher can now dynamically modify VLAN and VxLAN host interfaces when a network body is UPDATEd: #120
And our prize jewel for this release: DANM now supports SR-IOV-based DPDK use-cases with both Intel and Mellanox NICs; and as the first & only CNI it communicates cloud resource allocations to userspace applications through standard kernel APIs: #155
Known issue
We are aware, and apologize for the architectural shortcoming described in #144.
Extensive performance testing of the 4.0 architecture has shown us that Webhook is not capable of serving hooks related to UPDATE operations at the moment.
As a result, we advise users not to add this operation to the component's configuration.
But we believe in every failure lies an opportunity! This shortcoming spurred us to improve our core architecture, which will open the door for plethora of new, and exciting use-cases!
Stay tuned ;)
DANM 4.0: Production grade multi-tenant network management arrives to Kubernetes!
Production grade network management
We are proud to announce that DANM 4.0 is finally released!
With it comes a feature which we feel is truly unique in the whole Kubernetes ecosystem: a production grade network management method drops today, capable of serving administrators, and end users alike.
DANM, as all CNIs was originally designed to serve the needs of a single tenant cloud, but as we have moved towards the production deployment of a multi tenant edge cloud solution, we faced a serious challenge: cloud users, and cloud administrators have vastly different needs an responsibilities, and no network management solution in Kubernetes is truly prepared to serve both!
In order to tackle the challenge we needed to take a long hard look at our APIs, and decided that some major improvements are required.
We have realized that the only way to reliable serve their differents needs at the same time is to create tailored APIs. As a result, TenantNetworks and ClusterNetworks were born!
It does not mean DanmNets are going away, but the new APIs allow you to do something which was not previously possible: allocate physical network resources to certain users in your cloud, and restrict them to use only what you want them to use!
You know, treating networks as the cloud resources they are :)
Got your attention? For more information on production grade network management mode, please take a look at our improved README!
New major component - Webhook!
As we continue our quest to streamline network management user experience, it was time to introduce a new component to DANM: the Webhook!
Besides doing the heavy lifting of production grade network management, Webhook also validates every and all DANM API objects before they are admitted into the cluster.
It doesn't matter if you use the existing lightweight, or the newly introduced production grade network management method: no more shall you suffer the consequences of syntactically, or semantically invalid networks!
DANM meets Akraino
Do you also save the tastiest bits to be the last, or is it only just me? :)
We are very proud that DANM became the official network management solution of a major, open source edge cloud infrastructure - the Akraino Radio Edge Cloud blueprint:
https://wiki.akraino.org/pages/viewpage.action?pageId=6128402
We are immensely grateful to be recognized as the best option when it comes to managing networks in a TelCo container cloud, and continues to strive to maintain this standard.
Want to help us in our quest by providing your feedback, use-cases, or just hang out with us and pick our brain? You are one Slack join away of being able to do so!
https://join.slack.com/t/danmws/shared_invite/enQtNjc2MzA3MjU0NzEwLWVkZDQ3ZDA1ZTU2YTM2MjI3YzJmYWFhZjUzZjNmNTcwMTUwYmI0NDhhNzQzMzA3NWRkNmMzNTIxMTI0MDlmYjk
DANM 3.3.0 - Have you heard the word?
It is user experience!
Okay, it is actually two words.
Anyway, we sincerely believe that DANM is da bomb when it comes to network management in Kubernetes, but at the same time it really needed some quality of life improvements.
So that's exactly what we did with DANM 3.3.0: we have a spent a month or two testing critical code paths, weeding out bugs, and tweaking the API to make your network management experience as seamless as possible!
Are you not entertained? How about then making all dynamic backends -IPVLAN, SR-IOV, MACVLAN- IPv6, AND dual-stack compliant?
Changelog
Project quality improvements
- The first one is for you, dear contributors: restructured the code to mirror the quasi-official Golang project layout (#58)
- Quality starts at automatically executing all tests before the merge of every PR, something which we now do!
- 100% UT coverage, and the squashing of all major bugs was long overdue for the ipam package (#60)
- Syncher package did not have that many bugs, but 100% coverage is still nice to have (#59)
- How about the same for cnidel package? Sounds important, right? (#71)
Usability improvements
- You can now configure via NetworkID which CNI config file to use when delegating to a static backend(#63)
- "container_prefix" now means what its name suggests, making it impossible for the user to accidentally misconfigure the name of their interfaces(#78)
- Including the infamous "eth0" - we know you use DANM with Kubernetes, so believe us: you need your first NIC to be always called that!
- Do you want control over which interface naming scheme DANM should use, or where do the CNI config files reside? Say no more!(#83)
- Weave was not working very well together with DANM, but that's in the past (#67)
- Fixed the hard restriction that DanmNet names, and their NetworkIDs need to match, which was essentially breaking a couple of features(#80, #81)
IPv6 and dual-stack improvements
Brought to you by @TothFerenc!
- DANM now takes care of all your IPv6 related kernel setting needs (#69, #76)
- Dual-stack support is now extended to all dynamic delegates (#74)
- Unnecessary gARP replies are not sent anymore for IPv6 interfaces (#79)
- Pure IPv6 support is now extended to all dynamic delegates (#85)
What's next - DANM 4.0.0 looming on the horizon
Unless something requires hotfixing, we won't release any new DANM Version 3 content from now on.
The reason behind this decision is that we are planning to completely renew DANM network management interfaces - and hopefully blow your mind in the process!
Once this super secret awesomeness has been fully implemented, we will turn the page on DANM 3.0, and will call it "DANM 4.0". (Totally unrelated: we do accept marketing and branding advice from basically anyone).
Want to know more? You never know, we might spoil a thing or two if you ask :)
DANM 3.2.0 - Device Plugin support arrives to DANM!
You asked for it, and now Device Plugins are officially supported!
We are happy to announce that from release 3.2.0 onwards DANM supports inter-working with Device Plugin-based network management!
By far this was our community's most sought-after feature, and as always, we are happy to listen to your inputs :)
The DanmNet schema is expanded with a new parameter, called device_pool . This generic purpose attribute is used to connect the logical representation of a network to a Kubernetes managed Device pool.
To showcase how the new framework can be used in a real-life use-case, we have also upgraded the dynamic-level integration of SR-IOV CNI to make a use of this attribute.
Virtual Functions are managed by the SR-IOV Device Plugin in the new architecture, and DANM will only take care of connecting the Pod to the network devices assigned by the Device Manager component of Kubelet.
This approach makes SR-IOV virtual functions visible to the Kubernetes scheduler, and will also enable us to make sure the CPU socket of all the Pods' compute resources are aligned - once Topology Manager feature hits Kubernetes main branch (1.14, fingers crossed).
For more information, please refer to the project's README, or check-out our new example manifests!
Important disclaimer: backward compatibility to older versions of the SR-IOV CNI project is not maintained. DANM only supports the dynamic-level integration to the latest and greatest version from both the CNI, and the Device Plugin component.
DANM 3.1.1 - Hotfix release for 3.1.0
Release 3.1.1 is a hotfix for release 3.1.0
A week ago the Kubernetes code generator project decided to cease generating versionless interfaces into their REST clients.
Unfortunately DANM code used this interface in two places. As a result, DANM stopped compiling.
While it is true that nobody should use a versionless interface of a REST client, this backward incompatible change was still not very nice IMHO :)
In any case, users are encouraged to use this release instead of 3.1.0.
We will store the generated code within the repo in the future to avoid such breaks happening again, and will also fix the version of the code generator dependency into our SCM pipeline.
DANM V3.1.0
Release 3.1.0 is a major feature release of DANM
Publishing the project on GitHub was definitely a major boon. We have received lots of awesome community ideas, while also had some new ones on our own.
I sincerely would like to thank everyone who took time to contribute code, ideas, or just simply let us know about their use-cases!
Changelog
We rely on you to make DANM better, and I think the following changelog demonstrates that we are on the right path:
- We got a minimalistic, but quite awesome logo from @libesz ! (#5)
- RBAC support (#29)
- Possibility to use in-cluster configuration with all our components (#6)
- IP routes and Policy-based IP routes can now be provisioned regardless of NetworkType (#13)
- MACVLAN joins the list of 100% DanmNet API managed CNI plugins!
- we got rid of our nasty Docker dependency, DANM now works with all container runtimes (#26)
- An API-driven hybrid interface naming scheme was introduced to give complete control in the hands of netadmins over how they want their container interfaces to be named (#27)
- DANM now supports namespace-wide default networks (#34), if you hate micromanaging your network connections
- Looking out for RedHat folks by adding buildah and prodman support to our automated build system (#38)
What's next
We have loads of exciting ideas already in our pipelines! But, of course we are constantly on the lookout for further suggestions :)
But to give you a sneak-peak into out current activities, our short-term focus is on providing better support for IPv6 both with documentation and functionality, creating automated cluster-upgrade playbooks for zero effort DANM integration, integrating SR-IOV device plugin, and making the services of our special IPAM plugin usable in all kinds of environments!
First DANM open source version released!
First open source release. Numbering reflects that the module had num… …erous Nokia-internal releases over the years.