Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TF Plan to ACI Payload Converter (DCNE-164) #1276

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -288,4 +288,16 @@ To compile the provider, run `make build`. This will build the provider with san

<strong>NOTE:</strong> Currently only resource properties supports the reflecting manual changes made in CISCO ACI. Manual changes to relationship is not taken care by the provider.
harismalk marked this conversation as resolved.
Show resolved Hide resolved

### Payload Generation

To export a Terraform Plan as an ACI Payload:

1. Navigate to conversion directory
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to run the code without changing the working directory to conversion

$cd cmd/conversion

2. Create your desired configuration in main.tf

3. Run:
$ go run main.go

4. Payload will be written to payload.json
harismalk marked this conversation as resolved.
Show resolved Hide resolved
330 changes: 330 additions & 0 deletions cmd/conversion/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,330 @@
package main

import (
"encoding/json"
"fmt"
"log"
"os"
"os/exec"
"strings"

"github.com/CiscoDevNet/terraform-provider-aci/v2/convert_funcs"
"github.com/CiscoDevNet/terraform-provider-aci/v2/dict"
)

type Plan struct {
PlannedValues struct {
RootModule struct {
Resources []Resource `json:"resources"`
} `json:"root_module"`
} `json:"planned_values"`
Changes []Change `json:"resource_changes"`
}

type Resource struct {
Type string `json:"type"`
Name string `json:"name"`
Values map[string]interface{} `json:"values"`
}

type Change struct {
Type string `json:"type"`
Change struct {
Actions []string `json:"actions"`
Before map[string]interface{} `json:"before"`
} `json:"change"`
}

func runTerraform() (string, error) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it be possible to have this as a method on the Plan struct? where the method is called something like getPlanOutput

planBin := "plan.bin"
planJSON := "plan.json"
Comment on lines +45 to +46
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we pass these to the function as arguments?


if err := exec.Command("terraform", "plan", "-out="+planBin).Run(); err != nil {
return "", fmt.Errorf("failed to run terraform plan: %w", err)
}

output, err := os.Create(planJSON)
if err != nil {
return "", fmt.Errorf("failed to create json file: %w", err)
}
defer output.Close()

cmdShow := exec.Command("terraform", "show", "-json", planBin)
cmdShow.Stdout = output
if err := cmdShow.Run(); err != nil {
return "", fmt.Errorf("failed to run terraform show: %w", err)
}

return planJSON, nil
}

func readPlan(jsonFile string) (Plan, error) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a reason why you separated this from the runTerraform() ?

var plan Plan
data, err := os.ReadFile(jsonFile)
if err != nil {
return plan, fmt.Errorf("failed to read input file: %w", err)
}

if err := json.Unmarshal(data, &plan); err != nil {
Comment on lines +76 to +82
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
var plan Plan
data, err := os.ReadFile(jsonFile)
if err != nil {
return plan, fmt.Errorf("failed to read input file: %w", err)
}
if err := json.Unmarshal(data, &plan); err != nil {
data, err := os.ReadFile(jsonFile)
if err != nil {
return plan, fmt.Errorf("failed to read input file: %w", err)
}
var plan Plan
if err := json.Unmarshal(data, &plan); err != nil {

return plan, fmt.Errorf("failed to parse input file: %w", err)
}

return plan, nil
}

func writeToFile(outputFile string, data map[string]interface{}) error {
outputData, err := json.MarshalIndent(data, "", " ")
if err != nil {
return fmt.Errorf("failed to convert data to JSON: %w", err)
}

if err := os.WriteFile(outputFile, outputData, 0644); err != nil {
return fmt.Errorf("failed to write output file: %w", err)
}

return nil
}

func createPayload(resourceType string, values map[string]interface{}, status string) map[string]interface{} {
if createFunc, exists := convert_funcs.ResourceMap[resourceType]; exists {
payload := createFunc(values, status)
return payload
}
return nil
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we log something or provide some alternative action when the resourceType does not exist?

}

func createPayloadList(plan Plan) []map[string]interface{} {
var data []map[string]interface{}

for _, change := range plan.Changes {
if len(change.Change.Actions) > 0 && change.Change.Actions[0] == "delete" {
Comment on lines +116 to +117
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if I understand correctly only deletes are withdrawn from plan.Changes, and then for the other payloads you can use plannedValues?

payload := createPayload(change.Type, change.Change.Before, "deleted")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we create a more simplified function for delete, since it typically only requires the dn and status deleted? Then the status argument is also not required in the function call.

if payload != nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we raise an error on payload nil? This goes hand in hand with the log comment made before.

data = append(data, payload)
}
}
}

for _, resource := range plan.PlannedValues.RootModule.Resources {
payload := createPayload(resource.Type, resource.Values, "")
if payload != nil {
for _, value := range payload {
if obj, ok := value.(map[string]interface{}); ok {
if attributes, ok := obj["attributes"].(map[string]interface{}); ok {
if parentDn, ok := resource.Values["parent_dn"].(string); ok && parentDn != "" {
attributes["parent_dn"] = parentDn
}
}
}
}
Comment on lines +128 to +136
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is teh purpose of this logic?

data = append(data, payload)
}
}

return data
}

type TreeNode struct {
Attributes map[string]interface{} `json:"attributes,omitempty"`
Children map[string]*TreeNode `json:"children,omitempty"`
ClassName string `json:"-"`
}

func NewTreeNode(className string, attributes map[string]interface{}) *TreeNode {
return &TreeNode{
ClassName: className,
Attributes: attributes,
Children: make(map[string]*TreeNode),
}
}

func constructTree(resources []map[string]interface{}) map[string]interface{} {
rootNode := NewTreeNode("uni", map[string]interface{}{"dn": "uni"})

dnMap := make(map[string]*TreeNode)
dnMap["uni"] = rootNode

for _, resourceList := range resources {
for resourceType, resourceData := range resourceList {
resourceAttributes := resourceData.(map[string]interface{})
attributes := safeMapInterface(resourceAttributes, "attributes")
dn := safeString(attributes, "dn")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we continue loop when attributes or dn is nil?


var children []map[string]interface{}
if rawChildren, ok := resourceAttributes["children"].([]interface{}); ok {
for _, child := range rawChildren {
if childMap, ok := child.(map[string]interface{}); ok {
children = append(children, childMap)
}
}
}

buildParentPath(dnMap, rootNode, resourceType, dn, attributes, children)
}
}

return map[string]interface{}{rootNode.ClassName: exportTree(rootNode)}
}

func buildParentPath(dnMap map[string]*TreeNode, rootNode *TreeNode, resourceType, dn string, attributes map[string]interface{}, children []map[string]interface{}) {
if dn == "" && resourceType == "" {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when would resourceType be ""

return
}

cursor := rootNode
if dn != "" {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this conditional seems redundant when above you exit the function already when it is ""

cursor = traverseOrCreatePath(dnMap, rootNode, resourceType, dn)
}

var leafNode *TreeNode
if existingLeafNode, exists := dnMap[dn]; exists {

for key, value := range attributes {
existingLeafNode.Attributes[key] = value
}
leafNode = existingLeafNode
} else {
leafNode = NewTreeNode(resourceType, attributes)
cursor.Children[dn] = leafNode
dnMap[dn] = leafNode
}

for _, child := range children {
for childClassName, childData := range child {
childAttributes := safeMapInterface(childData.(map[string]interface{}), "attributes")
childDn := safeString(childAttributes, "dn")

childKey := childDn
if childDn == "" {
childKey = generateUniqueKeyForNonDnNode(childClassName, childAttributes)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is a non dn node? a rn without identifier?

}

if _, exists := leafNode.Children[childKey]; !exists {
childNode := NewTreeNode(childClassName, childAttributes)
leafNode.Children[childKey] = childNode
dnMap[childKey] = childNode
}

if grandChildren, ok := childData.(map[string]interface{})["children"].([]interface{}); ok {
for _, grandchild := range grandChildren {
if grandchildMap, ok := grandchild.(map[string]interface{}); ok {
buildParentPath(dnMap, leafNode, childClassName, childDn, safeMapInterface(grandchildMap, "attributes"), []map[string]interface{}{grandchildMap})
}
}
}
}
}
}

func generateUniqueKeyForNonDnNode(resourceType string, attributes map[string]interface{}) string {
return fmt.Sprintf("%s-%v", resourceType, attributes["name"])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if attributes does not contain name?

}

func traverseOrCreatePath(dnMap map[string]*TreeNode, rootNode *TreeNode, resourceType, dn string) *TreeNode {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the function actually doing? Bit confused on what it is you are trying to achieve here. For more complex functions it might be an idea to add some comments in the code to make it easier to understand.

pathSegments := strings.Split(dn, "/")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is likely not enough to handle complex dn patterns. There's a function in our provider that could be utilized here. Let me look for it.

cursor := rootNode

classNames := parseClassNames(pathSegments, resourceType)

for i := 1; i < len(pathSegments); i++ {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not start loop with 0 then classnames can just be indexed with i?

className := classNames[i-1]
currentDn := strings.Join(pathSegments[:i+1], "/")

if existingNode, exists := dnMap[currentDn]; exists {
cursor = existingNode
} else {
newNode := NewTreeNode(className, map[string]interface{}{
"dn": currentDn,
})
cursor.Children[currentDn] = newNode
cursor = newNode
dnMap[currentDn] = newNode
}
}

return cursor
}

func parseClassNames(pathSegments []string, resourceType string) []string {
classNames := []string{resourceType}
for i := len(pathSegments) - 2; i >= 0; i-- {
prefix := strings.Split(pathSegments[i], "-")[0]
if pathSegments[i] == "uni" {
break
}
className := dict.GetDnToAciClassMap(classNames[len(classNames)-1], prefix)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see comment in the function itself

classNames = append(classNames, className)
}

for i, j := 0, len(classNames)-1; i < j; i, j = i+1, j-1 {
classNames[i], classNames[j] = classNames[j], classNames[i]
}
return classNames

}

func exportTree(node *TreeNode) map[string]interface{} {
if node == nil {
return nil
}

result := map[string]interface{}{
"attributes": node.Attributes,
}

if len(node.Children) > 0 {
children := []map[string]interface{}{}
for _, child := range node.Children {
children = append(children, map[string]interface{}{
child.ClassName: exportTree(child),
})
}
result["children"] = children
}

return result
}

func safeMapInterface(data map[string]interface{}, key string) map[string]interface{} {
if value, ok := data[key].(map[string]interface{}); ok {
return value
}
return nil
}

func safeString(data map[string]interface{}, key string) string {
if value, ok := data[key].(string); ok {
return value
}
return ""
}

func main() {
if len(os.Args) != 1 {
fmt.Println("Usage: no arguments needed")
harismalk marked this conversation as resolved.
Show resolved Hide resolved
os.Exit(1)
}

outputFile := "payload.json"

planJSON, err := runTerraform()
if err != nil {
log.Fatalf("Error running Terraform: %v", err)
}

plan, err := readPlan(planJSON)
if err != nil {
log.Fatalf("Error reading plan: %v", err)
}

payloadList := createPayloadList(plan)

aciTree := constructTree(payloadList)

err = writeToFile(outputFile, aciTree)
if err != nil {
log.Fatalf("Error writing output file: %v", err)
}

fmt.Printf("ACI Payload written to %s\n", outputFile)
}
Loading
Loading