Pulumi Native Provider Boilerplate

This repository is a boilerplate showing how to create a native Pulumi provider.

Background

This repository is part of the guide for authoring and publishing a Pulumi Package.

Learn about the concepts behind Pulumi Packages.

Follow this link to see an architecture diagram for Pulumi.

A Pulumi Resource Provider:

• is a gRPC server which allows for the Pulumi engine to create resources in a specific cloud
• holds the lifecycle logic for these cloud resources
• holds a pulumi JSON schema that describes the provider
• provides language-specific SDKs so resources can be created in whichever language you prefer

When we speak of a "native" provider, we mean that all implementation is native to Pulumi, as opposed to Terraform based providers.

Authoring a Pulumi Native Provider

The following instructions assume that the provider is written for the Pulumi organisation. In the future, we will add instruction for providers published and maintained by the Pulumi community, referred to as "third-party" providers.

This boilerplate creates a working Pulumi-owned provider named xyz. It implements a random number generator that you can build and test out for yourself and then replace the Random code with code specific to your provider.

Prerequisites

Ensure the following tools are installed and present in your $PATH:

• pulumictl
• Go 1.17 or 1.latest
• NodeJS 14.x. We recommend using nvm to manage NodeJS installations.
• Yarn
• TypeScript
• Python (called as python3). For recent versions of MacOS, the system-installed version is fine.
• .NET

Creating and Initializing the Repository

Pulumi offers this repository as a GitHub template repository for convenience. From this repository:

1. Click "Use this template".
2. Set the following options:
   • Owner: pulumi
   • Repository name: pulumi-xyz-native (replace "xyz" with the name of your provider)
   • Description: Pulumi provider for xyz
   • Repository type: Public
3. Clone the generated repository.

From the templated repository:

1. Search-replace xyz with the name of your desired provider.

Build the provider and install the plugin

$ make build install

This will:

1. Create the SDK codegen binary and place it in a ./bin folder (gitignored)
2. Create the provider binary and place it in the ./bin folder (gitignored)
3. Generate the dotnet, Go, Node, and Python SDKs and place them in the ./sdk folder
4. Install the provider on your machine.

Test against the example

$ cd examples/simple
$ yarn link @pulumi/xyz
$ yarn install
$ pulumi stack init test
$ pulumi up

Now that you have completed all of the above steps, you have a working provider that generates a random string for you.

A brief repository overview

You now have:

1. A provider/ folder containing the building and implementation logic
   1. cmd/
      1. pulumi-gen-xyz/ - generates language SDKs from the schema
      2. pulumi-resource-xyz/ - holds the package schema, injects the package version, and starts the gRPC server
   2. pkg
      1. provider - holds the gRPC methods (and for now, the sample implementation logic) required by the Pulumi engine
      2. version - semver package to be consumed by build processes
2. deployment-templates - a set of files to help you around deployment and publication
3. sdk - holds the generated code libraries created by pulumi-gen-xyz/main.go
4. examples a folder of Pulumi programs to try locally and/or use in CI.
5. A Makefile and this README.

Writing the schema

The JSON schema file is used by pulumi-gen-xyz to create language-specific SDKs. It is, therefore, a central requirement for any resource provider. Provider schemas can be handwritten, or alternatively machine-generated by combining API specification with pulumi-specific logic.

When writing the schema by hand, it is helpful to associate the JSON schema in your IDE for completion or Intellisense features to work:

• Visual Studio Code: the easiest option is to map the schema file in your User Settings which enables it for all your provider projects:

      "json.schemas": [
        {
            "fileMatch": [
                "/provider/cmd/pulumi-*/schema.json"
            ],
            "url": "https://raw.githubusercontent.com/pulumi/pulumi/master/pkg/codegen/schema/pulumi.json"
        }
    ]

This repository provides the xyz example schema to get you started.
The AWS Native Provider schema provides a much larger example.
Refer to the package schema documentation for additional details when writing the schema.

Implementing the gRPC methods

Once you have a schema that describes all the resources and metadata for your provider, you will need to implement the desired gRPC methods. You will find a mostly blank implementation of these in pkg/provider/provider.go. Note that these methods do not link 1:1 to the Pulumi CLI commands.

Basic Functionality

The struct and creation of the provider are implemented already:

// provider/pkg/provider.go
type xyzProvider struct {
	host    *provider.HostClient
	name    string
	version string
	schema  []byte
}

func makeProvider(host *provider.HostClient, name, version string, pulumiSchema []byte) (pulumirpc.ResourceProviderServer, error) {
   // Return the new provider
   return &xyzProvider{
   host:    host,
   name:    name,
   version: version,
   schema:  pulumiSchema,
   }, nil
}

You need to provide the following methods:

1. Check - validates resource Inputs
2. Diff - calculates the differences between the actual and the desired state of a resource
3. Create - creates a new instance of a resource from an Input
4. Update - updates a resource in-place (i.e. without deleting/recreating)
5. Read - reads current inputs and state for a resource
6. Delete - deletes a resource and its corresponding state

Resource lifecycle methods are documented here.

The following methods are necessary for every provider and are already implemented:

1. GetPluginInfo - returns generic information about this plugin, like its version
2. GetSchema - returns the Pulumi schema to the provider

Additional Methods

The resource provider service includes a few more gRPC methods that you may need to implement and can read more about.

Build Examples

Create an example program using the resources defined in your provider, and place it in the examples/ folder.

You can now repeat the steps for build, install, and test.

Documentation

Please follow this guide to add documentation to your provider.

Configuring CI and releases

1. Follow the instructions laid out in the deployment templates.

References

Other resources for learning about the Pulumi resource model:

• Pulumi Kubernetes provider
• Pulumi Terraform Remote State provider
• Dynamic Providers