Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce AssetOrArchive to support the SDK's Asset type which can be both #242

Merged
merged 20 commits into from
Jun 13, 2024

Conversation

thomas11
Copy link
Contributor

@thomas11 thomas11 commented Jun 4, 2024

Resolves #237.

This PR addresses an issue with the design of the assets and archives support in Pulumi core. Quoting #237:

When defining a property of type resource.Asset, the generated schema will contain a $ref of pulumi.json#/Asset. However, in the SDKs, an "asset" is represented by pulumi.AssetOrArchive because an archive is currently always a valid asset.

Therefore, when deserializing the PropertyValue for the provider to use, it's valid for the user to have specified one of the Archive types which then isn't deserializable into the resource.Asset field within the provider.

To work around this, we define a new type types.AssetOrArchive here in pulumi-go-provider. In the schema, it's represented as a standard pulumi.json#/Asset for interoperability. An incoming core Asset is translated into a types.AssetOrArchive, in Decode, with either Asset or Archive populated. Vice versa, Encode pulls the asset/archive out of AssetOrArchive.

Notes

  • The decode and encode implementations are bespoke and there might be opportunities to move it to a new, more generic extension point.

@thomas11 thomas11 force-pushed the tkappler/assetorarchive branch from 952f307 to eb6f04a Compare June 4, 2024 12:18
@thomas11 thomas11 requested review from iwahbe and a team June 4, 2024 14:09
Copy link
Member

@iwahbe iwahbe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The serde stuff looks good to me. This is the kind of place that I'd really appreciate integration tests to lock in good behavior.

types/asset.go Outdated Show resolved Hide resolved
infer/internal/ende/ende.go Outdated Show resolved Hide resolved
@thomas11 thomas11 requested a review from a team June 7, 2024 07:38
infer/types/asset.go Outdated Show resolved Hide resolved
infer/internal/ende/ende.go Outdated Show resolved Hide resolved
@thomas11 thomas11 requested review from a team and danielrbradley June 10, 2024 16:58
@thomas11 thomas11 marked this pull request as ready for review June 10, 2024 16:59
@thomas11 thomas11 force-pushed the tkappler/assetorarchive branch from 8e03a4a to 26a5278 Compare June 10, 2024 17:08
infer/resource.go Outdated Show resolved Hide resolved
infer/internal/ende/ende.go Outdated Show resolved Hide resolved
infer/internal/ende/ende.go Outdated Show resolved Hide resolved
infer/types/asset.go Show resolved Hide resolved
infer/internal/ende/ende.go Outdated Show resolved Hide resolved
infer/types/asset.go Show resolved Hide resolved
Copy link
Member

@danielrbradley danielrbradley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty close. I'm happy the implementation looks correct, just a few last bits of polish.

infer/internal/ende/ende.go Outdated Show resolved Hide resolved
Comment on lines 370 to 371
panic(`Encountered both an asset and an archive in the same AssetOrArchive. This
should never happen. Please file an issue at https://github.com/pulumi/pulumi-go-provider/issues.`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is a panic the best course of action here? I know we can't have both in one right now, but it's not beyond the realms of possibility that platform created some new type which is both an asset archive for some reason. I'm guessing we don't have any kind of spec saying is can't have both sigs, just it's not currently possible?

Perhaps we should just continue here as it doesn't break our model to have both given how our union type is moddled?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that this is when we are serializing. We need to serialize AssetOrArchive into either an asset or an archive, and we can't do that. We can't compress struct { Asset, Archive } into an Asset or Archive without losing information. It is always a provider bug and right now panicing is a reasonable option. It would be better to return an error, but it's not clear how.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Platform will not create a new type that has our sigs (we assume that sigs won't conflict by accident).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was the other way round - when we're parsing - we're adding a panic if we recieve something which contains both asset or archive information. My suggestion here would be to just check if it's an asset first, then check if it's an archive. The code below would already do that if the panic removed, if I read it correctly, though it prefers archive over asset.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should panic on ambiguity, as it is always a provider or framework bug. The user will never see the ambiguity and the engine can only pass ambiguous information if it copies the sigs we use, which it should not.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree if it's user or framework error, but this data is coming from the engine – for which we don't have a super-good specification of what they might send, so I'd consider being defensive over making contract assertions. But perhaps it's reasonable here to fail hard because it would be a weird change engine side that hopefully they wouldn't make because it's likely not backward compatible (not that we define backward compatibility in our protocols either 😞).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no remotely backwards compatible change (beyond copying the sigs we created so they wouldn't be copied, on purpose or by accident) that would allow this to be an engine bug. It will always be a provider author or framework bug, and it should panic to allow the responsible party to fix it, instead of doing the wrong thing for end users.

infer/internal/ende/ende.go Show resolved Hide resolved
infer/internal/ende/ende.go Outdated Show resolved Hide resolved
infer/internal/ende/ende.go Outdated Show resolved Hide resolved
infer/schema.go Show resolved Hide resolved
@thomas11 thomas11 force-pushed the tkappler/assetorarchive branch from fbd137e to f50c880 Compare June 11, 2024 18:43
@thomas11 thomas11 force-pushed the tkappler/assetorarchive branch from 84deb60 to a5ed00b Compare June 12, 2024 04:35
Copy link
Member

@danielrbradley danielrbradley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think this is good to go.

Discussions are just asides at this point.

This reverts commit f50c880.

The problem, as implemented, is that it'll warn if the Asset only
occurs as output, like in local.Command.
Copy link
Member

@iwahbe iwahbe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. There are a couple simple changes to make before merging, but it mostly looks good to go.

infer/internal/ende/ende.go Outdated Show resolved Hide resolved
infer/schema.go Show resolved Hide resolved
infer/types/asset.go Outdated Show resolved Hide resolved
infer/types/asset.go Outdated Show resolved Hide resolved
@thomas11 thomas11 enabled auto-merge (squash) June 13, 2024 10:19
@thomas11 thomas11 merged commit 7093fe0 into main Jun 13, 2024
6 checks passed
@thomas11 thomas11 deleted the tkappler/assetorarchive branch June 13, 2024 10:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Asset properties can be Archives at runtime
3 participants