Metadata & manifest-builder refactoring #366

boozook · 2024-05-28T21:49:55Z

playdate-build:
- metadata & manifest-builder refactoring
  - add cargo-targets tables that overrides main manifest' fields
    - closes Manifest fields override for cargo dev targets (e.g. examples) #354
  - add proper support for custom manifest fields (Support custom fields in manifest #209)
- tests, improvements, minimal validation
- env resolver:
  - fix possible recursion
  - remove panic on missed var (should we panic here? #295, should we panic here? #296)
cargo-playdate:
- little cargo-playdate refactoring 🥵
- add sources by host cargo: (for Try to get rid of cargo build context #362, playdate-sys causes rust to always rebuild everything, even when nothing has changed #177)
  - use it instead of build-ctx by underlying cargo (in next part, somewhen in near future)
  - tests (in next part, somewhen in near future)
  - unit-graph
  - full metadata
- use new manifest builder from playdate-build

⚠️ Breaking changes:

[package.metadata.playdate.assets.options] not supported from now, use playdate.options.assets

…nifest' fields

* use new manifest builder from `playdate-build` pre-0.4 * little `cargo-playdate` refactoring 🥵

cargo/src/build/mod.rs

support/build/src/metadata/format.rs

support/build/src/metadata/source.rs

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

support/build/src/metadata/source.rs

paulstraw · 2024-05-30T00:50:19Z

@boozook sorry for another noobish question/issue, I'd appreciate a nudge in the right direction. I tried updating my project's dependencies to point at playdate = { git = "https://github.com/boozook/playdate.git", branch = "dev/meta-assets" }, as well as manually updating playdate-tool and playdate-bindgen:

cargo install --git https://github.com/boozook/playdate.git --branch dev/meta-assets playdate-tool
cargo install --git https://github.com/boozook/playdate.git --branch dev/meta-assets playdate-bindgen

However, I still seem to be missing some part of the update, as builds give the enclosed error.

» cargo playdate run --example 01_text_2d
    Updating git repository `https://github.com/boozook/playdate.git`
   Compiling proc-macro2 v1.0.82
   Compiling unicode-ident v1.0.12
   Compiling libc v0.2.154
   Compiling glob v0.3.1
   Compiling utf8parse v0.2.1
   Compiling cfg-if v1.0.0
   Compiling is_terminal_polyfill v1.70.0
   Compiling anstyle v1.0.7
   Compiling colorchoice v1.0.1
   Compiling anstyle-query v1.0.3
   Compiling anstyle-parse v0.2.4
   Compiling memchr v2.7.2
   Compiling option-ext v0.2.0
   Compiling clap_lex v0.7.0
   Compiling heck v0.5.0
   Compiling regex-syntax v0.8.3
   Compiling strsim v0.11.1
   Compiling anstream v0.6.14
   Compiling minimal-lexical v0.2.1
   Compiling libloading v0.8.3
   Compiling bindgen v0.69.4
   Compiling clang-sys v1.7.0
   Compiling clap_builder v4.5.2
   Compiling either v1.11.0
   Compiling nom v7.1.3
   Compiling semver v1.0.23
   Compiling lazycell v1.3.0
   Compiling itertools v0.12.1
   Compiling shlex v1.3.0
   Compiling bitflags v2.5.0
   Compiling lazy_static v1.4.0
   Compiling rustc-hash v1.1.0
   Compiling arrayvec v0.7.4
   Compiling quote v1.0.36
   Compiling dirs-sys v0.4.1
   Compiling syn v2.0.63
   Compiling dirs v5.0.1
   Compiling autocfg v1.3.0
   Compiling regex-automata v0.4.6
   Compiling playdate-build-utils v0.3.2 (https://github.com/boozook/playdate.git?branch=dev/meta-assets#f23f07bb)
   Compiling libm v0.2.8
   Compiling num-traits v0.2.19
   Compiling az v1.2.1
   Compiling version_check v0.9.4
   Compiling serde v1.0.201
   Compiling cexpr v0.6.0
   Compiling ahash v0.8.11
   Compiling byteorder v1.5.0
   Compiling regex v1.10.4
   Compiling serde_json v1.0.117
   Compiling micromath v2.1.0
   Compiling zerocopy v0.7.34
   Compiling once_cell v1.19.0
   Compiling float-cmp v0.9.0
   Compiling anyhow v1.0.83
   Compiling erased_set v0.8.0
   Compiling embedded-graphics-core v0.4.0
   Compiling spin v0.5.2
   Compiling cty v0.1.5
   Compiling itoa v1.0.11
   Compiling ryu v1.0.18
   Compiling allocator-api2 v0.2.18
   Compiling cstr_core v0.1.2
   Compiling euclid v0.22.9
   Compiling embedded-graphics v0.8.1
   Compiling hashbrown v0.14.5
   Compiling clap_derive v4.5.4
   Compiling serde_derive v1.0.201
   Compiling tinybmp v0.5.0
   Compiling clap v4.5.4
   Compiling playdate-bindgen-cfg v0.1.6 (https://github.com/boozook/playdate.git?branch=dev/meta-assets#f23f07bb)
   Compiling playdate-bindgen v0.1.10 (https://github.com/boozook/playdate.git?branch=dev/meta-assets#f23f07bb)
   Compiling playdate-sys v0.4.0 (https://github.com/boozook/playdate.git?branch=dev/meta-assets#f23f07bb)
   Compiling playdate-fs v0.2.7 (https://github.com/boozook/playdate.git?branch=dev/meta-assets#f23f07bb)
   Compiling playdate-color v0.2.3 (https://github.com/boozook/playdate.git?branch=dev/meta-assets#f23f07bb)
   Compiling playdate-system v0.3.11 (https://github.com/boozook/playdate.git?branch=dev/meta-assets#f23f07bb)
   Compiling playdate-display v0.3.5 (https://github.com/boozook/playdate.git?branch=dev/meta-assets#f23f07bb)
   Compiling playdate-scoreboards v0.1.1 (https://github.com/boozook/playdate.git?branch=dev/meta-assets#f23f07bb)
   Compiling playdate-lua v0.1.0 (https://github.com/boozook/playdate.git?branch=dev/meta-assets#f23f07bb)
   Compiling playdate-menu v0.2.3 (https://github.com/boozook/playdate.git?branch=dev/meta-assets#f23f07bb)
   Compiling playdate-controls v0.3.4 (https://github.com/boozook/playdate.git?branch=dev/meta-assets#f23f07bb)
   Compiling playdate-graphics v0.4.0 (https://github.com/boozook/playdate.git?branch=dev/meta-assets#f23f07bb)
   Compiling playdate-sound v0.4.0 (https://github.com/boozook/playdate.git?branch=dev/meta-assets#f23f07bb)
   Compiling playdate-sprite v0.2.10 (https://github.com/boozook/playdate.git?branch=dev/meta-assets#f23f07bb)
   Compiling playdate v0.2.0 (https://github.com/boozook/playdate.git?branch=dev/meta-assets#f23f07bb)
   Compiling pd3d v0.1.0 (/Users/paulstraw/dev/pd3d)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 6.59s
        Skip drop pd3d, crate 01_text_2d, can't build staticlib targeting simulator
   Compiling dylib of pd3d v0.1.0 (/Users/paulstraw/dev/pd3d)
    Finished dylib of pd3d
        Skip drop pd3d, crate 01_text_2d, can't build rlib targeting simulator
   Packaging (single-target) pd3d:01_text_2d (source: dylib, target: dylib)
error: [package.metadata.playdate] not found in the manifest file Cargo.toml

boozook · 2024-05-30T07:52:52Z

@paulstraw,

I tried updating my project's dependencies to point at playdate = { git = "https://github.com/boozook/playdate.git", branch = "dev/meta-assets" }, as well as manually updating playdate-tool and playdate-bindgen

That's not needed. Here is just not-ready (but seems nothing is broken) version of cargo-playdate cargo plugin in this PR. I'm still working on this.
So you're only interested in the cargo-playdate tool in this brunch.

I recommend use API-crates such as playdate, playdate-sys, playdate-sound,.. from the crates.io.

error: [package.metadata.playdate] not found in the manifest file Cargo.toml

Hmmmm, and now I'm not sure that nothing is broken 🤔
Thanks, I'll try to reproduce.

cargo/src/package/mod.rs

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

cargo/src/assets/plan.rs

boozook · 2024-05-30T13:44:50Z

@paulstraw I don't think that's possible 🤯.

However, I still seem to be missing some part of the update, as builds give the enclosed error.
   Packaging (single-target) pd3d:01_text_2d (source: dylib, target: dylib)
   error: [package.metadata.playdate] not found in the manifest file Cargo.toml

Are you sure that using this version of cargo-playdate?

I'm absolutely sure in it because I've deleted (here) that error in this pre1 version.

boozook · 2024-05-30T13:46:26Z

But anyway I found that asset-dependencies are not works properly (#372).

boozook · 2024-05-30T17:51:47Z

@paulstraw
Oh, seems I'v got it. May be you used one of previous commits where it still was. Fixed now.

paulstraw · 2024-05-31T03:42:01Z

Thanks for clarifying. After a cargo install --git https://github.com/boozook/playdate.git --branch dev/meta-assets cargo-playdate locally, this is working for me as expected. I tried both the [package.metadata.playdate.example.my_example] and [[package.metadata.playdate.example]] + target versions, and got the expected results both ways.

boozook · 2024-05-31T07:29:25Z

Good, thank you.
Also I suppose there could be [package.metadata.playdate.assets.options]? That is a breaking change, from now please use options.assets instead of assets.options.

boozook · 2024-06-02T11:16:54Z

6008903 is to fix KyleMayes/clang-sys#181

cargo/src/utils/cargo/meta_deps.rs

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

cargo/src/utils/cargo/meta_deps.rs

boozook added 4 commits May 27, 2024 23:25

metadata refactoring, add cargo-targets tables that overrides main ma…

dd8fb0e

…nifest' fields

tests, improvements, minimal validation

c5048c0

Fix possible recursion in env resolver, remove panic on missed var

34606e2

Add sources: unit-graph and full metadata by host cargo,

e8c02cc

* use new manifest builder from `playdate-build` pre-0.4 * little `cargo-playdate` refactoring 🥵

github-actions bot added docs Improvements or additions to documentation cargo About cargo or cargo-playdate labels May 28, 2024

boozook added assets Assets Build System build-utils manifest Package manifest -related things labels May 28, 2024

github-actions bot reviewed May 28, 2024

View reviewed changes

boozook and others added 2 commits May 29, 2024 01:53

env-resolver: remove panic on missed var (missed part)

c9ac85f

Apply clippy suggestions

9c72320

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

github-actions bot reviewed May 28, 2024

View reviewed changes

support/build/src/metadata/source.rs Outdated Show resolved Hide resolved

support/build/src/metadata/source.rs Outdated Show resolved Hide resolved

boozook mentioned this pull request May 28, 2024

Add Rusty Playdate tooling/libs rust-gamedev/rust-gamedev.github.io#1508

Merged

boozook added 2 commits May 30, 2024 11:08

add targets, improve validation for crate-level source

3b415c2

update deps and rust toolchain

eb710e5

simplification, get rid of Package dep in layout

a2227e3

boozook force-pushed the dev/meta-assets branch from f23f07b to a2227e3 Compare May 30, 2024 08:38

github-actions bot reviewed May 30, 2024

View reviewed changes

cargo/src/package/mod.rs Outdated Show resolved Hide resolved

cargo/src/package/mod.rs Outdated Show resolved Hide resolved

boozook and others added 2 commits May 30, 2024 12:56

Apply clippy suggestions

5da646a

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Merge branch 'main' into dev/meta-assets

59d54b1

boozook commented May 30, 2024

View reviewed changes

cargo/src/assets/plan.rs Outdated Show resolved Hide resolved

fix totally ignored assets

65bab4f

improve deserialisation of legacy and error for AssetsRules

149506a

fix missed feature-gate

f3f1cc2

fix AssetsRules again

9ffb277

boozook added 7 commits May 31, 2024 15:35

improve deserialisation and errors

14c9e4e

upd deps

650c62e

parametrize strings in metadata format

5f9405b

reduce usage of Package where only PackageId needed

7f496b5

set correct version

7d16e3c

allow use other strings

570684c

deps: downgraded clang-sys v1.8.2 -> v1.8.1

6008903

boozook added 2 commits June 4, 2024 15:08

improve CrateInfoSource

e96cbd4

add draft for meta-tree (unit-graph + metadata)

575ee8c

github-actions bot reviewed Jun 4, 2024

View reviewed changes

cargo/src/utils/cargo/meta_deps.rs Outdated Show resolved Hide resolved

cargo/src/utils/cargo/meta_deps.rs Outdated Show resolved Hide resolved

cargo/src/utils/cargo/meta_deps.rs Outdated Show resolved Hide resolved

apply clippy suggestions

9f1ce14

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

github-actions bot reviewed Jun 4, 2024

View reviewed changes

cargo/src/utils/cargo/meta_deps.rs Outdated Show resolved Hide resolved

cargo/src/utils/cargo/meta_deps.rs Outdated Show resolved Hide resolved

cargo/src/utils/cargo/meta_deps.rs Outdated Show resolved Hide resolved

boozook marked this pull request as ready for review June 4, 2024 16:11

github-actions bot approved these changes Jun 4, 2024

View reviewed changes

github-actions bot enabled auto-merge (squash) June 4, 2024 16:11

github-actions bot merged commit 88781e2 into main Jun 4, 2024
40 checks passed

boozook deleted the dev/meta-assets branch June 4, 2024 16:57

This was referenced Jun 8, 2024

Assets-builder refactoring #376

Merged

Assets build-plan merge method #24

Closed

This was linked to issues Jun 15, 2024

Support custom fields in manifest #209

Closed

should we panic here? #295

Closed

should we panic here? #296

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Metadata & manifest-builder refactoring #366

Metadata & manifest-builder refactoring #366

boozook commented May 28, 2024 •

edited

Loading

paulstraw commented May 30, 2024

boozook commented May 30, 2024

boozook commented May 30, 2024 •

edited

Loading

boozook commented May 30, 2024 •

edited

Loading

boozook commented May 30, 2024 •

edited

Loading

paulstraw commented May 31, 2024 •

edited

Loading

boozook commented May 31, 2024 •

edited

Loading

boozook commented Jun 2, 2024

Metadata & manifest-builder refactoring #366

Metadata & manifest-builder refactoring #366

Conversation

boozook commented May 28, 2024 • edited Loading

paulstraw commented May 30, 2024

boozook commented May 30, 2024

boozook commented May 30, 2024 • edited Loading

boozook commented May 30, 2024 • edited Loading

boozook commented May 30, 2024 • edited Loading

paulstraw commented May 31, 2024 • edited Loading

boozook commented May 31, 2024 • edited Loading

boozook commented Jun 2, 2024

boozook commented May 28, 2024 •

edited

Loading

boozook commented May 30, 2024 •

edited

Loading

boozook commented May 30, 2024 •

edited

Loading

boozook commented May 30, 2024 •

edited

Loading

paulstraw commented May 31, 2024 •

edited

Loading

boozook commented May 31, 2024 •

edited

Loading