KHR_texture_procedurals Draft Proposal #2381

kwokcb · 2024-04-03T15:12:11Z

Draft Proposal for `KHR_texture_procedurals`

Specification proposal ready for initial feedback.
Draft JSON Schema is still progress.
Example assets will be forthcoming in a glTF Sample Assets branch.

* Start document consolidation. * Transferred doc.

- Update checkerboard example with USD equiivalent and mermaid graph. - Add USD/ / MaterialX / glTF mapping

…o spec README.

extensions/2.0/Khronos/KHR_texture_procedurals/README.md

1. KHR_texture_procedural 2. EXT_texture_procedural_mx_1_39

…Add in mapping section for texture_infos.

kwokcb · 2024-06-12T15:16:43Z

extensions/2.0/Vendor/EXT_texture_procedurals_mx_1_39/README.md

+
+## Handling MaterialX glTF PBR Nodes
+
+The glTF PBR node specified within MaterialX exposes a number of input ports which may not map 1:1 with the glTF PBR material


Currently there is no way to do a proper mapping for metallic and roughness. Should a separate mapping for metallic and roughness be proposed. Should the MaterialX glTF PBR node be updated ?

rsahlin · 2024-07-09T08:09:47Z

extensions/2.0/Khronos/KHR_texture_procedurals/README.md

+
+    More specifically __only__ nodes defined in the MaterialX specification are supported. This ensures that the procedural graphs can be easily converted to MaterialX and USDShade graphs.
+
+    For the first version of this extension nodes which are used to define shading models are not allowed. Please refer to the [resources](#resources) section for links to supported  MaterialX node definitions.


For the first version of this extension....
I do not understand this statement - this extension concerns creating images from operators/functions, instead of stored pixel values.
Where do shading models fit into this?

I propose to remove this statement.

rsahlin · 2024-07-09T08:11:23Z

extensions/2.0/Khronos/KHR_texture_procedurals/README.md

+
+### Motivation
+
+Textures represented as procedural graphs provides a way to extend the capabilities of glTF materials beyond what is possible with traditional texture maps. Key objectives includes:


Please reword to:
Images represented as procedural operators or functions provides a way to extend the capabilities of glTF textures beyond what is possible with bitmap based images.

rsahlin · 2024-07-09T08:27:58Z

extensions/2.0/Khronos/KHR_texture_procedurals/README.md

+
+1. **Interoperability**: Adhere to an industry a standard node schema (MaterialX) with a runtime friendly JSON representation. 
+
+    More specifically __only__ nodes defined in the MaterialX specification are supported. This ensures that the procedural graphs can be easily converted to MaterialX and USDShade graphs.


This ensures that the procedural graphs can be easily converted to MaterialX and USDShade graphs.

I don't see this as the goal for the extension - the goal is to create an agnostic way to store procedural images - that can then be used as texture sources in glTF.

rsahlin · 2024-07-09T08:31:55Z

extensions/2.0/Khronos/KHR_texture_procedurals/README.md

+
+    For the first version of this extension nodes which are used to define shading models are not allowed. Please refer to the [resources](#resources) section for links to supported  MaterialX node definitions.
+
+2. **Fidelity**: Provide the ability to generate complex patterns, noise, or other effects that currently must be "baked" into texture maps. Provide the ability to provide mappings from one shading model to another without baking in a consistent manner via procedural graphs. Provide the ability to map procedurals graphs to unlit materials. Reduces runtime memory usage by generating patterns / shading programmatically.


I propose something along the lines of:

Provide the ability to generate procedural patterns, noise or other visual effects of sufficiently high fidelity to replace the use of pixel (bitmap) based images.
Reduce the transfer size by generating images, either at load- or run-time.

rsahlin · 2024-07-09T08:33:22Z

extensions/2.0/Khronos/KHR_texture_procedurals/README.md

+
+2. **Fidelity**: Provide the ability to generate complex patterns, noise, or other effects that currently must be "baked" into texture maps. Provide the ability to provide mappings from one shading model to another without baking in a consistent manner via procedural graphs. Provide the ability to map procedurals graphs to unlit materials. Reduces runtime memory usage by generating patterns / shading programmatically.
+
+3. **Editability and Extensibility**: Extend runtime editability by exposing logic and interfaces for procedural graphs as well as providing a means to create new or extend existing node definitions.


This is not the goal of this extension - this extension should simply reference already existing operators/functions and provide a means to store them with a glTF asset.

rsahlin · 2024-07-09T08:43:24Z

extensions/2.0/Khronos/KHR_texture_procedurals/README.md

+
+### Material Binding
+
+To connect a graph `output` to a surface or displacement shader input the procedural extension can be declared within the a texture reference for a given material in the `materials` array.


I propose to simply connect the procedurals to an image in the imagesarray - if this image is created at load time or dynamically created at runtime is up to the implementation.

rsahlin · 2024-07-09T08:48:55Z

extensions/2.0/Khronos/KHR_texture_procedurals/README.md

+</details>
+
+
+### Procedural Definitions


I propose a solution where the procedural image extension uses a set of already declared operators/functions.
These functions are what are available, if need for future additions that will need to be done in a separate extension.

The upside of having pre-declared operators is that its straightforward to do implementation testcases to verify if an implementation is compliant or not.
Ie, for each operator there are a number of testcases that provides input data and are used to validate the output.

The total amount of operators needed is not that large, probably around 20, maybe even less.

For a non complete comparison - have a look at what we specified in the Ikea procedural prototypes
This is not the complete declaration but it's enough to show the differences in design:

https://github.com/idlabs/procedural-image

rsahlin · 2024-07-09T09:12:24Z

Food for thought!

Perhaps it's better to make this into a vendor extension (remove KHR prefix) and rename to materialx_shader_graph or something to that effect?

That way you would be able to keep the current design, if the extension receives traction it can be made into a KHR extension in the future.

What do you think?

Thanks for the fix.

… comes from the nodes. Inputs on nodes can be aniated. Further discussion on defails on existing extension required. Explicit note N:M graph -> material bindings possible.

* KHR_materials_ior: Minor typo: plural instead of singular (#2413) * Update anisotropy extension spec (#2409) * Clarify and simplify formula for attenuation. (#2414) * Add TRACE prefix (#2417) * Update clearcoat extension spec (#2415) * Update README. (#2427) * Add KHR_node_hoverability (#2428) * Add KHR_node_hoverability * Resort extension list * EXT_mesh_gpu_instancing: Suggest extensionRequired usage (#2405) * Add button/link to glTF Discord (#2429) As there is a lot more traffic to the Khronos glTF Discord than there is to Slack, a button pointing users to Discord will be helpful. --------- Co-authored-by: Andreas Atteneder <andreas.atteneder@unity3d.com> Co-authored-by: Alexey Knyazev <3479527+lexaknyazev@users.noreply.github.com> Co-authored-by: Ed Mackey <elm19087@gmail.com> Co-authored-by: Marco Hutter <javagl@javagl.de> Co-authored-by: Khronos Group Web Services <webservices@khronosgroup.org> Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com> Co-authored-by: James Riordon <outofcontrol@users.noreply.github.com>

Change max from 4 to 16 for data vector so 3x3 and 4x4 matrices can be supported.

kwokcb added 7 commits March 13, 2024 09:57

Branch for KHR_texture_procedurals

00395e6

Merge branch 'main' into KHR_texture_procedurals

fca8b93

Transfer over working document (#2376)

4c76d35

* Start document consolidation. * Transferred doc.

Cleanup spec document.

5897028

KHR_texture_procedurals:

6bccd66

- Update checkerboard example with USD equiivalent and mermaid graph. - Add USD/ / MaterialX / glTF mapping

Finish transfer of google docs information (minus verbose examples) t…

c1c37ee

…o spec README.

Merge branch 'main' into KHR_texture_procedurals

a12b0ed

lexaknyazev marked this pull request as draft April 16, 2024 16:20

ashwinbhat reviewed Apr 30, 2024

View reviewed changes

extensions/2.0/Khronos/KHR_texture_procedurals/README.md Show resolved Hide resolved

ashwinbhat reviewed Apr 30, 2024

View reviewed changes

extensions/2.0/Khronos/KHR_texture_procedurals/README.md Show resolved Hide resolved

ashwinbhat reviewed Apr 30, 2024

View reviewed changes

extensions/2.0/Khronos/KHR_texture_procedurals/README.md Outdated Show resolved Hide resolved

ashwinbhat reviewed Apr 30, 2024

View reviewed changes

extensions/2.0/Khronos/KHR_texture_procedurals/README.md Outdated Show resolved Hide resolved

ashwinbhat reviewed Apr 30, 2024

View reviewed changes

extensions/2.0/Khronos/KHR_texture_procedurals/README.md Outdated Show resolved Hide resolved

Update versioned extension declaration syntax.

63cc8e6

kwokcb commented May 11, 2024

View reviewed changes

extensions/2.0/Khronos/KHR_texture_procedurals/README.md Outdated Show resolved Hide resolved

Update to use abbreviated name for MaterialX extension

bca76d2

DennisSmolek mentioned this pull request Jun 5, 2024

NodeMaterial: Support MaterialX Standard Nodes mrdoob/three.js#20541

Open

ashwinbhat and others added 4 commits June 9, 2024 11:24

Separate into two proposals

7d43bfd

1. KHR_texture_procedural 2. EXT_texture_procedural_mx_1_39

Fixup some links

b53e09f

Merge branch 'main' into KHR_texture_procedurals

a080e17

Cleanup EXT_texture_procedurals_mx_1.39 for extension specification. …

4e3d894

…Add in mapping section for texture_infos.

kwokcb commented Jun 12, 2024

View reviewed changes

rsahlin reviewed Jul 9, 2024

View reviewed changes

kwokcb and others added 9 commits July 24, 2024 08:26

Update language for texture coordinate / transform bindings.

d1e4309

Fix typo in "multioutput" (#2432)

6b487dc

Thanks for the fix.

Update on multi-channel texture_info and 3-channel texcoord bindings.

7e1798e

Add clarifications for texute placement and sampling. All information…

81e0d38

… comes from the nodes. Inputs on nodes can be aniated. Further discussion on defails on existing extension required. Explicit note N:M graph -> material bindings possible.

Fix schemas: remove mimetype and added in extensionsUsed.

67bf359

Fix vector to support matrix

7a489b2

Change max from 4 to 16 for data vector so 3x3 and 4x4 matrices can be supported.

Update animation binding using animation pointer extension.

b24ce23

Merge branch 'main' into KHR_texture_procedurals

2dd5f10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KHR_texture_procedurals Draft Proposal #2381

KHR_texture_procedurals Draft Proposal #2381

kwokcb commented Apr 3, 2024 •

edited by lexaknyazev

Loading

kwokcb Jun 12, 2024

rsahlin Jul 9, 2024

rsahlin Jul 9, 2024

rsahlin Jul 9, 2024

rsahlin Jul 9, 2024

rsahlin Jul 9, 2024

rsahlin Jul 9, 2024

rsahlin Jul 9, 2024

rsahlin Jul 9, 2024

rsahlin commented Jul 9, 2024


		## Handling MaterialX glTF PBR Nodes

		The glTF PBR node specified within MaterialX exposes a number of input ports which may not map 1:1 with the glTF PBR material


		More specifically __only__ nodes defined in the MaterialX specification are supported. This ensures that the procedural graphs can be easily converted to MaterialX and USDShade graphs.

		For the first version of this extension nodes which are used to define shading models are not allowed. Please refer to the [resources](#resources) section for links to supported MaterialX node definitions.


		### Motivation

		Textures represented as procedural graphs provides a way to extend the capabilities of glTF materials beyond what is possible with traditional texture maps. Key objectives includes:


		1. Interoperability: Adhere to an industry a standard node schema (MaterialX) with a runtime friendly JSON representation.

		More specifically __only__ nodes defined in the MaterialX specification are supported. This ensures that the procedural graphs can be easily converted to MaterialX and USDShade graphs.


		For the first version of this extension nodes which are used to define shading models are not allowed. Please refer to the [resources](#resources) section for links to supported MaterialX node definitions.

		2. Fidelity: Provide the ability to generate complex patterns, noise, or other effects that currently must be "baked" into texture maps. Provide the ability to provide mappings from one shading model to another without baking in a consistent manner via procedural graphs. Provide the ability to map procedurals graphs to unlit materials. Reduces runtime memory usage by generating patterns / shading programmatically.


		2. Fidelity: Provide the ability to generate complex patterns, noise, or other effects that currently must be "baked" into texture maps. Provide the ability to provide mappings from one shading model to another without baking in a consistent manner via procedural graphs. Provide the ability to map procedurals graphs to unlit materials. Reduces runtime memory usage by generating patterns / shading programmatically.

		3. Editability and Extensibility: Extend runtime editability by exposing logic and interfaces for procedural graphs as well as providing a means to create new or extend existing node definitions.


		### Material Binding

		To connect a graph `output` to a surface or displacement shader input the procedural extension can be declared within the a texture reference for a given material in the `materials` array.

KHR_texture_procedurals Draft Proposal #2381

Are you sure you want to change the base?

KHR_texture_procedurals Draft Proposal #2381

Conversation

kwokcb commented Apr 3, 2024 • edited by lexaknyazev Loading

Draft Proposal for KHR_texture_procedurals

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rsahlin commented Jul 9, 2024

kwokcb commented Apr 3, 2024 •

edited by lexaknyazev

Loading

Draft Proposal for `KHR_texture_procedurals`