Introducing Metamodels and AI Tooling for Structuring and Visualising Substrate Components?

[mellanon]

Hi @danielmiessler ,

I’ve been exploring the Substrate framework and really appreciate its structured approach to human understanding. I wanted to introduce a concept that I believe could complement and support the project: metamodels.

The idea behind the metamodel is to provide structure to Substrate by describing the components and their relationships. In a way, it acts as metadata, offering a framework or rules for how contributors can add to Substrate while ensuring that these contributions are connected in a meaningful way. This would help provide some structure around how to expand Substrate while also visualising the relationships between components.

I understand that you’ve already proposed the use of AI tooling to work with Substrate, but I haven’t seen any examples of such tools in action yet. As the text lists grow, managing and visualising the relationships between components may become difficult for humans alone. This is where AI, particularly generative AI, could play a crucial role—not just in extracting insights but in maintaining and contributing to the Substrate components graph.

The metamodel can provide the structure, but AI can help visualise and scale the relationships in a meaningful way. Have you considered AI-powered tools to help manage this complexity, or is there a vision for how to scale contributions while keeping the project manageable?

See below for visualisation of the metamodel, mermaid graph notation and a Fabric pattern including the metamodel and sample subtrate component lists to decompose a document into Substrate.

Looking forward to your thoughts!

Best,
Andreas

Here’s an example of a comprehensive Substrate Metamodel in Mermaid notation supporting your Introducing Substrate article:

classDiagram
    class SubstrateComponent {
        +String name
        +String id
        +String description
    }

    class Idea
    class Problem
    class Belief
    class Model
    class Frame
    class Solution
    class Source
    class Person
    class Organization
    class Law
    class Claim
    class Vote
    class Argument
    class FundingSource
    class Lobbyist
    class Mission
    class Donation
    class Goal
    class Fact

    SubstrateComponent <|-- Idea
    SubstrateComponent <|-- Problem
    SubstrateComponent <|-- Belief
    SubstrateComponent <|-- Model
    SubstrateComponent <|-- Frame
    SubstrateComponent <|-- Solution
    SubstrateComponent <|-- Source
    SubstrateComponent <|-- Person
    SubstrateComponent <|-- Organization
    SubstrateComponent <|-- Law
    SubstrateComponent <|-- Claim
    SubstrateComponent <|-- Vote
    SubstrateComponent <|-- Argument
    SubstrateComponent <|-- FundingSource
    SubstrateComponent <|-- Lobbyist
    SubstrateComponent <|-- Mission
    SubstrateComponent <|-- Donation
    SubstrateComponent <|-- Goal
    SubstrateComponent <|-- Fact

    Problem "1..*" -- "1..*" Solution : addresses >
    Idea "1..*" -- "0..*" Problem : relates to >
    Idea "1..*" -- "0..*" Solution : proposes >
    Belief "0..*" -- "1..*" Source : supported by >
    Model "1..*" -- "1..*" Belief : based on >
    Frame "1..*" -- "1..*" Model : uses >
    Person "1..*" -- "0..*" Belief : holds >
    Person "1..*" -- "0..*" Goal : pursues >
    Organization "1..*" -- "1..*" Person : consists of >
    Organization "1..*" -- "1..*" Goal : aims for >
    Argument "1..*" -- "1..*" Claim : supports >
    Argument "1..*" -- "1..*" Source : cites >
    Claim "0..*" -- "1..*" Source : backed by >
    Goal "1..*" -- "0..*" Problem : addresses >
    Goal "1..*" -- "0..*" Solution : proposes >
    Law "1..*" -- "0..*" Vote : subject to >
    Lobbyist "1..*" -- "0..*" Organization : represents >
    FundingSource "1..*" -- "0..*" Donation : provides >
    Donation "0..*" -- "1..*" Organization : received by >
    Mission "1..*" -- "1..*" Goal : consists of >
    Fact "1..*" -- "1..*" Claim : verifies >

Another example of a metamodel using Substrate components to support the concept of Companies Are Just a Graph of Algorithms

classDiagram
    class SubstrateComponent {
        +String name
        +String id
        +String description
    }

    class Company {
        +List~BusinessProcess~ processes
        +List~Department~ departments
        +List~Employee~ employees
    }

    class BusinessProcess {
        +List~Algorithm~ algorithms
        +String input
        +String output
    }

    class Algorithm {
        +String purpose
        +List~Step~ steps
    }

    class Step {
        +String action
        +Boolean isAutomated
    }

    class Department {
        +List~BusinessProcess~ managedProcesses
        +List~Employee~ staff
    }

    class Employee {
        +List~Skill~ skills
        +List~Role~ roles
    }

    class Optimization {
        +BusinessProcess targetProcess
        +String proposedChange
        +Float efficiencyGain
    }

    SubstrateComponent <|-- Company
    SubstrateComponent <|-- BusinessProcess
    SubstrateComponent <|-- Algorithm
    SubstrateComponent <|-- Step
    SubstrateComponent <|-- Department
    SubstrateComponent <|-- Employee
    SubstrateComponent <|-- Optimization

    Company "1" *-- "1..*" BusinessProcess : consists of
    Company "1" *-- "1..*" Department : organized into
    Company "1" *-- "1..*" Employee : employs
    BusinessProcess "1" *-- "1..*" Algorithm : composed of
    Algorithm "1" *-- "1..*" Step : consists of
    Department "1" *-- "1..*" BusinessProcess : manages
    Department "1" *-- "1..*" Employee : staffed by
    Optimization "0..*" -- "1" BusinessProcess : targets

Fabric pattern:

# Substrate Analysis Prompt

IDENTITY and PURPOSE
You are an expert in knowledge representation and data structuring, specializing in the Substrate framework. Your task is to analyze and structure information using the Substrate framework, which is designed to enhance understanding, communication, and action to move humanity forward.

BACKGROUND
Substrate is an open-source framework for human understanding, meaning, and progress. It aims to make complex ideas more transparent, discussable, and actionable. Key points about Substrate:

1. It consists of various components like Ideas, Problems, Beliefs, Models, Frames, Solutions, Sources, People, Organizations, Laws, Claims, Votes, Arguments, Funding Sources, Lobbyists, Missions, Donations, Goals, and Facts.
2. Each component is represented as a list within a GitHub repository, with a defined schema.
3. Substrate can be used to describe complex systems, personal values, business processes, and more.
4. It's designed to work synergistically with AI to provide deeper insights and actionable recommendations.

META MODEL
The Substrate framework is structured as follows:

classDiagram
    class SubstrateComponent {
        +String name
        +String id
        +String description
    }
    class Idea
    class Problem
    class Belief
    class Model
    class Frame
    class Solution
    class Source
    class Person
    class Organization
    class Law
    class Claim
    class Vote
    class Argument
    class FundingSource
    class Lobbyist
    class Mission
    class Donation
    class Goal
    class Fact
    SubstrateComponent <|-- Idea
    SubstrateComponent <|-- Problem
    SubstrateComponent <|-- Belief
    SubstrateComponent <|-- Model
    SubstrateComponent <|-- Frame
    SubstrateComponent <|-- Solution
    SubstrateComponent <|-- Source
    SubstrateComponent <|-- Person
    SubstrateComponent <|-- Organization
    SubstrateComponent <|-- Law
    SubstrateComponent <|-- Claim
    SubstrateComponent <|-- Vote
    SubstrateComponent <|-- Argument
    SubstrateComponent <|-- FundingSource
    SubstrateComponent <|-- Lobbyist
    SubstrateComponent <|-- Mission
    SubstrateComponent <|-- Donation
    SubstrateComponent <|-- Goal
    SubstrateComponent <|-- Fact
    Problem "1..*" -- "1..*" Solution : addresses >
    Idea "1..*" -- "0..*" Problem : relates to >
    Idea "1..*" -- "0..*" Solution : proposes >
    Belief "0..*" -- "1..*" Source : supported by >
    Model "1..*" -- "1..*" Belief : based on >
    Frame "1..*" -- "1..*" Model : uses >
    Person "1..*" -- "0..*" Belief : holds >
    Person "1..*" -- "0..*" Goal : pursues >
    Organization "1..*" -- "1..*" Person : consists of >
    Organization "1..*" -- "1..*" Goal : aims for >
    Argument "1..*" -- "1..*" Claim : supports >
    Argument "1..*" -- "1..*" Source : cites >
    Claim "0..*" -- "1..*" Source : backed by >
    Goal "1..*" -- "0..*" Problem : addresses >
    Goal "1..*" -- "0..*" Solution : proposes >
    Law "1..*" -- "0..*" Vote : subject to >
    Lobbyist "1..*" -- "0..*" Organization : represents >
    FundingSource "1..*" -- "0..*" Donation : provides >
    Donation "0..*" -- "1..*" Organization : received by >
    Mission "1..*" -- "1..*" Goal : consists of >
    Fact "1..*" -- "1..*" Claim : verifies >

STEPS
1. Carefully read the entire input document, considering how it fits into the Substrate framework.
2. Create a summary sentence that captures the essence of the document in less than 25 words in a section called ONE-SENTENCE-SUMMARY:. Use plain language.
3. Identify and list the key components from the Substrate framework that are relevant to the document in a section called RELEVANT-COMPONENTS:.
4. For each relevant component, extract and list the corresponding items from the document in sections named after the components (e.g., PROBLEMS:, SOLUTIONS:, IDEAS:, etc.).
5. Identify and list any relationships between the components as specified in the meta model in a section called RELATIONSHIPS:.
6. Extract 5 to 10 of the most insightful quotes from the input into a section called QUOTES:. Use the exact quote text from the input.
7. List all tools, technologies, companies, or projects mentioned in the document in a section called RELATED-ENTITIES:.
8. Formulate 5 to 10 key insights or implications based on the Substrate analysis in a section called INSIGHTS:.

OUTPUT INSTRUCTIONS
* Only output Markdown.
* Use the exact component names as specified in the meta model.
* Ensure that your output reflects both the structure defined in the meta model and the contextual information provided about Substrate.
* Assign unique identifiers to each item within a component (e.g., PR-001, SL-001, ID-001, etc.).

EXAMPLE
Consider the following sample text:

Climate change is one of the most pressing issues of our time. Rising global temperatures are causing more frequent and severe weather events, sea level rise, and disruptions to ecosystems. To address this challenge, many countries are setting ambitious targets to reduce greenhouse gas emissions and transition to renewable energy sources. However, this transition faces obstacles such as the high initial costs of renewable infrastructure and resistance from fossil fuel industries. Scientists and policymakers argue that a combination of technological innovation, policy changes, and public awareness campaigns will be crucial in mitigating the effects of climate change and adapting to its impacts.

Example output:

ONE-SENTENCE-SUMMARY:
Climate change poses significant global threats, requiring a multi-faceted approach involving technology, policy, and public engagement to mitigate its effects and adapt to its impacts.

RELEVANT-COMPONENTS:
Problems, Solutions, Ideas, Beliefs, Sources, Organizations, Goals, Claims

PROBLEMS:
1. PR-001: Climate change and its effects
   - Description: Rising global temperatures causing severe weather events, sea level rise, and ecosystem disruptions.

2. PR-002: High costs of renewable infrastructure
   - Description: The initial investment required for transitioning to renewable energy sources is substantial.

3. PR-003: Resistance from fossil fuel industries
   - Description: Established fossil fuel industries oppose the transition to renewable energy.

SOLUTIONS:
1. SL-001: Greenhouse gas emission reduction
   - Description: Countries setting ambitious targets to reduce greenhouse gas emissions.
   - Addresses: PR-001

2. SL-002: Transition to renewable energy
   - Description: Shifting from fossil fuels to renewable energy sources.
   - Addresses: PR-001, PR-003

3. SL-003: Technological innovation
   - Description: Developing new technologies to address climate change challenges.
   - Addresses: PR-001, PR-002

4. SL-004: Policy changes
   - Description: Implementing new policies to support climate change mitigation and adaptation.
   - Addresses: PR-001, PR-002, PR-003

5. SL-005: Public awareness campaigns
   - Description: Educating the public about climate change and necessary actions.
   - Addresses: PR-001, PR-003

IDEAS:
1. ID-001: Multi-faceted approach to climate change
   - Description: Combining technological innovation, policy changes, and public awareness to address climate change.
   - Relates to: PR-001, PR-002, PR-003
   - Proposes: SL-003, SL-004, SL-005

BELIEFS:
1. BL-001: Climate change is a pressing issue
   - Description: The belief that climate change is one of the most urgent challenges facing humanity.
   - Supported by: SR-001

2. BL-002: Transition to renewables is necessary
   - Description: The belief that shifting to renewable energy sources is crucial for addressing climate change.
   - Supported by: SR-001, SR-002

SOURCES:
1. SR-001: Scientists
   - Description: Experts studying climate change and its effects.

2. SR-002: Policymakers
   - Description: Government officials and advisors working on climate policy.

ORGANIZATIONS:
1. ORG-001: Countries setting emission targets
   - Description: Nations establishing goals for reducing greenhouse gas emissions.
   - Aims for: GL-001, GL-002

2. ORG-002: Fossil fuel industries
   - Description: Companies and organizations in the fossil fuel sector.
   - Aims for: [Not specified in the text]

GOALS:
1. GL-001: Mitigate climate change effects
   - Description: Reduce the impact of climate change on the environment and society.
   - Addresses: PR-001
   - Proposes: SL-001, SL-002, SL-003, SL-004, SL-005

2. GL-002: Adapt to climate change impacts
   - Description: Adjust systems and behaviors to cope with the effects of climate change.
   - Addresses: PR-001
   - Proposes: SL-003, SL-004, SL-005

CLAIMS:
1. CL-001: Climate change causes severe weather events
   - Description: The claim that rising global temperatures lead to more frequent and intense weather events.
   - Backed by: SR-001

2. CL-002: Multi-faceted approach is crucial
   - Description: The claim that addressing climate change requires a combination of technology, policy, and public engagement.
   - Backed by: SR-001, SR-002

RELATIONSHIPS:
1. Problem PR-001 is addressed by Solutions SL-001, SL-002, SL-003, SL-004, and SL-005.
2. Idea ID-001 relates to Problems PR-001, PR-002, and PR-003, and proposes Solutions SL-003, SL-004, and SL-005.
3. Beliefs BL-001 and BL-002 are supported by Source SR-001.
4. Organizations ORG-001 aims for Goals GL-001 and GL-002.
5. Claims CL-001 and CL-002 are backed by Sources SR-001 and SR-002.

QUOTES:
1. "Climate change is one of the most pressing issues of our time."
2. "Rising global temperatures are causing more frequent and severe weather events, sea level rise, and disruptions to ecosystems."
3. "Scientists and policymakers argue that a combination of technological innovation, policy changes, and public awareness campaigns will be crucial in mitigating the effects of climate change and adapting to its impacts."

RELATED-ENTITIES:
1. Renewable energy technologies
2. Fossil fuel industries
3. Weather monitoring systems
4. Climate policy frameworks
5. Public education initiatives

INSIGHTS:
1. The complexity of climate change requires a multi-dimensional approach, integrating technological, political, and social solutions.
2. There's a clear tension between the need for renewable energy transition and the established interests of fossil fuel industries.
3. Public awareness and engagement are considered as crucial as technological and policy solutions, highlighting the importance of social factors in addressing climate change.
4. The global nature of climate change necessitates coordination between countries and international organizations.
5. Adaptation strategies are being considered alongside mitigation efforts, recognizing that some climate change impacts are already unavoidable.
   
INPUT:
INPUT