Skip to content

Commit

Permalink
Merge pull request #147 from funnelfiasco/reorganize_demos
Browse files Browse the repository at this point in the history 
Reorganize demos to enable user flow tracking
  • Loading branch information
@pxp928
pxp928 authored Sep 23, 2024
2 parents dacc67a + 7c4ba15 commit 3edbd6c
Show file tree
Hide file tree
Showing 10 changed files with 282 additions and 331 deletions.
195 changes: 96 additions & 99 deletions expanding-your-view.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ within the ingested SBOM.
and [GUAC Data](https://github.com/guacsec/guac-data/archive/refs/heads/main.zip)
extracted to `guac-data-main`.

## Step 1: Ingest Vault’s SBOM
## Ingest Vault’s SBOM

For demo purposes, let's ingest Vault’s SBOM. To do this, we will use the help
of the `guacone collect files` command.
Expand All @@ -60,35 +60,6 @@ of the `guacone collect files` command.
{"level":"info","ts":1684774157.09894,"caller":"cmd/files.go:187","msg":"completed ingesting 1 documents of 1"}
```

## Step 2: Check the ingestion logs

We can pull the logs from docker to see the progress of the ingestion:

```bash
docker logs guac-guac-ingestor-1
```

The results for the Vault SBOM ingestion will look like the following:

```bash
{"level":"info","ts":1681992933.3817039,"caller":"emitter/nats_emitter.go:121","msg":"creating stream \"DOCUMENTS\" and subjects \"DOCUMENTS.*\""}
{"level":"info","ts":1681994359.306624,"caller":"process/process.go:97","msg":"[processor: 856ef1f5-1627-4ec9-b9e1-507b7bdee58f] docTree Processed: {Collector:FileCollector Source:file:///../guac-data/top-dh-sboms/vault.json}"}
{"level":"info","ts":1681994359.31475,"caller":"parser/parser.go:128","msg":"parsing document tree with root type: SPDX"}
{"level":"info","ts":1681994359.3263202,"caller":"helpers/assembler.go:34","msg":"assembling CertifyScorecard: 0"}
{"level":"info","ts":1681994359.32635,"caller":"helpers/assembler.go:39","msg":"assembling IsDependency: 2260"}
{"level":"info","ts":1681994359.7843368,"caller":"helpers/assembler.go:44","msg":"assembling IsOccurence: 963"}
{"level":"info","ts":1681994359.954613,"caller":"helpers/assembler.go:49","msg":"assembling HasSLSA: 0"}
{"level":"info","ts":1681994359.954643,"caller":"helpers/assembler.go:54","msg":"assembling CertifyVuln: 0"}
{"level":"info","ts":1681994359.954647,"caller":"helpers/assembler.go:59","msg":"assembling IsVuln: 0"}
{"level":"info","ts":1681994359.954649,"caller":"helpers/assembler.go:64","msg":"assembling HasSourceAt: 0"}
{"level":"info","ts":1681994359.9546518,"caller":"helpers/assembler.go:69","msg":"assembling CertifyBad: 0"}
{"level":"info","ts":1681994359.954654,"caller":"helpers/assembler.go:74","msg":"assembling CertifyGood: 0"}
{"level":"info","ts":1681994359.9546711,"caller":"cmd/ingest.go:118","msg":"got collect entries to add: 349"}
{"level":"info","ts":1681994359.9560268,"caller":"parser/parser.go:110","msg":"[ingestor: 04462d2a-a2c7-4aa9-95eb-2183cb5f249d] ingested docTree: {Collector:FileCollector Source:file:///../guac-data/top-dh-sboms/vault.json}"}
```

## Step 3: Review the automated query for more information

As the ingestion process occurs, the collector subscriber service of GUAC
collects pURLs, OCI strings, and others to determine if there is more
information available to be pulled into the graph DB.
Expand All @@ -103,80 +74,19 @@ As the SBOM is ingested it:
This process is recursive, meaning that the pURLs that the dependency relies on
will also be queried!

We can pull the logs from docker to see which packages deps.dev collector found:
If you want to watch the collector subscriber work, see the
[checking the logs](#ingestion-logs) section at the bottom of this page.

```bash
docker logs guac-depsdev-collector-1
```

The results from the deps.dev collector logs will look like the following:

```bash
{"level":"info","ts":1681994369.748968,"caller":"deps_dev/deps_dev.go:217","msg":"obtained additional metadata for package: pkg:golang/cloud.google.com/go@v0.65.0"}
{"level":"info","ts":1681994372.493675,"caller":"deps_dev/deps_dev.go:217","msg":"obtained additional metadata for package: pkg:golang/cloud.google.com/go/spanner@v1.5.1"}
{"level":"info","ts":1681994375.3482509,"caller":"deps_dev/deps_dev.go:217","msg":"obtained additional metadata for package: pkg:golang/cloud.google.com/go/storage@v1.10.0"}
{"level":"info","ts":1681994376.722956,"caller":"deps_dev/deps_dev.go:217","msg":"obtained additional metadata for package: pkg:golang/code.cloudfoundry.org/gofileutils@v0.0.0-20170111115228-4d0c80011a0f"}
{"level":"info","ts":1681994377.476279,"caller":"deps_dev/deps_dev.go:217","msg":"obtained additional metadata for package: pkg:golang/github.com/Azure/azure-pipeline-go@v0.2.3"}
{"level":"info","ts":1681994380.7538428,"caller":"deps_dev/deps_dev.go:217","msg":"obtained additional metadata for package: pkg:golang/github.com/Azure/azure-sdk-for-go@v61.4.0+incompatible"}
{"level":"info","ts":1681994382.8232992,"caller":"deps_dev/deps_dev.go:217","msg":"obtained additional metadata for package: pkg:golang/github.com/Azure/azure-storage-blob-go@v0.14.0"}
```

If we go back to the ingestor logs, we will see deps.dev documents being
ingested.

```bash
docker logs guac-guac-ingestor-1
```

These logs will show the following with the collector and source being from
deps.dev.

```bash
{"level":"info","ts":1681994398.146413,"caller":"parser/parser.go:128","msg":"parsing document tree with root type: DEPS_DEV"}
{"level":"info","ts":1681994398.146731,"caller":"helpers/assembler.go:34","msg":"assembling CertifyScorecard: 1"}
{"level":"info","ts":1681994398.148156,"caller":"helpers/assembler.go:39","msg":"assembling IsDependency: 12"}
{"level":"info","ts":1681994398.156023,"caller":"helpers/assembler.go:44","msg":"assembling IsOccurence: 0"}
{"level":"info","ts":1681994398.156051,"caller":"helpers/assembler.go:49","msg":"assembling HasSLSA: 0"}
{"level":"info","ts":1681994398.1560571,"caller":"helpers/assembler.go:54","msg":"assembling CertifyVuln: 0"}
{"level":"info","ts":1681994398.156063,"caller":"helpers/assembler.go:59","msg":"assembling IsVuln: 0"}
{"level":"info","ts":1681994398.156069,"caller":"helpers/assembler.go:64","msg":"assembling HasSourceAt: 4"}
{"level":"info","ts":1681994398.157971,"caller":"helpers/assembler.go:69","msg":"assembling CertifyBad: 0"}
{"level":"info","ts":1681994398.1579862,"caller":"helpers/assembler.go:74","msg":"assembling CertifyGood: 0"}
{"level":"info","ts":1681994398.157998,"caller":"cmd/ingest.go:118","msg":"got collect entries to add: 12"}
{"level":"info","ts":1681994398.158665,"caller":"parser/parser.go:110","msg":"[ingestor: 04462d2a-a2c7-4aa9-95eb-2183cb5f249d] ingested docTree: {Collector:deps.dev Source:deps.dev}"}
```

From the logs we see that `CertifyScorecard`, `IsDependency` and `HasSourceAt`
are being ingested. We will further inspect this information in the coming
sections.

## Step 4: Review the automated query for vulnerabilities

As we saw in the section above, GUAC automatically looks for more information
for an ingested SBOM. What about vulnerabilities?
As we saw above, GUAC automatically looks for more information for an ingested
SBOM. What about vulnerabilities?

The certifier (currently utilizing the OSV database, with more integrations to
come) is configured to run and query the vulnerability database to determine if
a package has a vulnerability.

We can pull the logs from docker to see the OSV certifier in action.

```bash
docker logs guac-osv-certifier-1
```

The results from the osv certifier logs will look like the following:

```bash
{"level":"info","ts":1681994498.498469,"caller":"cmd/osv.go:115","msg":"[209.458µs] completed doc {Collector:guac Source:guac}"}
{"level":"info","ts":1681994498.4986901,"caller":"cmd/osv.go:115","msg":"[216µs] completed doc {Collector:guac Source:guac}"}
{"level":"info","ts":1681994498.4989061,"caller":"cmd/osv.go:115","msg":"[211.042µs] completed doc {Collector:guac Source:guac}"}
{"level":"info","ts":1681994498.498911,"caller":"cmd/osv.go:122","msg":"certifier ended gracefully"}
```

We will further inspect these vulnerabilities in the following section.

## Step 5: Examine the information collected
## Examine the information collected

To understand what was collected, we will utilize the GraphQL playground. The
playground is accessible via: `http://localhost:8080/graphql`
Expand Down Expand Up @@ -675,10 +585,97 @@ integrate GUAC services into an IDE to provide information on whether or not a
package should be used due to a low OpenSSF Scorecard score or critical
vulnerability.

## Cleanup
Next, let's see [what is known and unknown about your software supply chain]({{ site.baseurl}}{%link known-and-unknown.md %}).
## Checking the logs
### Ingestion logs
We can pull the logs from docker to see the progress of the
[ingestion](#ingest-vaults-sbom):
```bash
docker logs guac-guac-ingestor-1
```
The results for the Vault SBOM ingestion will look like the following:
```bash
{"level":"info","ts":1681992933.3817039,"caller":"emitter/nats_emitter.go:121","msg":"creating stream \"DOCUMENTS\" and subjects \"DOCUMENTS.*\""}
{"level":"info","ts":1681994359.306624,"caller":"process/process.go:97","msg":"[processor: 856ef1f5-1627-4ec9-b9e1-507b7bdee58f] docTree Processed: {Collector:FileCollector Source:file:///../guac-data/top-dh-sboms/vault.json}"}
{"level":"info","ts":1681994359.31475,"caller":"parser/parser.go:128","msg":"parsing document tree with root type: SPDX"}
{"level":"info","ts":1681994359.3263202,"caller":"helpers/assembler.go:34","msg":"assembling CertifyScorecard: 0"}
{"level":"info","ts":1681994359.32635,"caller":"helpers/assembler.go:39","msg":"assembling IsDependency: 2260"}
{"level":"info","ts":1681994359.7843368,"caller":"helpers/assembler.go:44","msg":"assembling IsOccurence: 963"}
{"level":"info","ts":1681994359.954613,"caller":"helpers/assembler.go:49","msg":"assembling HasSLSA: 0"}
{"level":"info","ts":1681994359.954643,"caller":"helpers/assembler.go:54","msg":"assembling CertifyVuln: 0"}
{"level":"info","ts":1681994359.954647,"caller":"helpers/assembler.go:59","msg":"assembling IsVuln: 0"}
{"level":"info","ts":1681994359.954649,"caller":"helpers/assembler.go:64","msg":"assembling HasSourceAt: 0"}
{"level":"info","ts":1681994359.9546518,"caller":"helpers/assembler.go:69","msg":"assembling CertifyBad: 0"}
{"level":"info","ts":1681994359.954654,"caller":"helpers/assembler.go:74","msg":"assembling CertifyGood: 0"}
{"level":"info","ts":1681994359.9546711,"caller":"cmd/ingest.go:118","msg":"got collect entries to add: 349"}
{"level":"info","ts":1681994359.9560268,"caller":"parser/parser.go:110","msg":"[ingestor: 04462d2a-a2c7-4aa9-95eb-2183cb5f249d] ingested docTree: {Collector:FileCollector Source:file:///../guac-data/top-dh-sboms/vault.json}"}
```
We can pull the logs from docker to see which packages deps.dev collector found:
```bash
docker logs guac-depsdev-collector-1
```
The results from the deps.dev collector logs will look like the following:
```bash
{"level":"info","ts":1681994369.748968,"caller":"deps_dev/deps_dev.go:217","msg":"obtained additional metadata for package: pkg:golang/cloud.google.com/go@v0.65.0"}
{"level":"info","ts":1681994372.493675,"caller":"deps_dev/deps_dev.go:217","msg":"obtained additional metadata for package: pkg:golang/cloud.google.com/go/spanner@v1.5.1"}
{"level":"info","ts":1681994375.3482509,"caller":"deps_dev/deps_dev.go:217","msg":"obtained additional metadata for package: pkg:golang/cloud.google.com/go/storage@v1.10.0"}
{"level":"info","ts":1681994376.722956,"caller":"deps_dev/deps_dev.go:217","msg":"obtained additional metadata for package: pkg:golang/code.cloudfoundry.org/gofileutils@v0.0.0-20170111115228-4d0c80011a0f"}
{"level":"info","ts":1681994377.476279,"caller":"deps_dev/deps_dev.go:217","msg":"obtained additional metadata for package: pkg:golang/github.com/Azure/azure-pipeline-go@v0.2.3"}
{"level":"info","ts":1681994380.7538428,"caller":"deps_dev/deps_dev.go:217","msg":"obtained additional metadata for package: pkg:golang/github.com/Azure/azure-sdk-for-go@v61.4.0+incompatible"}
{"level":"info","ts":1681994382.8232992,"caller":"deps_dev/deps_dev.go:217","msg":"obtained additional metadata for package: pkg:golang/github.com/Azure/azure-storage-blob-go@v0.14.0"}
```
If we go back to the ingestor logs, we will see deps.dev documents being
ingested.
```bash
docker logs guac-guac-ingestor-1
```
These logs will show the following with the collector and source being from
deps.dev.
```bash
{"level":"info","ts":1681994398.146413,"caller":"parser/parser.go:128","msg":"parsing document tree with root type: DEPS_DEV"}
{"level":"info","ts":1681994398.146731,"caller":"helpers/assembler.go:34","msg":"assembling CertifyScorecard: 1"}
{"level":"info","ts":1681994398.148156,"caller":"helpers/assembler.go:39","msg":"assembling IsDependency: 12"}
{"level":"info","ts":1681994398.156023,"caller":"helpers/assembler.go:44","msg":"assembling IsOccurence: 0"}
{"level":"info","ts":1681994398.156051,"caller":"helpers/assembler.go:49","msg":"assembling HasSLSA: 0"}
{"level":"info","ts":1681994398.1560571,"caller":"helpers/assembler.go:54","msg":"assembling CertifyVuln: 0"}
{"level":"info","ts":1681994398.156063,"caller":"helpers/assembler.go:59","msg":"assembling IsVuln: 0"}
{"level":"info","ts":1681994398.156069,"caller":"helpers/assembler.go:64","msg":"assembling HasSourceAt: 4"}
{"level":"info","ts":1681994398.157971,"caller":"helpers/assembler.go:69","msg":"assembling CertifyBad: 0"}
{"level":"info","ts":1681994398.1579862,"caller":"helpers/assembler.go:74","msg":"assembling CertifyGood: 0"}
{"level":"info","ts":1681994398.157998,"caller":"cmd/ingest.go:118","msg":"got collect entries to add: 12"}
{"level":"info","ts":1681994398.158665,"caller":"parser/parser.go:110","msg":"[ingestor: 04462d2a-a2c7-4aa9-95eb-2183cb5f249d] ingested docTree: {Collector:deps.dev Source:deps.dev}"}
```
From the logs we see that `CertifyScorecard`, `IsDependency` and `HasSourceAt`
are being ingested.
To delete all the GUAC resources, run:
### OSV certifier logs
We can pull the logs from docker to see the OSV certifier in action.
```bash
docker compose down
docker logs guac-osv-certifier-1
```
The results from the osv certifier logs will look like the following:
```bash
{"level":"info","ts":1681994498.498469,"caller":"cmd/osv.go:115","msg":"[209.458µs] completed doc {Collector:guac Source:guac}"}
{"level":"info","ts":1681994498.4986901,"caller":"cmd/osv.go:115","msg":"[216µs] completed doc {Collector:guac Source:guac}"}
{"level":"info","ts":1681994498.4989061,"caller":"cmd/osv.go:115","msg":"[211.042µs] completed doc {Collector:guac Source:guac}"}
{"level":"info","ts":1681994498.498911,"caller":"cmd/osv.go:122","msg":"certifier ended gracefully"}
```
2 changes: 1 addition & 1 deletion guac-use-cases.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ title: GUAC demos
permalink: /guac-use-cases/
parent: Getting started with GUAC
has_children: true
nav_order: 3
nav_order: 2
---

# GUAC demos
Expand Down
23 changes: 6 additions & 17 deletions known-and-unknown.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,26 +38,11 @@ to determine what we know and don't know about the artifacts.

- A fresh copy of the [GUAC service infrastructure through Docker Compose]({{
site.baseurl }}{%link setup.md %}). Including the `guacone` binary in your path
and [GUAC Data](https://github.com/guacsec/guac-data/archive/refs/heads/main.zip)
extracted to `guac-data-main`.
and [demo data]({{ site.baseurl}}{% link setup-ingest-data.md %}) extracted to
`guac-data-main`.
- Completion of the [Expanding your view of the software supply
chain demo]({{ site.baseurl }}{%link expanding-your-view.md %})

## Ingest GUAC Data (if needed)

If you have not already done so, ingest the GUAC data to be used by the demo.

```bash
guacone collect files guac-data-main/docs/
```

Once ingested we will see the following message (the number of documents may
vary):

```bash
{"level":"info","ts":1681864775.1161852,"caller":"cmd/files.go:201","msg":"completed ingesting 67 documents of 67"}
```

## Understanding the data

GUAC, at the time of the beta release, can store various metadata about an
Expand Down Expand Up @@ -525,3 +510,7 @@ Based on the information gathered above:
immediately. Knowing the unknown is the first key step in securing the supply
chain. If the security teams and developers have no knowledge of these, how
can we keep shifting left effectively?
Now that you know something about your software supply chain, let's look at what
you probably want to know about right away:
[vulnerabilties]({{ site.baseurl}}{% link querying-via-cli.md %}).
30 changes: 3 additions & 27 deletions patch-cli.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,27 +30,11 @@ resolve a security incident.

- A fresh copy of the [GUAC service infrastructure through Docker Compose]({{
site.baseurl }}{%link setup.md %}). Including the `guacone` binary in your path
and [GUAC Data](https://github.com/guacsec/guac-data/archive/refs/heads/main.zip)
ingested.

and [demo data]({{ site.baseurl}}{% link setup-ingest-data.md %}) extracted to
`guac-data-main`.
- The [GUAC visualizer]({{ site.baseurl }}{%link guac-visualizer.md %}) up and running.

## Step 1: Ingest GUAC Data (if needed)

If you have not already done so, ingest the GUAC data to be used by the demo.

```bash
guacone collect files guac-data-main/docs/
```

Once ingested we will see the following message (the number of documents may
vary):

```bash
{"level":"info","ts":1681864775.1161852,"caller":"cmd/files.go:201","msg":"completed ingesting 67 documents of 67"}
```

## Step 2: Run the Query Patch CLI
## Run the Query Patch CLI

**Packge Name Node Example**

Expand Down Expand Up @@ -173,14 +157,6 @@ attached to the version we inputted. This is because if a specific package
version is vulnerable, it implies that the name one level above it is affected
as well.

## Cleanup

To delete the all the GUAC components run:

```bash
docker compose down
```

## Conclusion

From this demo, we learned how we can quickly analyze the GUAC data to find a
Expand Down
20 changes: 7 additions & 13 deletions querying-via-cli.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,11 @@ vulnerability.

- A fresh copy of the [GUAC service infrastructure through Docker Compose]({{
site.baseurl }}{%link setup.md %}). Including the `guacone` binary in your path
and [GUAC Data](https://github.com/guacsec/guac-data/archive/refs/heads/main.zip)
extracted to `guac-data-main`.

and [demo data]({{ site.baseurl}}{% link setup-ingest-data.md %}) extracted to
`guac-data-main`.
- The [GUAC visualizer]({{ site.baseurl }}{%link guac-visualizer.md %}) up and running.

## Step 1. Ingest a vulnerability SPDX SBOM
## Ingest a vulnerability SPDX SBOM

For demo purposes, let's ingest a known bad SPDX SBOM that contains several
vulnerabilities. To do this, we will use the `guacone` command, which is an
Expand All @@ -53,7 +52,7 @@ Once ingested you will see the following message:
{"level":"info","ts":1681821120.1626382,"caller":"cmd/files.go:201","msg":"completed ingesting 1 documents of 1"}
```

## Step 2: Draw further insight from OSV.dev
## Draw further insight from OSV.dev

One of the benefits of GUAC is that it’s not a static database; it is constantly
evolving and trying to find more information on the artifacts ingested. To
Expand Down Expand Up @@ -97,7 +96,7 @@ GUAC. After a set period of time (set by the user), it will re-query the
information to ensure that it's always up-to-date. For demo purposes, we ran it
just once.

## Step 3: Run the Query Vulnerability CLI
## Run the Query Vulnerability CLI

Now that our GUAC instance is up and running with up-to-date information on the
vulnerable image that we ingest, we will look at how we can utilize this data
Expand Down Expand Up @@ -244,10 +243,5 @@ specific pURL we are interested in contains a direct or in-direct vulnerability.
We also learned that this is just one of many utilizations of GUAC’s GraphQL API
to create more tools such as these quickly and easily!

## Cleanup

To delete the all the GUAC components run:

```bash
docker compose down
```
Now that you've indentified a vulnerability, how do you react to it? The next
step in the demo covers [reacting to a supply chain incident]({{ site.baseurl}}{%link supply-chain.md %}).
Loading

0 comments on commit 3edbd6c

Please sign in to comment.