Merge pull request #188 from quixio/tonybedford/sc-47076/todos-relate…

…d-to-template-projects

[47076] - Fixes for Template Project tutorials

docs/index.md

@@ -10,27 +10,53 @@ Welcome to the Quix developer documentation!
 
     Our docs support hotkeys. Press ++slash++, ++s++, or ++f++ to activate search, ++p++ or ++comma++ to go to the previous page, ++n++ or ++period++ to go to the next page.
 
-## Documentation changelog
+## See Quix in action
 
-The documentation changelog can be found in the [documentation repository Wiki](https://github.com/quixio/quix-docs/wiki/Docs-Releases).
+See the following live demos and running pipelines:
 
-This is in addition to general product changes which are summarized in the [changes documentation](./platform/changes.md).
+<div class="grid cards" markdown>
 
-## Get started
+- __Image processing demo__
 
-If you're new to Quix, here are some resources to help get you started quickly.
+    ---
 
-First, sign up for a [free account](https://portal.platform.quix.ai/self-sign-up){target=_blank}.
+    See a real-time image processing live demo!
 
-<div class="grid cards" markdown>
+    [:octicons-arrow-right-24: Image processing demo](https://app-demo-computervisiondemo-prod.deployments.quix.ai/){target=_blank}
 
-- __What is Quix?__
+- __Image processing pipeline__
 
     ---
 
-    New to Quix? Find out more!
+    See a real-time image processing pipeline running!
 
-    [:octicons-arrow-right-24: What is Quix?](./platform/what-is-quix.md)
+    [:octicons-arrow-right-24: Image processing pipeline](https://portal.platform.quix.ai/pipeline?workspace=demo-computervisiondemo-prod&token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Ik1qVTBRVE01TmtJNVJqSTNOVEpFUlVSRFF6WXdRVFF4TjBSRk56SkNNekpFUWpBNFFqazBSUSJ9.eyJodHRwczovL3F1aXguYWkvb3JnX2lkIjoiZGVtbyIsImh0dHBzOi8vcXVpeC5haS9vd25lcl9pZCI6ImF1dGgwfDI4YWQ4NWE4LWY1YjctNGFjNC1hZTVkLTVjYjY3OGIxYjA1MiIsImh0dHBzOi8vcXVpeC5haS90b2tlbl9pZCI6ImMzNzljNmVlLWNkMmYtNDExZC1iOGYyLTMyMDU0ZDc5MTY2YSIsImh0dHBzOi8vcXVpeC5haS9leHAiOiIxNzM3ODI5NDc5LjIyMyIsImlzcyI6Imh0dHBzOi8vYXV0aC5xdWl4LmFpLyIsInN1YiI6ImtyMXU4MGRqRllvUUZlb01nMGhqcXZia29lRkxFRDVBQGNsaWVudHMiLCJhdWQiOiJxdWl4IiwiaWF0IjoxNjk1NzE2MDI4LCJleHAiOjE2OTgzMDgwMjgsImF6cCI6ImtyMXU4MGRqRllvUUZlb01nMGhqcXZia29lRkxFRDVBIiwiZ3R5IjoiY2xpZW50LWNyZWRlbnRpYWxzIiwicGVybWlzc2lvbnMiOltdfQ.Ndm0K2iNHPxDq1ohF-yb-6LzIqx_UY8Ptcq0kAwSNye12S3deX_eDkC4XqZqW2NoSLd3GsmWV9PZGetGGp2IlqshQFZtUMp6WP6hq917ZC1i8JFx93PAbY7NT_88nFDovVlaRcoTpWvI-03KbryLkAoB28c6qb3EFwjCWFBuy_yA4yjQ8uF0-AZ0R9Qi4IBaekXWqcgO0a91gVRg0oA_hnzJFoR-EnZ2G1ZSxtuVgnyyPuQTMUvzJuUT_IJTLzEB_kejX0pcXRZBIwHP8MWLB4mE5DtIdz4jm8WIA4eZJZ7ZCG4dk-adQwZ2BdkNknV5eEwRgRJL4ybaplkaDlR-dg){target=_blank}
+
+- __Sentiment analysis demo__
+
+    ---
+
+    See a sentiment analysis live demo!
+
+    [:octicons-arrow-right-24: Sentiment analysis demo](https://sentimentdemoui-demo-chatappdemo-prod.deployments.quix.ai/chat){target=_blank}
+
+- __Sentiment analysis pipeline__
+
+    ---
+
+    See a sentiment analysis pipeline running!
+
+    [:octicons-arrow-right-24: Sentiment analysis pipeline](https://portal.platform.quix.ai/pipeline?workspace=demo-chatappdemo-prod&token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Ik1qVTBRVE01TmtJNVJqSTNOVEpFUlVSRFF6WXdRVFF4TjBSRk56SkNNekpFUWpBNFFqazBSUSJ9.eyJodHRwczovL3F1aXguYWkvb3JnX2lkIjoiZGVtbyIsImh0dHBzOi8vcXVpeC5haS9vd25lcl9pZCI6ImF1dGgwfDI4YWQ4NWE4LWY1YjctNGFjNC1hZTVkLTVjYjY3OGIxYjA1MiIsImh0dHBzOi8vcXVpeC5haS90b2tlbl9pZCI6ImMzNzljNmVlLWNkMmYtNDExZC1iOGYyLTMyMDU0ZDc5MTY2YSIsImh0dHBzOi8vcXVpeC5haS9leHAiOiIxNzM3ODI5NDc5LjIyMyIsImlzcyI6Imh0dHBzOi8vYXV0aC5xdWl4LmFpLyIsInN1YiI6ImtyMXU4MGRqRllvUUZlb01nMGhqcXZia29lRkxFRDVBQGNsaWVudHMiLCJhdWQiOiJxdWl4IiwiaWF0IjoxNjk1NzE2MDI4LCJleHAiOjE2OTgzMDgwMjgsImF6cCI6ImtyMXU4MGRqRllvUUZlb01nMGhqcXZia29lRkxFRDVBIiwiZ3R5IjoiY2xpZW50LWNyZWRlbnRpYWxzIiwicGVybWlzc2lvbnMiOltdfQ.Ndm0K2iNHPxDq1ohF-yb-6LzIqx_UY8Ptcq0kAwSNye12S3deX_eDkC4XqZqW2NoSLd3GsmWV9PZGetGGp2IlqshQFZtUMp6WP6hq917ZC1i8JFx93PAbY7NT_88nFDovVlaRcoTpWvI-03KbryLkAoB28c6qb3EFwjCWFBuy_yA4yjQ8uF0-AZ0R9Qi4IBaekXWqcgO0a91gVRg0oA_hnzJFoR-EnZ2G1ZSxtuVgnyyPuQTMUvzJuUT_IJTLzEB_kejX0pcXRZBIwHP8MWLB4mE5DtIdz4jm8WIA4eZJZ7ZCG4dk-adQwZ2BdkNknV5eEwRgRJL4ybaplkaDlR-dg){target=_blank}
+
+</div>
+
+## Get started
+
+If you're new to Quix, here are some resources to help get you started quickly.
+
+First, sign up for a [free account](https://portal.platform.quix.ai/self-sign-up){target=_blank}.
+
+<div class="grid cards" markdown>
 
 - __Quickstart__
 
@@ -47,14 +73,30 @@ First, sign up for a [free account](https://portal.platform.quix.ai/self-sign-up
     Build a complete stream processing pipeline in under 30 minutes.
 
     [:octicons-arrow-right-24: Quix Tour](./platform/quixtour/overview.md)
-
+
+- __What is Quix?__
+
+    ---
+
+    New to Quix? Find out more!
+
+    [:octicons-arrow-right-24: What is Quix?](./platform/what-is-quix.md)
+
 - __Glossary__
 
     ---
 
     List of Quix terms.
 
     [:octicons-arrow-right-24: Glossary](./platform/glossary.md)
+
+- __Ingest data__
+
+    ---
+
+    Read about some of the many ways you can get your data into Quix.
+
+    [:octicons-arrow-right-24: Ingest data](./platform/ingest-data.md)
 
 -   __Help__
 
@@ -146,6 +188,12 @@ Read more about the Quix Streams Client Library and APIs.
 
 </div>
 
+## Documentation changelog
+
+The documentation changelog can be found in the [documentation repository Wiki](https://github.com/quixio/quix-docs/wiki/Docs-Releases).
+
+This is in addition to general product changes which are summarized in the [changes documentation](./platform/changes.md).
+
 ## Contribute
 
 <div class="grid cards" markdown>

docs/platform/how-to/environment-variables.md

@@ -58,3 +58,7 @@ To create an environment variable that uses a secret variable:
 3. Select the corresponding secret variable from the dropdown list.
 
 Now, when the environment variable appears in code or the UI, its value is the secret variable you assigned, but you cannot not see the value of the secret variable, only its name.
+
+## See also
+
+* [Configuring environments with YAML variables](./yaml-variables.md)

docs/platform/how-to/personal-access-token-pat.md

@@ -1,7 +1,9 @@
 # Personal access token (PAT)
 
-Personal Access Tokens, or PATS, are bearer tokens that can be used to authenticate the various Quix APIs, such as the Portal API.
+Personal Access Tokens, or PATs, are bearer tokens that can be used to authenticate the various Quix APIs, such as the Portal API.
 
-To obtain a PAT, log in to the Quix Portal, and click `Settings` in the main left-hand navigation. Then click `APIs and tokens`.
+To obtain a PAT, log in to Quix, and click on your profile icon in the top right corner, then click `Personal Access Tokens`. You can then generate a PAT with a lifetime suitable for your use case.
+
+Alternatively, log in to the Quix Portal, and click `Settings` in the main left-hand navigation. Then, for a specific environment, click `APIs and tokens`.
 
 In the `APIs and tokens` dialog, you can click `Personal Access Tokens` to generate PATs, or a Streaming Token for use with the Quix Streams client library.

docs/platform/how-to/testing-data-store.md

@@ -4,7 +4,7 @@ Quix provides a data store for testing and debugging purposes.
 
 While [topics](../glossary.md#topic) do provide a configurable retention time, persisting data into a database provides advantages - for example, you can perform powerful queries to retrieve historical data. This data can be retrieved and displayed using the Data Explorer, or retrieved using the [Query API](../../apis/query-api/intro.md).
 
-Quix provides a very simple way to persist data in a topic. Simply locate the topic in your topic list, and click the `Persistance` button. 
+Quix provides a very simple way to persist data in a topic. Simply locate the topic in your topic list, and click the `Persistence` button. 
 
 !!! important
 

docs/platform/tutorials/image-processing/get-project.md

@@ -114,17 +114,31 @@ Open the service and edit the environment variable as shown here:
 
 ### Web UI service
 
-When testing the UI you might find Google Maps does not load correctly for you - this is because the code has the Quix Google Maps API key. To work around this, you can set the Google Maps API key to an empty string, and then enable "developer mode" in your browser - the maps then display correctly. 
+Note if you just want to try out the UI without performing the following steps, you can do that in the [demo](https://app-demo-computervisiondemo-prod.deployments.quix.ai/){target=_blank}.
 
-To set the Google Maps API key to an empty string, you need to edit `app.module.ts` and modify the `apiKey` field in `AgmCoreModule.forRoot` to the following:
+When testing the UI you might find Google Maps does not load correctly for you - this is because the code has the Quix Google Maps API key. To work around this, you can set the Google Maps API key to an empty string. Google maps will then default to "developer mode" - the map will display correctly, but will be watermarked with 'for developer use'. To do this, in the Applications list, click on the `TfL image processing UI` application.
+
+To set the Google Maps API key to an empty string, you need to edit `src/app/app.module.ts` and modify the `apiKey` field in `AgmCoreModule.forRoot` to the following:
 
 ``` typescript
 AgmCoreModule.forRoot({
       apiKey: ''
     }),
 ```
 
-Other optional services may require similar configuration, for example, the Quix Amazon S3 connector service requires your S3 credentials fo you want to use it.
+Also, you need your [workspace ID](../../how-to/get-environment-id.md) and a [PAT](../../how-to/personal-access-token-pat.md) to get the UI fully working. Once you have these, you'll need to access the code for the service, and set these values.
+
+In the file `src/app/services/quix.service.ts`, locate the following code, and replace the place holders with your values:
+
+``` typescript
+/*~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-*/
+  /*WORKING LOCALLY? UPDATE THESE!*/
+  private workingLocally = false; // set to true if working locally
+  private token: string = '<your_pat>'; // Create a token in the Tokens menu and paste it here
+  public workspaceId: string = '<your_workspace_id>'; // Look in the URL for the Quix Portal your workspace ID is after 'workspace='
+```
+
+Other optional services may require similar configuration, for example, the Quix Amazon S3 connector service requires your S3 credentials if you want to use it.
 
 ## See also
 

docs/platform/tutorials/image-processing/index.md

@@ -6,6 +6,20 @@ The pipeline uses the Transport for London (TfL) traffic cameras, known as Jam C
 
 You'll fork the complete project from GitHub, and then create a Quix project from the forked repo, so you have a copy of the full pipeline code running in your Quix account. You then examine the data flow through the pipeline, using tools provided by the Quix Portal.
 
+## Live demo
+
+You can see the project running live on Quix:
+
+<div id="wrap">
+    <iframe id="frame" src="https://app-demo-computervisiondemo-prod.deployments.quix.ai/" title="Live real-time image processing"></iframe>
+</div>
+
+You can interact with it here, on this page, or open the page to view it more clearly [here](https://app-demo-computervisiondemo-prod.deployments.quix.ai/){target="_blank"}.
+
+## See the pipeline running
+
+See the [pipeline running](https://portal.platform.quix.ai/pipeline?workspace=demo-computervisiondemo-prod&token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Ik1qVTBRVE01TmtJNVJqSTNOVEpFUlVSRFF6WXdRVFF4TjBSRk56SkNNekpFUWpBNFFqazBSUSJ9.eyJodHRwczovL3F1aXguYWkvb3JnX2lkIjoiZGVtbyIsImh0dHBzOi8vcXVpeC5haS9vd25lcl9pZCI6ImF1dGgwfDI4YWQ4NWE4LWY1YjctNGFjNC1hZTVkLTVjYjY3OGIxYjA1MiIsImh0dHBzOi8vcXVpeC5haS90b2tlbl9pZCI6ImMzNzljNmVlLWNkMmYtNDExZC1iOGYyLTMyMDU0ZDc5MTY2YSIsImh0dHBzOi8vcXVpeC5haS9leHAiOiIxNzM3ODI5NDc5LjIyMyIsImlzcyI6Imh0dHBzOi8vYXV0aC5xdWl4LmFpLyIsInN1YiI6ImtyMXU4MGRqRllvUUZlb01nMGhqcXZia29lRkxFRDVBQGNsaWVudHMiLCJhdWQiOiJxdWl4IiwiaWF0IjoxNjk1NzE2MDI4LCJleHAiOjE2OTgzMDgwMjgsImF6cCI6ImtyMXU4MGRqRllvUUZlb01nMGhqcXZia29lRkxFRDVBIiwiZ3R5IjoiY2xpZW50LWNyZWRlbnRpYWxzIiwicGVybWlzc2lvbnMiOltdfQ.Ndm0K2iNHPxDq1ohF-yb-6LzIqx_UY8Ptcq0kAwSNye12S3deX_eDkC4XqZqW2NoSLd3GsmWV9PZGetGGp2IlqshQFZtUMp6WP6hq917ZC1i8JFx93PAbY7NT_88nFDovVlaRcoTpWvI-03KbryLkAoB28c6qb3EFwjCWFBuy_yA4yjQ8uF0-AZ0R9Qi4IBaekXWqcgO0a91gVRg0oA_hnzJFoR-EnZ2G1ZSxtuVgnyyPuQTMUvzJuUT_IJTLzEB_kejX0pcXRZBIwHP8MWLB4mE5DtIdz4jm8WIA4eZJZ7ZCG4dk-adQwZ2BdkNknV5eEwRgRJL4ybaplkaDlR-dg) without needing to sign up to Quix.
+
 ## Technologies used
 
 Some of the technologies used by this template project are listed here.
@@ -39,26 +53,6 @@ Some of the technologies used by this template project are listed here.
 * [Microsoft SignalR](https://learn.microsoft.com/en-us/aspnet/signalr/){target=_blank}
 * [Google Maps](https://developers.google.com/maps){target=_blank}
 
-## Live demo
-
-You can see the project running live on Quix:
-
-<div id="wrap">
-    <iframe id="frame" src="https://app-demo-computervisiondemo-prod.deployments.quix.ai/" title="Live real-time image processing"></iframe>
-</div>
-
-You can interact with it here, on this page, or open the page to view it more clearly [here](https://app-demo-computervisiondemo-prod.deployments.quix.ai/){target="_blank"}.
-
-## Watch a video
-
-Explore the pipeline:
-
-**Loom video coming soon.**
-
-??? Transcript
-
-    **Transcript**
-
 ## GitHub repository
 
 The complete code for this project can be found in the [Quix GitHub repository](https://github.com/quixio/computer-vision-demo){target="_blank"}.
@@ -131,7 +125,7 @@ The second part of the pipeline is:
 
 There are several *main* stages in the pipeline:
 
-1. *TfL camera feed* - TfL Camera feed or "Jam Cams". This service retrieves the raw data from the TfL API endpoint. A list of all JamCams is retrieved, along with the camera data. The camera data contains a link to a video clip from the camera. These video clips are hosted by TfL in MP4 format on AWS S3. A stream is created for each camera, and the camera data published to this stream. Using multiple streams in this way enables a solution capable of horizontal scaling, through additional topic partitions and, optionally, replicated services in a consumer group. Once the camera list has been scanned, the service sleeps for two minutes, and then repeats the previous code. This reduces the load, and also means the API limit of 500 requests per minute is not exceeded. Messages are passed to the frame grabber.
+1. *TfL camera feed* - TfL Camera feed or "Jam Cams". This service retrieves the raw data from the TfL API endpoint. A list of all JamCams is retrieved, along with the camera data. The camera data contains a link to a video clip from the camera. These video clips are hosted by TfL in MP4 format on AWS S3. A stream is created for each camera, and the camera data published to this stream. Using multiple streams in this way enables a solution capable of horizontal scaling, through additional topic partitions and, optionally, replicated services in a consumer group. Once the camera list has been scanned, the service sleeps for a configurable amount of time, and then repeats the previous code. This reduces the load, and also means the API limit of 500 requests per minute is not exceeded. Messages are passed to the frame grabber.
 
 2. *TfL traffic camera frame grabber* - this service grabs frames from a TfL video file (MP4 format) at the rate specified. By default the grabber extracts one frame every 100 frames, which is typically one per five seconds of video. Messages are passed to the object detection service.
 
@@ -147,9 +141,18 @@ There are also some additional services in the pipeline:
 
 2. *Max vehicle window* - calculates the total vehicles over a time window of one day. This service publishes messages its output topic. 
 
-3. *Data API* - this REST API service provide two endpoints: one returns the *Max vehicle window* values for the specified camera, and the other endpoint returns camera data for the specified camera. This API is called by the UI to obtain useful data.
+3. *Data buffer* - this provides a one second data buffer. This helps reduce load on the Data API service.
+
+4. *Data API* - this REST API service provides the following endpoints: 
+
+    * `max_vehicles` - last known maximum vehicle count for each camera. 24 hour rolling window.
+    * `detected_objects` - output from the computer vision service for all cameras. excludes images
+    * `vehicles` - the last known vehicle count for each camera
+    * `image` - the last image from the specified camera
+
+    This API is called by the UI to obtain useful data.
 
-4. *S3* - stores objects in Amazon Web Services (AWS) S3. This service enables you to persist any data or results you might like to keep more permanently.
+5. *S3* - stores objects in Amazon Web Services (AWS) S3. This service enables you to persist any data or results you might like to keep more permanently.
 
 More details are provided on all these services later in the tutorial.
 

docs/platform/tutorials/image-processing/other-services.md

@@ -14,7 +14,14 @@ Briefly, these services are:
 
 * *Data buffer* - this provides a one second data buffer. This helps reduce load on the Data API service.
 
-* *Data API* - this REST API service provide two endpoints: one returns the *Max vehicle window* values for the specified camera, and the other endpoint returns camera data for the specified camera. This API is called by the UI to obtain useful data.
+* *Data API* - this REST API service provides the following endpoints:
+
+    * `max_vehicles` - last known maximum vehicle count for each camera. 24 hour rolling window.
+    * `detected_objects` - output from the computer vision service for all cameras. excludes images
+    * `vehicles` - the last known vehicle count for each camera
+    * `image` - the last image from the specified camera
+
+    This API is called by the UI to obtain useful data.
 
 * *S3* - stores objects in Amazon Web Services (AWS) S3. This service enables you to persist any data or results you might like to keep more permanently.
 
@@ -150,10 +157,12 @@ See the documentation on [using buffers](https://quix.io/docs/client-library/pub
 
 The data service offloads calculations that could be done in the web client, and instead provides key data only when the UI needs it. The UI can request this data when it needs it through the REST API of the Data API service.
 
-The Data API provides these endpoints:
+The Data API provides the following endpoints:
 
 * max_vehicles
 * detected_objects
+* vehicles
+* image
 
 These are used by the UI to obtain and then display the data on the web interface.
 
@@ -212,6 +221,18 @@ Example response JSON:
 ]
 ```
 
+### Vehicles
+
+The last known vehicle count for each camera.
+
+See the [README](https://github.com/quixio/computer-vision-demo/tree/main/Data%20API){target=_blank} for more information.
+
+### Image
+
+The last image from the specified camera.
+
+See the [README](https://github.com/quixio/computer-vision-demo/tree/main/Data%20API){target=_blank} for more information.
+
 ## S3
 
 This is the standard Quix code sample [AWS S3 destination connector](https://quix.io/docs/library_readmes/connectors/s3-destination.html). It takes messages on the input topic and writes them to S3. There is an optional batching facility whereby you can batch messages and then write them to S3 in a batch - this can be more efficient for higher frequency data. You can control batching based on time interval or message count.