Skip to content

Commit

Permalink
Merge pull request #3864 from raspberrypi/develop
Browse files Browse the repository at this point in the history
Roll out minor updates
  • Loading branch information
nathan-contino authored Sep 30, 2024
2 parents 362ac17 + 21b99df commit c64ac22
Show file tree
Hide file tree
Showing 3 changed files with 7 additions and 9 deletions.
2 changes: 1 addition & 1 deletion documentation/asciidoc/accessories/ai-camera/about.adoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[[ai-camera]]
== About

The Raspberry Pi AI Camera uses the Sony IMX500 imaging sensor to provide low-latency and high-performance AI capabilities to any camera application. The tight integration with https://www.raspberrypi.com/documentation/computers/camera_software.adoc[Raspberry Pi's camera software stack] allows users to deploy their own neural network models with minimal effort.
The Raspberry Pi AI Camera uses the Sony IMX500 imaging sensor to provide low-latency and high-performance AI capabilities to any camera application. Tight integration with https://www.raspberrypi.com/documentation/computers/camera_software.adoc[Raspberry Pi's camera software stack] allows users to deploy their own neural network models with minimal effort.

image::images/ai-camera.png[The Raspberry Pi AI Camera]

Expand Down
8 changes: 4 additions & 4 deletions documentation/asciidoc/accessories/ai-camera/details.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,15 @@ image::images/imx500-comparison.svg[Traditional versus IMX500 AI camera systems]

The left side demonstrates the architecture of a traditional AI camera system. In such a system, the camera delivers images to the Raspberry Pi. The Raspberry Pi processes the images and then performs AI inference. Traditional systems may use external AI accelerators (as shown) or rely exclusively on the CPU.

The right side demonstrates the architecture of a system that uses IMX500. The camera module contains a small Image Signal Processor (ISP) which turns the raw camera image data into an _input tensor_. The camera module sends this tensor directly into the AI accelerator within the camera, which produces an _output tensor_ that contains the inferencing results. The AI accelerator sends this tensor to the Raspberry Pi. There is no need for an external accelerator, nor for the Raspberry Pi to run neural network software on the CPU.
The right side demonstrates the architecture of a system that uses IMX500. The camera module contains a small Image Signal Processor (ISP) which turns the raw camera image data into an **input tensor**. The camera module sends this tensor directly into the AI accelerator within the camera, which produces an **output tensor** that contains the inferencing results. The AI accelerator sends this tensor to the Raspberry Pi. There is no need for an external accelerator, nor for the Raspberry Pi to run neural network software on the CPU.

To fully understand this system, familiarise yourself with the following concepts:

The _Input Tensor_:: The part of the sensor image passed to the AI engine for inferencing. Produced by a small on-board ISP which also crops and scales the camera image to the dimensions expected by the neural network that has been loaded. The input tensor is not normally made available to applications, though it is possible to access it for debugging purposes.
Input Tensor:: The part of the sensor image passed to the AI engine for inferencing. Produced by a small on-board ISP which also crops and scales the camera image to the dimensions expected by the neural network that has been loaded. The input tensor is not normally made available to applications, though it is possible to access it for debugging purposes.

The _Region of Interest_ (ROI):: Specifies exactly which part of the sensor image is cropped out before being rescaled to the size demanded by the neural network. Can be queried and set by an application. The units used are always pixels in the full resolution sensor output. The default ROI setting uses the full image received from the sensor, cropping no data.
Region of Interest (ROI):: Specifies exactly which part of the sensor image is cropped out before being rescaled to the size demanded by the neural network. Can be queried and set by an application. The units used are always pixels in the full resolution sensor output. The default ROI setting uses the full image received from the sensor, cropping no data.

The _Output Tensor_:: The results of inferencing performed by the neural network. The precise number and shape of the outputs depend on the neural network. Application code must understand how to handle the tensor.
Output Tensor:: The results of inferencing performed by the neural network. The precise number and shape of the outputs depend on the neural network. Application code must understand how to handle the tensor.

=== System Architecture

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,9 @@ The Model Compression Toolkit generates a quantised model in the following forma
* Keras (TensorFlow)
* ONNX (PyTorch)


=== Conversion

To convert a model, first install the converter tools.

To convert a model, first install the converter tools:

[tabs%sync]
======
Expand Down Expand Up @@ -96,7 +94,7 @@ To package the model into an RPK file, run the following command:

[source,console]
----
imx500-package.sh -i <path to packerOut.zip> -o <output folder>
$ imx500-package.sh -i <path to packerOut.zip> -o <output folder>
----

This command should create a file named `network.rpk` in the output folder. You'll pass the name of this file to your IMX500 camera applications.
Expand Down

0 comments on commit c64ac22

Please sign in to comment.