Can we specify the layers you are interested in for a custom nn ?

[r4hul77]

I am currently deploying a custom neural network on a DepthAI device, leveraging a RegNet backbone followed by two fully connected layers. Based on its architecture, I anticipated a superior frame rate performance compared to traditional object detectors. For context, models like MobileNet SSD achieve around 30 FPS on the same setup. However, my custom model is significantly underperforming in terms of speed, yielding only about 2 FPS.

I suspect the bottleneck may be due to the model's output handling, where the device waits for the entire neural network (NN) message before proceeding. This leads me to question if there's a method to predefine the layers of interest, allowing me to streamline the output process by focusing only on specific layers' data before executing the xout function. Such a capability would presumably reduce processing time and enhance frame rate efficiency.

Is there an existing feature within the DepthAI API or a workaround that facilitates this selective output processing? Any guidance on optimizing the FPS by limiting the output to certain layers would be greatly appreciated.

[Erol444]

@tersekmatija do you perhaps know if there are any tools from openvino that would "benchmark" model's layers?

[r4hul77]

Just to add on to it, the output layers in my network are 217 and 222, I want to just access them with xout nothing else.

[tersekmatija]

Not sure how your code currently looks like (where the post-processing of the outputs happen), but when you are compiling the model, you can provide --output flag to model optimizer. You should name the output layers there, and this will essentially cut away all the layers that follow. I assume this is what you are looking for?

If you want to define multiple outputs and get results from an earlier layer sooner than from the last layer, this is not something that we currently expose.

[r4hul77]

I'm using blobconvertor, from_onxx function. It appears that this function doesn't take the outputs argument.

[tersekmatija]

Not directly, but you can pass it as part of model optimizer arguments.

[r4hul77]

Can you point me to an example ?

[tersekmatija]

You should be able to do it in the same manner as for TensorFlow described here

[r4hul77]

Can I have multiple outputs ?