Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CameraExposureOffset and Fix Disparity to Depth Computation #346

Closed
wants to merge 6 commits into from

Conversation

borongyuan
Copy link
Contributor

@borongyuan borongyuan commented Jul 1, 2023

Overview

Author: Borong Yuan
These updates are mainly for SLAM applications.

  • We prefer to use dai::CameraExposureOffset::MIDDLE for image timestamp. dai::CameraExposureOffset::END is set as default, which is almost consistent with previous behavior.
  • Some SLAM algorithms require the input images to have exactly the same timestamp. When we use a separate Publisher for each camera, the timestamps will not be exactly the same since the exposure is controlled individually. Only those algorithms that support approx sync can be used. Modified so that multiple synced images can now be published in one BridgePublisher. One image acts as the main image, and other images are synchronized with it by comparing sequenceNum. All images are assigned the exact same timestamp as the main image. For example, we want to publish exact synced left, right and depth images. We use the depth image as the main image since it always arrives last. In the callback function of depth image, we find the frame with the same sequenceNum from left and right image queues. They will arrive no later than the depth image, so there will be no blocking. We found it easier to achieve exact sync without using BridgePublisher. So revert to the previous version.
  • The disparity to depth computation was incorrect. You should do dimensional analysis. In CameraInfo, abs(Tx) = fx' * baseline[m]. depth[mm] = fx' * baseline[mm] / disparity = fx' * abs(Tx) / fx' * 1000 / disparity = abs(P[3]) * 1000 / disparity.
  • You shouldn't just declare std::runtime_error. It doesn't work. You should throw them.

Changes

ROS distro: Noetic
List of changes:

  • add CameraExposureOffset to ImageConverter
  • fix disparity to depth computation

Testing

Hardware used:
Depthai library version: v2.22.0
CI fails due to warnings generated by API changes.

Visuals from testing

Add screenshots/gifs/videos from RVIZ or other visualizers demonstrating the effect of the changes when applicable.

@borongyuan borongyuan changed the title add CameraExposureOffset to ImageConverter Add CameraExposureOffset and Exact Sync Function Jul 10, 2023
@borongyuan borongyuan changed the title Add CameraExposureOffset and Exact Sync Function Add CameraExposureOffset and Fix Disparity to Depth Computation Jul 18, 2023
@Serafadam Serafadam mentioned this pull request Aug 31, 2023
@Serafadam
Copy link
Collaborator

Hi @borongyuan , thanks for the PR, those changes were added as parts of other PRs in the newer releases, so I'm closing the PR, please reopen it if you have further comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants