Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvement : External tracker concept introduction #444

Open
BearrGod opened this issue May 3, 2024 · 0 comments
Open

Improvement : External tracker concept introduction #444

BearrGod opened this issue May 3, 2024 · 0 comments
Labels
question Theory or implementation question

Comments

@BearrGod
Copy link

BearrGod commented May 3, 2024

Hello everyone. I've been poking around the openvin's codebase and i've done some diagnostics at runtime through the debug mode. And i've realized that the feature tracking is by far the most time consuming part of the calculations ,which is fairly normal, especially when you consider high resolution images. I was wondering if it wouldn't be interesting to add an external tracker concept, especially for gpu and vpu enabled devices. And then instead of a full image we could provide a vector of tracked points directly as input. We could extend the CameraData class like this :

struct  TrackedPoint
{
  uint32_t 	age = 0 ;  //Number of frame we tracked for
  float 	harrisScore = 0.f ;  // Score of the tracked point
  uint32_t 	id = 0 ;  // Persistent id to track the point
  cv::Point2i 	position ;  // Position in image of the tracked point
 float 	trackingError = 0.f ;  // tracking error
};


/**
 * @brief Struct for a collection of camera measurements.
 *
 * For each image we have a camera id and timestamp that it occured at.
 * If there are multiple cameras we will treat it as pair-wise stereo tracking.
 */
struct CameraData {

  /// Timestamp of the reading
  double timestamp;

  /// Camera ids for each of the images collected
  std::vector<int> sensor_ids;

  /// Raw image we have collected for each camera
  std::vector<cv::Mat> images;

  /// Image sizes in case we decide to only work with trackers and not full images. 
  std::vector<cv::Size> image_sizes;

  /// Eventual tracking points revieved. These are tracked point from last frames to this frame. 
  std::vector<std::vector<TrackedPoint>> tracked_points ; 

  /// Tracking masks for each camera we have
  std::vector<cv::Mat> masks;

  /// Sort function to allow for using of STL containers
  bool operator<(const CameraData &other) const {
    if (timestamp == other.timestamp) {
      int id = *std::min_element(sensor_ids.begin(), sensor_ids.end());
      int id_other = *std::min_element(other.sensor_ids.begin(), other.sensor_ids.end());
      return id < id_other;
    } else {
      return timestamp < other.timestamp;
    }
  }
};

I was wondering how to create a TrackExternal class which obey to the TrackBase schemas to be able to do this.
Thanks a lot.

@goldbattle goldbattle added the question Theory or implementation question label Sep 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Theory or implementation question
Projects
None yet
Development

No branches or pull requests

2 participants