Skip to content

Latest commit

 

History

History
96 lines (52 loc) · 6.43 KB

collaborations.md

File metadata and controls

96 lines (52 loc) · 6.43 KB
Raw

What’s the deal with AI for Earth camera trap collaborations?

Table of contents

  1. Overview
  2. Information about our camera trap work
  3. Questions about specific camera trap use cases

Overview

This page summarizes what we do to help our collaborators, typically ecologists, more specifically ecologists who are overwhelmed by camera trap images. This page also includes some questions we ask new collaborators, to help assess whether our tools are useful, and – if so – what the right set of tools is for a particular project.

Basically this page is the response we give when someone emails us and says “I have too many camera trap images! Can you help me?!?!”.

Information about our camera trap work

All of our work in this space is open-sourced here:

    https://github.com/Microsoft/cameratraps

The core of our work is a model – we call it the “MegaDetector” – that we use to separate images that are empty/animals/people/vehicles:

    https://github.com/microsoft/CameraTraps/blob/master/megadetector.md

Watch the fun video there and you’ll get the idea. Note that this model does not identify specific species. We believe there will never be a one-size-fits-all species classifier for the whole world, but at this more generic level, we can help people in lots of ecosystems without training custom models for each project.

Of course, a machine learning model isn’t useful to most ecologists by itself, so we “package” this model in a variety of ways. For most of our collaborators, they send us images (anywhere from thousands to millions), which we run through this detector in the cloud, then we send back a results file, and we’ve integrated with a variety of tools that camera trap researchers already use, to make it relatively painless to use our results in the context of a real workflow. Our most mature integration is with an open-source tool called Timelapse:

    https://github.com/microsoft/CameraTraps/blob/master/api/batch_processing/integration/timelapse.md

In some cases, we also train species classifiers for specific ecosystems, and we have tools for training classifiers (we actually train classifiers on individual animals found by our detector, not on whole images):

    https://github.com/microsoft/CameraTraps/blob/master/classification/TUTORIAL.md

…but we believe that most people will see much larger efficiency gains at the more generic level, so this is where we have focused so far.

We also package our camera trap model into two different APIs: a batch processing API for processing large volumes of images with some latency, and a real-time API for interactive scenarios (e.g. anti-poaching applications).

A slightly more detailed description of this work is available here, including a list of some of the organizations currently leveraging our tools:

    https://medium.com/microsoftazure/accelerating-biodiversity-surveys-with-azure-machine-learning-9be53f41e674

Usually the first step with a new collaborator is just running our model on a few thousand images and seeing what happens, so if you’re interested in trying this on your images, we can work out a way to transfer a set of example images.

After that, we’ll typically send back a page of sample results; depending on whether you already know the “right” answer for these images, the results will look like one of these:

    http://dolphinvm.westus2.cloudapp.azure.com/data/snapshot-serengeti/s7-eval/postprocessing-no-gt/
    [with no ground truth, i.e. without knowing the right answers]

    http://dolphinvm.westus2.cloudapp.azure.com/data/snapshot-serengeti/s7-eval/postprocessing-detection-gt/
    [with ground truth, i.e. where we know the right answers]

In case it’s helpful, we also maintain a literature survey on “everything we know about machine learning for camera traps”:

    https://github.com/agentmorris/camera-trap-ml-survey

…and we also maintain a repository of public, labeled camera trap data to facilitate training new models (the largest such repository that we’re aware of):

    http://lila.science/datasets

Our camera trap work is part of our larger efforts in using machine learning for biodiversity monitoring:

    http://aka.ms/biodiversitysurveys

Questions about specific camera trap use cases

These questions help us assess how we can best help a new collaborator, and which of our tools will be most applicable to a particular project.

  1. Can you provide a short overview of your project? What ecosystem are you working in, and what are the key species of interest?

  2. About how many images do you have that you’ve already annotated, from roughly the same environments as the photos you need to process in the future?

  3. If you have some images you’ve already annotated:

  • Did you keep all the empty images, or only the images with animals?
  • Are they from exactly the same camera locations that you need to process in the future (as in, cameras literally bolted in place), or from similar locations?

  1. About how many images do you have waiting for processing right now?

  2. About how many images do you expect to process in the next, e.g., 1 year?

  3. What tools do you use to process and annotate images? For example, do you:

  • Move images to folders named by species
  • Keep an Excel spreadsheet open and fill it with filenames and species IDs
  • Use a tool like Timelapse or Reconyx MapView that’s specifically for camera traps
  • Use a tool like Adobe Bridge or digiKam that’s for general-purpose image management

  1. About what percentage of your images are empty?

  2. About what percentage of your images typically contain vehicles or people, and what do you want to do with those images? I.e., do you consider those “noise” (the same as empty images), or do you need those labeled as well?

  3. What is your level of fluency in Python?

  4. Do you have a GPU available (or access to cloud-based GPUs)? “I don't know what a GPU is” is a perfectly good answer.