Into the Unknown

This repository contains the implementation and data used in the paper Into the Unknown: Active Monitoring of Neural Networks, published at RV 2021. To cite the work, you can use:

@inproceedings{intotheunknown21,
  author    = {Anna Lukina and
               Christian Schilling and
               Thomas A. Henzinger},
  editor    = {Lu Feng and
               Dana Fisman},
  title     = {Into the Unknown: Active Monitoring of Neural Networks},
  booktitle = {{RV}},
  series    = {LNCS},
  volume    = {12974},
  pages     = {42--61},
  publisher = {Springer},
  year      = {2021},
  url       = {https://doi.org/10.1007/978-3-030-88494-9\_3},
  doi       = {10.1007/978-3-030-88494-9\_3}
}

Installation

We use Python 3.6 but other Python versions may work as well. The easiest is to use a conda environment. The package requirements that need to be installed are found in the file requirements.txt.

Since the datasets are large and have mostly been used in our previous work, we do not include most of them here. You need to manually download them (see the links below) and extract them to the data folder of this repository.

Modify the file called paths.txt in the base folder, which contains two lines that are the paths to the model and dataset folders:

.../models/
.../data/

Here replace the ... with the absolute path to your clone of the repository.

Links to dataset files

• MNIST
• Fashion MNIST
• CIFAR-10
• GTSRB (You need to manually extract the file train.zip because the content is too large for Github.)
• EMNIST: This dataset is already included in the repository.

Recreation of the results

Below we describe how to obtain the results shown in the paper.

Models

The repository contains the pretrained models used in the evaluation. The models have been trained using the scripts run/train_INSTANCE.py where INSTANCE is the name of the model/data combination.

Evaluation

The scripts to reproduce the figures and tables of the paper are found in the folder run/:

• run_experiments_online.py (This script runs all experiments, which can also be run individually by modifying the script accordingly.)
• plot_experiments_online.py (This script creates all plots and requires that all results from the previous script have been obtained.)

Intermediate results of the experiments are stored in .csv files and the final plots are stored as .pdf files in the run/ folder.