Adversarial Reprogramming of Neural Networks

This repository contains a PyTorch implementation of the Adversarial Reprogramming framework, described in Adversarial Reprogramming of Neural Networks - Gamaleldin F. Elsayed, Ian Goodfellow, Jascha Sohl-Dickstein.

While the implementation adheres to the methodology and structure outlined by the authors, some parameters cannot be directly inferred from the paper, while others have been modified in order to reproduce comparable results.

The table below provides a description of the parameters used in this implementation.

Setting	Value
Batch size	64
Optimizer	Adam
Learning Rate	0.01
LR exp decay	0.96
LR epochs/decay	5
$\lambda$ (regularization)	$1\cdot 10^{-6}$
Epochs	100

The experiments were conducted using a configuration that includes the ResNet50 v2 network as the source model, pre-trained on ImageNet, and the CIFAR10 and MNIST as the target tasks.

In addition, two alternative methodologies were explored for mapping labels from the source task (ImageNet, 1000 classes) to the target task (CIFAR10/MNIST, 10 classes). Alongside the approach described in the paper, which maps the first 10 classes of ImageNet directly to the 10 target task classes, the following strategies were investigated:

• An affine transformation, to enable data-driven optimization of the label mapping. This approach demonstrated high accuracy.
• An informed mapping designed to select the most "semantically distinct labels" among the 1000 ImageNet classes. The selection process aimed to identify labels that the pretrained network could more effectively separate, using a criterion based on the GloVe algorithm (although this approach is based on the semantic difference between class labels, we believe it is more efficient to leverage the class distance in some network's embedding layer).

Instructions

To replicate the experiments, set the desired parameters on config_cifar10.py or config_mnist.py,
then run $python train_base.py --cfg mnist or $python train_base.py --cfg cifar10

To use the alternative affine label mapping, run $python train_affine.py --cfg mnist or $python train_affine.py --cfg cifar10

Datasets will be downloaded automatically at first run.

Results

Test-set accuracy values

Source model	Source dataset	Target dataset	label map	Accuracy (paper)	Accuracy (our)
ResNet50 v2	Imagenet	MNIST	first 10	94.96 %	95.89 %
ResNet50 v2	Imagenet	CIFAR10	first 10	58.58 %	63.44 %
ResNet50 v2	Imagenet	MNIST	semantic	-	${\color{red}\text{not yet run}}$
ResNet50 v2	Imagenet	CIFAR10	semantic	-	63.41 %
ResNet50 v2	Imagenet	MNIST	affine	-	${\color{red}\text{not yet run}}$
ResNet50 v2	Imagenet	CIFAR10	affine	-	79.55 %

The implemented framework also saves the Program map at best epoch. Below are some examples for comparison with those presented in the original paper:

MNIST + first 10 mapping	CIFAR + first 10 mapping	CIFAR + affine mapping