random projection in influence functions

[haochend413]

Description

Add random projection for some IF attributors

• IF_Explicit
• IF_EKFAC
• IF_DataInf

Closes #227

[haochend413]

I ran the IF attributors on the influence_functions_lds example after adding random projection.

Explicit on mnist_lr: ? -> 0.4851
EKFAC on mnist_mlp: 0.1151 -> 0.1140
DataInf on mnist_lr: 0.3173->0.2978

[sleepymalc]

This seems redundant:

https://github.com/TRAIS-Lab/dattri/pull/231/changes#diff-e756f81a0441b8319db94cc2a72041edee5cee3936e2150a424ada0afb6803bbR21

[haochend413]

fixed

[TonyZhou05]

Could we check the attribute function call to make sure that the projection has some effect on larger models such as mobilenet_v2? Currently calculating the influence score on any IF attributor is really slow (No progress on A100 GPU for mobilenet_v2 in 10 minutes) I can create this as a separate bug if needed.

Code snippets

train_subset = Subset(full_train_noisy, range(0, 5000))
train_loader = DataLoader(train_subset, batch_size=32, shuffle=False)
val_loader = DataLoader(val_subset, batch_size=32, shuffle=False)
with torch.no_grad():
    # This steps won't finish
    influence_scores = if_attributor.attribute(train_loader, val_loader)