alstonlo / torch-influence Goto Github PK

Thanks for your wonderful work If Influence Functions are the Answer, Then What is the Question?.

I had trouble reimplementing PBRF on MNIST+MLP since I found that the initial loss can be quite small and unstable in optimization. Besides, I am not quite sure whether my implementation is true. Could you please help me out or release the code of PBRF?

Thanks in advance!

Hello,

Why does LiSSA need a test dataloader in the constructor?

I want to get influence values for a single sample, do i have to initialize the module for each new sample? Is there a straightforward way to initialize the object and use it with different test data?

Thanks,
Galip

Hi,

Thanks for sharing the code.

For the LiSSAInfluenceModule, I could not find any example for it.
Specifically, I wonder what are recommended values for
depth (int) – the recurrence depth
scale (float) – the scaling factor
for widely used datasets like CIFAR-10, CIFAR-100 and ImageNet.

Best,

RC

Hi,

I notice that the influence scores = stest @ training sample gradient; however the original influence function is - stest @ training sample gradient. I'm wondering it is intentionally implemented like this? (Maybe I'm wrong)

Many thanks

Hi,

Thanks for sharing this code, I met a problem when I checked examples/analyze_dogfish.py.

line 22: DOGFISH_EMB_PATH = BASE_DIR / "dataset_dog-fish_embeds.npz"

As I couldn't find dataset_dog-fish_embeds.npz at the link you offered.

Is there any typo of dataset_dog-fish_embeds.npz which should be dataset_koda.npz instead?

Thanks in advance!

Hello,

I am trying to restrict the computations to only use the final layer parameters. Is it enough to override the _model_params() of the BaseInfluenceModule, or do I need to do further modifications.

I am asking to be sure, because it takes a long time to generate influence scores.

Thank you!

Hi,

I'm wondering the parameter selection for LiSSA, especially for the scale param.

Many thanks indeed!

I bumped into the awesome new paper Studying Large Language Model Generalization with Influence Functions (https://arxiv.org/pdf/2308.03296.pdf).

Just wondering if it is possible to extend this framework to reproduce some experiments in this paper. If it is possible, what are the rough steps to implement the Eq 25 and Eq 26 in this repo?

Hi, thank you for developing this package; it has proven incredibly useful for my ongoing project. I've encountered an issue while working with GPU. Specifically, I'm attempting to compute influences using an LSTM model. Interestingly, when using the CPU, everything runs seamlessly. However, upon switching to GPU, I encountered the following error message:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
[<ipython-input-61-99d9acc8245f>](https://localhost:8080/#) in <cell line: 4>()
      3 all_train_idxs = list(range(X_train.shape[0]))
      4 for test_idx in tqdm(test_idxs, desc="Computing Influences"):
----> 5     influences = module.influences(train_idxs=all_train_idxs, test_idxs=[test_idx[0]])
      6     saved_influences.append(influences)
      7 

5 frames
[/usr/local/lib/python3.10/dist-packages/torch/autograd/__init__.py](https://localhost:8080/#) in grad(outputs, inputs, grad_outputs, retain_graph, create_graph, only_inputs, allow_unused, is_grads_batched)
    301         return _vmap_internals._vmap(vjp, 0, 0, allow_none_pass_through=True)(grad_outputs_)
    302     else:
--> 303         return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
    304             t_outputs, grad_outputs_, retain_graph, create_graph, t_inputs,
    305             allow_unused, accumulate_grad=False)  # Calls into the C++ engine to run the backward pass

RuntimeError: cudnn RNN backward can only be called in training mode

I believe this issue might affect others who are utilizing RNN on a GPU. Could you please help me resolve this issue? Thank you!