Skip to content

Grad Error! #6

@JackAILab

Description

@JackAILab

When I run the code train.py, some error occurred in "default_runner.py", specifically, loss.backward()

The followings are the detail debug information:
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [2507, 256]], which is output 0 of ReluBackward0, is at version 1; expected version 0 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

I can find where the grad have been modified? Anyone meet this issue? Thank you for your help!

image

My environments are:
image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions