test error

[ztx313]

def load(self, args):
    # Load a trained model and vocabulary that you have fine-tuned
    assert args.reload_from>=0, "please specify the checkpoint iteration in args.reload_from"
    output_dir = os.path.join(f"./output/{args.model}/{args.model_size}/models/", f'checkpoint-{args.reload_from}')
    self.model = DialogBERT.from_pretrained(output_dir)
    self.model.to(args.device)

def from_pretrained(self, model_dir):
    self.encoder_config = BertConfig.from_pretrained(model_dir)
    self.tokenizer = BertTokenizer.from_pretrained(path.join(model_dir, 'tokenizer'), do_lower_case=True)
    self.utt_encoder = BertForPreTraining.from_pretrained(path.join(model_dir, 'utt_encoder'))
    self.context_encoder = BertForSequenceClassification.from_pretrained(path.join(model_dir, 'context_encoder'))
    self.context_mlm_trans = BertPredictionHeadTransform(self.encoder_config)
    self.context_mlm_trans.load_state_dict(torch.load(path.join(model_dir, 'context_mlm_trans.pkl')),strict= False)
    self.context_order_trans = SelfSorting(self.encoder_config.hidden_size)
    self.context_order_trans.load_state_dict(torch.load(path.join(model_dir, 'context_order_trans.pkl')), strict= False)
    self.decoder_config = BertConfig.from_pretrained(model_dir)
    self.decoder = BertLMHeadModel.from_pretrained(path.join(model_dir, 'decoder'))

File "D:\NLP\DialogBERT-master\solvers.py", line 77, in load
self.model.to(args.device)
AttributeError: 'NoneType' object has no attribute 'to'
DialogBERT.from_pretrained is none ,how can i solve it?

[guxd]

I modified
self.model=DialogBERT.from_pretrained(output_dir)
to
self.model.from_pretrained(output_dir)
and have committed to github. Please check it out.

[ztx313]

thanks for your answer,but the code meet a new error, I try to print(self.context_encoder(None, ctx_attn_mask, None, None, None, utt_encodings)), and the result is that (tensor([[ 0.0142, -0.0242]], device='cuda:0'),) , only return one tensor def context_encoding(self, context, utts_attn_mask, ctx_attn_mask): #with torch.no_grad(): utt_encodings = self.utt_encoding(context, utts_attn_mask) context_hiddens, pooled_output, *_ = self.context_encoder( None, ctx_attn_mask, None, None, None, utt_encodings) # context_hiddens:[batch_size x ctx_len x dim]; pooled_output=[batch_size x dim] return context_hiddens, pooled_output Traceback (most recent call last): File "main.py", line 115, in <module> main() File "main.py", line 110, in main results = solver.evaluate(args).to(device) File "D:\NLP\DialogBERT-master\solvers.py", line 139, in evaluate result, generated_text = Learner().run_eval(args, self.model, test_set) File "D:\NLP\DialogBERT-master\learner.py", line 332, in run_eval loss = model1.validate(*batch_gpu) File "D:\NLP\DialogBERT-master\models\dialogBERT.py", line 361, in validate context, context_utts_attn_mask, context_attn_mask, context_lm_targets, context_position_perm_id, context_position_ids, response) File "D:\NLP\DialogBERT-master\models\dialogBERT.py", line 275, in train_decoder context_hiddens, context_encoding = self.context_encoding(context, context_utts_attn_mask, context_attn_mask) File "D:\NLP\DialogBERT-master\models\dialogBERT.py", line 223, in context_encoding None, ctx_attn_mask, None, None, None, utt_encodings) ValueError: not enough values to unpack (expected at least 2, got 1) Closing remaining open files:./data/dailydial\test.h5...done

[guxd]

Maybe the latest checkpoint of BERT sets the "return_dict" to True by default.
So you can first use

output = self.context_encoder(None, ctx_attn_mask, None, None, None, utt_encodings)

Here the returned variable output can be a dictionary type.
Then add this line:

context_hiddens, pooled_output = output.last_hidden_state, output.pooler_output

[ztx313]

hi! thanks for your reply! Following your way,I run the code, output = self.context_encoder(None, ctx_attn_mask, None, None, None, utt_encodings) context_hiddens, pooled_output = output.last_hidden_state, output.pooler_output but the output type is tuple and the error is : File "D:\NLP\DialogBERT-master\models\dialogBERT.py", line 221, in context_encoding context_hiddens, pooled_output = output.last_hidden_state, output.pooler_output AttributeError: 'tuple' object has no attribute 'last_hidden_state'

…

------------------ 原始邮件 ------------------ 发件人: "guxd/DialogBERT" ***@***.***&gt;; 发送时间:&nbsp;2021年11月10日(星期三) 晚上8:07 ***@***.***&gt;; ***@***.******@***.***&gt;; 主题:&nbsp;Re: [guxd/DialogBERT] test error (Issue #9) Maybe the latest checkpoint of BERT sets the "return_dict" to True by default. So you can first use output = self.context_encoder(None, ctx_attn_mask, None, None, None, utt_encodings) Here the returned variable output can be a dictionary type. Then add this line: context_hiddens, pooled_output = output.last_hidden_state, output.pooler_output — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[guxd]

Then, what is the size of this tuple?
How about

(context_hiddens, pooled_output, *_) = self.context_encoder(
            None, ctx_attn_mask, None, None, None, utt_encodings)

[guxd]

How about

output = self.context_encoder(None, ctx_attn_mask, None, None, None, utt_encodings)
context_hiddens, pooled_output = output[0], output[1]

[ztx313]

hi! the tuple is:&nbsp;(tensor([[0.0288, 0.2465]], device='cuda:0'),) only return one parameter

…

------------------&nbsp;原始邮件&nbsp;------------------ 发件人: "guxd/DialogBERT" ***@***.***&gt;; 发送时间:&nbsp;2021年11月10日(星期三) 晚上8:34 ***@***.***&gt;; ***@***.******@***.***&gt;; 主题:&nbsp;Re: [guxd/DialogBERT] test error (Issue #9) Then, what is the size of this tuple? How about (context_hiddens, pooled_output, *_) = self.context_encoder( None, ctx_attn_mask, None, None, None, utt_encodings) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[ztx313]

the error is the following: &nbsp; File "D:\NLP\DialogBERT-master\models\dialogBERT.py", line 225, in context_encoding &nbsp; &nbsp; context_hiddens, pooled_output = output[0], output[1] IndexError: tuple index out of range

…

------------------&nbsp;原始邮件&nbsp;------------------ 发件人: "guxd/DialogBERT" ***@***.***&gt;; 发送时间:&nbsp;2021年11月10日(星期三) 晚上8:37 ***@***.***&gt;; ***@***.******@***.***&gt;; 主题:&nbsp;Re: [guxd/DialogBERT] test error (Issue #9) How about output = self.context_encoder(None, ctx_attn_mask, None, None, None, utt_encodings) context_hiddens, pooled_output = output[0], output[1] — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[guxd]

Did you use the default transformer in the root folder of this repository?
Or did you install the latest version of Transformers by huggineface on your machine?
You are recommended to use the one provided in this repository.

[ztx313]

I use the default transformer in the root folder of this repository and I created a new environment to test ,but it's the same error!

…

------------------&nbsp;原始邮件&nbsp;------------------ 发件人: "guxd/DialogBERT" ***@***.***&gt;; 发送时间:&nbsp;2021年11月10日(星期三) 晚上8:46 ***@***.***&gt;; ***@***.******@***.***&gt;; 主题:&nbsp;Re: [guxd/DialogBERT] test error (Issue #9) Did you use the default transformer in the root folder of this repository? Or did you install the latest version of Transformers by huggineface on your machine? You are recommended to use the one provided in this repository. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.