1.

What do you mean by Masked language modelling?

Answer»

Masked language MODELLING is an NLP technique for extracting the output from a contaminated input. Learners can use this approach to master deep representations in downstream tasks. Using this NLP technique, you may PREDICT a WORD based on the other words in the sentence.

The FOLLOWING is the process for Masked language modelling:

  • Our text is tokenized. We start with text TOKENIZATION, just as we would with transformers.
  • Make a tensor of labels. We're using a labels tensor to calculate loss against — and optimise towards — as we train our model.
  • Tokens in input ids are masked. We can mask a random selection of tokens now that we've produced a duplicate of input ids for labels.
  • Make a loss calculation. We use our model to process the input ids and labels tensors and determine the loss between them.


Discussion

No Comment Found

Related InterviewSolutions