Edit RoBERTa is an extension of BERT with changes to the pretraining procedure. The modifications include: training the model longer, with bigger batches, over more dataNevertheless, in the vocabulary size growth in RoBERTa allows to encode almost any word or subword without using the unknown token,