RoBERTa

RoBERTa: A Robustly Optimized BERT Pretraining Approach. RoBERTa iterates on BERT’s pretraining procedure, including training the model longer, with bigger batches over more data; removing the next sentence prediction objective; training on longer sequences; and dynamically changing the masking pattern applied to the training data. See the associated paper for more details.


References in zbMATH (referenced in 28 articles )

Showing results 1 to 20 of 28.
Sorted by year (citations)

1 2 next

  1. Aghahadi, Zeinab; Talebpour, Alireza: Avicenna: a challenge dataset for natural language generation toward commonsense syllogistic reasoning (2022)
  2. Feiliang Ren, Yongkang Liu, Bochao Li, Shilei Liu, Bingchao Wang, Jiaqi Wang, Chunchao Liu, Qi Ma: An Understanding-Oriented Robust Machine Reading Comprehension Model (2022) arXiv
  3. Koto, Fajri; Baldwin, Timothy; Lau, Jey Han: FFCI: a framework for interpretable automatic evaluation of summarization (2022)
  4. Kriebel, Johannes; Stitz, Lennart: Credit default prediction from user-generated text in peer-to-peer lending using deep learning (2022)
  5. Liu, Jingping; Chen, Tao; Wang, Chao; Liang, Jiaqing; Chen, Lihan; Xiao, Yanghua; Chen, Yunwen; Jin, Ke: VoCSK: verb-oriented commonsense knowledge mining with taxonomy-guided induction (2022)
  6. Li, Zuchao; Zhou, Junru; Zhao, Hai; Zhang, Zhisong; Li, Haonan; Ju, Yuqi: Neural character-level syntactic parsing for Chinese (2022)
  7. Loureiro, Daniel; Mário Jorge, Alípio; Camacho-Collados, Jose: LMMS reloaded: transformer-based sense embeddings for disambiguation and beyond (2022)
  8. Md Tanvirul Alam, Dipkamal Bhusal, Youngja Park, Nidhi Rastogi: CyNER: A Python Library for Cybersecurity Named Entity Recognition (2022) arXiv
  9. Xing, Bowen; Tsang, Ivor W.: Out of context: a new clue for context modeling of aspect-based sentiment analysis (2022)
  10. Zeng, Zhiyuan; Xiong, Deyi: Unsupervised and few-shot parsing from pretrained language models (2022)
  11. Zhang, Hongming; Liu, Xin; Pan, Haojie; Ke, Haowen; Ou, Jiefu; Fang, Tianqing; Song, Yangqiu: ASER: towards large-scale commonsense knowledge acquisition via higher-order selectional preference over eventualities (2022)
  12. Zhang, Wei; Xue, Ziyun; Li, Zhong; Yin, Huichao: DCE-dforest: a deep forest model for the prediction of anticancer drug combination effects (2022)
  13. Juan Manuel Pérez, Juan Carlos Giudici, Franco Luque: pysentimiento: A Python Toolkit for Sentiment Analysis and SocialNLP tasks (2021) arXiv
  14. Kiritchenko, Svetlana; Nejadgholi, Isar; Fraser, Kathleen C.: Confronting abusive language online: a survey from the ethical and human rights perspective (2021)
  15. Nelson Tavares de Sousa, Wilhelm Hasselbring: JavaBERT: Training a transformer-based model for the Java programming language (2021) arXiv
  16. Sai Muralidhar Jayanthi, Kavya Nerella, Khyathi Raghavi Chandu, Alan W Black: CodemixedNLP: An Extensible and Open NLP Toolkit for Code-Mixing (2021) arXiv
  17. Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, Ge Li, Lidong Zhou, Linjun Shou, Long Zhou, Michele Tufano, Ming Gong, Ming Zhou, Nan Duan, Neel Sundaresan, Shao Kun Deng, Shengyu Fu, Shujie Liu: CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation (2021) arXiv
  18. Škrlj, Blaž; Martinc, Matej; Lavrač, Nada; Pollak, Senja: autoBOT: evolving neuro-symbolic representations for explainable low resource text classification (2021)
  19. Tao Gui, Xiao Wang, Qi Zhang, Qin Liu, Yicheng Zou, Xin Zhou, Rui Zheng, Chong Zhang, Qinzhuo Wu, Jiacheng Ye, Zexiong Pang, Yongxin Zhang, Zhengyan Li, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xinwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Bolin Zhu, Shan Qin, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang: TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021) arXiv
  20. Tripathy, Jatin Karthik; Sethuraman, Sibi Chakkaravarthy; Cruz, Meenalosini Vimal; Namburu, Anupama; P., Mangalraj; R., Nandha Kumar; S., Sudhakar Ilango; Vijayakumar, Vaidehi: Comprehensive analysis of embeddings and pre-training in NLP (2021)

1 2 next