Language Models are Unsupervised Multitask Learners ​

← Back to topic

Authors: Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever
Year: 2019
Journal: OpenAI Blog
DOI: 10.48550/arXiv.1911.02116
Publisher: https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf

Keywords: gpt-2, language model, zero-shot

Abstract ​

We demonstrate that language models begin to learn these tasks without any explicit supervision when trained on a new dataset of millions of webpages called WebText.

Cite this paper ​

@misc{gpt22019,
  title  = {Language Models are Unsupervised Multitask Learners},
  author = {Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever},
  year   = {2019},
  journal = {OpenAI Blog},
  doi    = {10.48550/arXiv.1911.02116},
  url    = {https://doi.org/10.48550/arXiv.1911.02116},
}

Source files ​

• metadata.json
• paper.bib