History of Transformers Timeline

(2017 – Present)

Transformers are a deep learning architecture introduced by Google researchers in 2017 that rely on self-attention mechanisms to process sequential data. Unlike earlier recurrent and convolutional neural networks, Transformers process entire sequences in parallel, enabling faster training and better handling of long-range dependencies. This breakthrough led to landmark models such as BERT, GPT-3, and ChatGPT, revolutionizing natural language processing, computer vision, and generative AI. Less

Origins & Early Development

Mar 2003

2004

Feb 2005

Jul 30, 2005

Michael Bay Films

2007

May 2007

Jun 2007

Sep 2007

Oct 2007

Nov 2007

2008

2008

2009

Jun 2009

Oct 1, 2009

2010

2010

Oct 2010

Jul 1, 2011

Feb 2012

Feb 2012

Nov 2012

2013

Jan 8, 2013

Mar 2013

Mar 26, 2013

May 1, 2013

May 6, 2013

Jun 2013

Jun 27, 2014

Jun 23, 2017

Post-Bay Films

2015 - Aug 2017

Mar 2015

Jul 2015

Jul 2015

Jan 2016

Apr 2016

2017

Jul 2017

Dec 2018

Dec 21, 2018

Jun 9, 2023

Future Projects

Mar 2019

Mar 2019

Jan 2020

May 2020

Mar 2021

Apr 2021

Jun 2021

Jul 2021

Sep 2021

2022

Feb 2022

Jun 2023

Jul 2023

Apr 2024

Jun 2024

Nov 20, 2024

Jun 2025

Jun 2025

Jul 2025

Jun 2026

Key Facts

  1. The Transformer architecture was introduced in the 2017 paper 'Attention Is All You Need' by Vaswani et al. at Google Brain.
  2. Transformers use self-attention mechanisms instead of recurrence, allowing parallel processing of entire input sequences.
  3. BERT (2018) demonstrated the power of bidirectional pre-training for language understanding tasks.
  4. OpenAI's GPT series, scaling from GPT-1 (2018) to GPT-3 (2020) and beyond, showed that larger Transformers dramatically improve generative capabilities.
  5. Transformers have expanded beyond NLP into computer vision (Vision Transformer, 2020), speech, biology (AlphaFold), and multimodal AI systems.

Source

This History of Transformers timeline was generated with the help of AI, using information found on the internet.

We work hard to keep these timelines accurate, but mistakes do get through. If you spot one, email us at [email protected] and we'll fix it for future visitors.

Want a timeline like this one?

Generate yours with AI or build it from scratch, for free.