Poster
in
Affinity Workshop: Blog Track Session 8
Understanding in-context learning in transformers
Simone Rossi · Rui Yuan · Thomas Hannagan
Halle B #1
Abstract:
We propose a critical review on the phenomenon of In-Context Learning (ICL) in transformer architectures. Focusing on the article Transformers Learn In-Context by Gradient Descent by J. von Oswald et al., published in ICML 2023 earlier this year, we provide detailed explanations and illustrations of the mechanisms involved. We also contribute novel analyses on ICL, discuss recent developments and we point to open questions in this area of research.
Live content is unavailable. Log in and register to view live content