Abstract

The question of why ChatGPT-like generative AI works so well -- and when it  won't -- has no clear answer as yet. The field of mechanistic interpretability has identified mesoscale circuits and head roles in simpler GPT versions. However, there is a lack of bottom-up insight starting from the microscale of the GPT's 'atom': the Attention head. This paper starts filling this gap, by focusing on the dynamical behavior of a very basic Attention head. Operating alone, it shows a tipping point n* in its output -- and we analyze a mathematical formula that predicts n* as a function of the user's prompt and the training embeddings. Though obviously far from any commercial generative AI system, our results show that this purposely over-simplified example nevertheless yields output content dynamics that mimic some large-scale LLM behaviors. We comment on the potential usefulness of our findings in the real-world domain of insurance, as well as the domains of health and judicial systems. 

Statistics