References#
All citations across the knowledge base resolve into this page. Each entry is back-linked from every page that cites it — click the ↩ arrows next to the key to jump back.
Bibliography source: references.bib in the repo root.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, and others. Towards monosemanticity: decomposing language models with dictionary learning. Transformer Circuits Thread, 2023. URL: https://transformer-circuits.pub/2023/monosemantic-features/index.html.
Ronen Eldan and Yuanzhi Li. Tinystories: how small can language models be and still speak coherent english? arXiv preprint arXiv:2305.07759, 2023. URL: https://arxiv.org/abs/2305.07759, arXiv:2305.07759.
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, and others. Toy models of superposition. Transformer Circuits Thread, 2022. URL: https://transformer-circuits.pub/2022/toy_model/index.html.
Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. arXiv preprint arXiv:2406.04093, 2024. Introduces TopK SAEs. URL: https://arxiv.org/abs/2406.04093, arXiv:2406.04093.
Thomas George, César Laurent, Xavier Bouthillier, Nicolas Ballas, and Pascal Vincent. Fast approximate natural gradient descent in a Kronecker-factored eigenbasis. Advances in Neural Information Processing Systems (NeurIPS), 2018. URL: https://arxiv.org/abs/1806.03884, arXiv:1806.03884.
Roger Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, Evan Hubinger, Kamilė Lukošiūtė, Karina Nguyen, Nicholas Joseph, Sam McCandlish, Jared Kaplan, and Samuel R. Bowman. Studying large language model generalization with influence functions. arXiv preprint arXiv:2308.03296, 2023. URL: https://arxiv.org/abs/2308.03296, arXiv:2308.03296.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In Proceedings of the 34th International Conference on Machine Learning (ICML). 2017. URL: https://arxiv.org/abs/1703.04730, arXiv:1703.04730.
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. In Advances in Neural Information Processing Systems (NeurIPS). 2018. URL: https://arxiv.org/abs/1712.09913, arXiv:1712.09913.
Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. Gemma scope: open sparse autoencoders everywhere all at once on gemma 2. arXiv preprint arXiv:2408.05147, 2024. URL: https://arxiv.org/abs/2408.05147, arXiv:2408.05147.
James Martens and Roger Grosse. Optimizing neural networks with Kronecker-factored approximate curvature. In Proceedings of the 32nd International Conference on Machine Learning (ICML). 2015. URL: https://arxiv.org/abs/1503.05671, arXiv:1503.05671.
Barak A. Pearlmutter. Fast exact multiplication by the Hessian. Neural Computation, 6(1):147–160, 1994.
Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, János Kramár, Neel Shah, and Neel Nanda. Improving dictionary learning with gated sparse autoencoders. arXiv preprint arXiv:2404.16014, 2024. URL: https://arxiv.org/abs/2404.16014, arXiv:2404.16014.
Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Shah. Jumping ahead: improving reconstruction fidelity with JumpReLU sparse autoencoders. arXiv preprint arXiv:2407.14435, 2024. URL: https://arxiv.org/abs/2407.14435, arXiv:2407.14435.
Jennifer Rosser and others. Infusion: reverse engineering influence functions. In 3rd DATA-FM Workshop at ICLR. 2026. Project page: https://jrosser.co.uk/infusion/.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Yu Wu, and Daya Guo. DeepSeekMath: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. Introduces GRPO. URL: https://arxiv.org/abs/2402.03300, arXiv:2402.03300.