Lexical Reference Frames: Interpreting LM Residual Streams in Native Semantics
The vocabulary of a language model is its native semantics: the embedding and the unembedding fix the meaning of a state at the two ends of the residual stream. Between them, existing methods either reuse a fixed lens or treat the states a…
