Add graphviz for transformer architecture

[klei22]

No description provided.

[Copilot]

Pull request overview

This pull request adds a Graphviz DOT file that visualizes the transformer architecture from a mechanistic interpretability perspective, showing how different weight matrices (readers and writers) interact with the residual stream. The diagram illustrates the flow of information through embedding layers, normalization, attention mechanisms, and MLP layers.

Changes:

• Added a new documentation diagram file that maps the transformer architecture components including Writers (W_E, W_down, W_O), Readers (W_Q, W_K, W_V, W_up, LM Head), the residual stream, and normalization layers

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Inconsistent quotation marks: "d_head" is quoted here but d_model is not quoted in the same line. This is part of the broader inconsistency pattern throughout the file where some dimension names are quoted and others are not.

[Copilot]

Inconsistent quotation marks: "d_ff" is quoted here but d_model is not quoted in the same line. This is part of the broader inconsistency pattern throughout the file where some dimension names are quoted and others are not.

[Copilot]

Inconsistent quotation marks: "vocab_size" is quoted here but d_model is not quoted in the same line. This is part of the broader inconsistency pattern throughout the file where some dimension names are quoted and others are not.

[Copilot]

Inconsistent quotation marks: "vocab_size" is quoted here but d_model is not quoted in the same line. This is part of the broader inconsistency pattern throughout the file where some dimension names are quoted and others are not.

[Copilot]

Extra space after closing parenthesis. The line ends with "vocab_size )" but should be "vocab_size)" for consistency with other similar lines (see lines 38, 56, 109, 127, 145, 163, 181).

Suggested change

	<TD COLSPAN="2">axes: x = d_model, y = vocab_size </TD>
	<TD COLSPAN="2">axes: x = d_model, y = vocab_size</TD>

[Copilot]

Inconsistent use of quotation marks around dimension names. Lines 24, 42, 60, and 131 use quotation marks around "vocab_size", "d_ff", and "d_head", but other dimension names like d_model, d_head (in other places) are not quoted. Either quote all dimension names consistently or remove quotes from all for consistency throughout the diagram.

[Copilot]

Inconsistent quotation marks: "d_ff" is quoted here but d_model is not quoted in the same line. This is part of the broader inconsistency pattern throughout the file where some dimension names are quoted and others are not.

Suggested change

	<B>"d_ff" row vectors</B> in &#8477;<SUP>d_model</SUP>
	<B>d_ff row vectors</B> in &#8477;<SUP>d_model</SUP>

[Copilot]

Inconsistent quotation marks: "d_head" is quoted here but d_model is not quoted in the same line. This is part of the broader inconsistency pattern throughout the file where some dimension names are quoted and others are not.

[Copilot]

Inconsistent quotation marks: "vocab_size" is quoted here but d_model is not quoted in the same line. This is part of the broader inconsistency pattern throughout the file where some dimension names are quoted and others are not.

Suggested change

	<B>Reader: LM Head</B> &nbsp; (d_model × "vocab_size")
	</TD>
	</TR>
	<TR>
	<TD COLSPAN="2">axes: x = "vocab_size", y = d_model</TD>
	<B>Reader: LM Head</B> &nbsp; (d_model × vocab_size)
	</TD>
	</TR>
	<TR>
	<TD COLSPAN="2">axes: x = vocab_size, y = d_model</TD>