✕
Set Demo Prompt
Ask a question. The demo will fetch response tokens from OpenAI via the same Lambda API.
Start Demo
Dismiss
☰
Transformer models
Attention
GPT-2
GPT-3
Llama-8B
Llama-70B
Llama-405B
Input token length (n)
Vocab size
Embedding dim (d)
Heads (h)
Head dim (derived d/h)
Layers
Parameters
OFF
Architecture Overview
Tokenization
Embedding
Input
100001
100317
100946
101312
101888
102224
<Q>
What
is
...
<A>
The
70,000
tokens
Embedding
Positional
Encoding
40 TIMES
Transformer
Block
(Attention)
Mixing & FFN
Unembedding
Prediction
→
→
→
→
→
'of'
1347
-
-
Params (B)
FLOPs (TFLOPs)
-
-
-
-
-
-
-
-
-
-
-
-
Details