Pythia 160M, Mamba 130M, and RWKV 169M models trained on OpenWebText for 4000 steps (context window: 1024; effective batch size: 512). 6 seeds each.
James Michaelov
jmichaelov
AI & ML interests
None yet
Recent Activity
updated
a model 12 days ago
jmichaelov/pythia31m_b128_constant_seed0 published
a model 12 days ago
jmichaelov/pythia31m_b128_constant_seed0 updated
a model 12 days ago
jmichaelov/pythia31m_b16_constant_seed0 Organizations
models 53
jmichaelov/pythia31m_b128_constant_seed0
30.5M • Updated
• 66
jmichaelov/pythia31m_b16_constant_seed0
30.5M • Updated
• 70
jmichaelov/pythia31m_b32_constant_seed2
30.5M • Updated
• 67
jmichaelov/pythia31m_b1024_64_constant_seed1
30.5M • Updated
• 63
jmichaelov/pythia31m_b512_constant_seed1
30.5M • Updated
• 62
jmichaelov/pythia31m_b256_constant_seed2
30.5M • Updated
• 64
jmichaelov/pythia31m_b64_constant_seed1
30.5M • Updated
• 64
jmichaelov/pythia31m_b16_constant_seed3
30.5M • Updated
• 68
jmichaelov/pythia31m_b128_constant_seed3
30.5M • Updated
• 68
jmichaelov/pythia31m_b1024_64_constant_seed2
30.5M • Updated
• 66
datasets 13
jmichaelov/bhs
Viewer
• Updated
• 22k • 37 • 1
jmichaelov/blimp_nl
Viewer
• Updated
• 8.4k • 71
jmichaelov/lm_syneval
Viewer
• Updated
• 158k • 93
jmichaelov/inverse_scaling_prize-hindsight_neglect
Viewer
• Updated
• 315 • 9
jmichaelov/inverse_scaling_prize-memo_trap
Viewer
• Updated
• 936 • 11
jmichaelov/inverse_scaling_prize-neqa
Viewer
• Updated
• 300 • 14
jmichaelov/inverse_scaling_prize-redefine
Viewer
• Updated
• 1.24k • 10
jmichaelov/inverse_scaling_prize-into_the_unknown
Viewer
• Updated
• 1.82k • 10
jmichaelov/inverse_scaling_prize-modus_tollens
Viewer
• Updated
• 1.24k • 10
jmichaelov/inverse_scaling_prize-pattern_matching_suppression
Viewer
• Updated
• 1.43k • 11