Spaces:
Running
Running
Update research.html
Browse files- research.html +1 -1
research.html
CHANGED
|
@@ -253,7 +253,7 @@
|
|
| 253 |
|
| 254 |
<ul>
|
| 255 |
<li><strong>The Setup:</strong> We are training an ultra-lean <strong>5M parameter Llama model</strong> using Hugging Face Transformers.</li>
|
| 256 |
-
<li><strong>The Data:</strong> Exactly <strong>
|
| 257 |
<br>1. 100% <code>FineWeb-Edu</code>
|
| 258 |
<br>2. 100% <code>DCLM-Edu</code>
|
| 259 |
<br>3. 100% <code>Cosmopedia-v2</code>
|
|
|
|
| 253 |
|
| 254 |
<ul>
|
| 255 |
<li><strong>The Setup:</strong> We are training an ultra-lean <strong>5M parameter Llama model</strong> using Hugging Face Transformers.</li>
|
| 256 |
+
<li><strong>The Data:</strong> Exactly <strong>100 Million tokens</strong> total per run, testing four configurations:
|
| 257 |
<br>1. 100% <code>FineWeb-Edu</code>
|
| 258 |
<br>2. 100% <code>DCLM-Edu</code>
|
| 259 |
<br>3. 100% <code>Cosmopedia-v2</code>
|