Update README.md
Browse files
README.md
CHANGED
|
@@ -71,4 +71,6 @@ python applyweights.py \
|
|
| 71 |
| `--alpha` | Controls the variance scale multiplier for the `down_proj` update. | `0.02` |
|
| 72 |
| `--gamma-cap` | Sets the maximum fractional adjustment allowed for the `gate_proj`. | `0.05` |
|
| 73 |
|
| 74 |
-
# Note:
|
|
|
|
|
|
|
|
|
| 71 |
| `--alpha` | Controls the variance scale multiplier for the `down_proj` update. | `0.02` |
|
| 72 |
| `--gamma-cap` | Sets the maximum fractional adjustment allowed for the `gate_proj`. | `0.05` |
|
| 73 |
|
| 74 |
+
# Note:
|
| 75 |
+
|
| 76 |
+
The default values for Alpha and Gamma are extremely conservative, they will not influence model behavior much at the defaults; it can and should, for pronounced change closer to Claude be pushed to around Alpha ~32 and Gamma-cap ~25 without breaking.
|