Update README.md

README.md

@@ -2,3 +2,71 @@
 license: mit
 ---
 
+# 🛠️ Gemma 4 31B SwiGLU Fusion Toolkit
+
+This toolkit provides a robust pipeline for generating and injecting custom, functionally correct MLP weights into a Gemma 4 31B safetensors model. It is specifically designed to handle Gemma 4's unique architectural features—namely its interleaved sliding-window attention (SWA) and global full-context layers—while preserving model stability through advanced alignment and fusion techniques.
+
+## ✨ Features
+
+*   **Targeted Architecture:** Explicitly accounts for Gemma 4's interleaved attention structure (5 SWA + 1 Global per period).
+*   **Double-Wide MLP Support:** Automatically detects and generates intermediate sizes for global full-context layers.
+*   **Sign-Symmetric Alignment:** Generates weights using paired frame vectors ($+gc+gc$ and $-gc-gc$) to ensure the net activation remains zero-centered, preventing distribution shifts.
+*   **Shape-Contoured Fusion (SCF):** Instead of naive addition, the fusion process smoothly merges synthetic weights into the model's existing learned contours, minimizing destabilization.
+*   **Controlled Adjustment:** Implements dynamic scaling ($\alpha$) and clamping ($\gamma$-cap) to control the magnitude of weight updates.
+*   **Selective Fusion:** Allows users to target specific layers for modification.
+
+## ⚙️ Pipeline Overview
+
+The toolkit operates in two stages, assuming you have generated the source Neuron and derivitive Neurons which are re-usable:
+
+### 1. The Generator (`weights.py`)
+
+This script creates synthetic, functionally correct MLP projections (`gate_proj`, `up_proj`, `down_proj`) based on reference source neurons. It maps piece-wise linear behaviors into the non-linear SwiGLU block without introducing mean-shift destabilization.
+
+*   **Key Function:** Generates balanced, sign-symmetric delta weights.
+*   **Output:** Saves the generated delta weights to an output directory.
+
+### 2. The Fuser (`applyweights.py`)
+
+This script applies the safetensor delta files generated by `weights.py` to your existing Gemma 4 31B model using Shape-Contoured Fusion (SCF).
+
+*   **`down_proj` (Contoured Multiplicative Delta):** The update is scaled by the existing weight profile and variance-normalized to ensure smooth integration.
+    $$\text{W}_{\text{down}} = \text{W}_{\text{down}} + (\alpha_{\text{dynamic}} \cdot \Delta_{\text{down}} \cdot \text{W}_{\text{down}})$$
+*   **`gate_proj` (Multiplicative Gamma Scaling):** Applies a clamped fractional adjustment to prevent unbounded activation spikes.
+    $$\text{W}_{\text{gate}} = \text{W}_{\text{gate}} \cdot (1 + \text{clamp}(\Delta_{\text{gate}}, -\gamma, \gamma))$$
+*   **`up_proj` (Untouched):** The linear value path is intentionally skipped during fusion.
+
+## 🚀 Usage
+
+### Stage 1: Generating Delta Weights
+
+Use `weights.py` to generate the synthetic delta weights.
+
+```bash
+python weights.py \
+    --base-model /path/to/original_gemma_model \
+    --output-dir generated_weights \
+    --seed 42
+```
+
+### Stage 2: Applying Weights (Fusion)
+
+Use `applyweights.py` to merge the generated weights into your target model.
+
+```bash
+python applyweights.py \
+    --model /path/to/original_gemma_model \
+    --weights generated_weights \
+    --output /path/to/fused_gemma_model \
+    --alpha 0.02 \
+    --gamma-cap 0.05
+```
+
+## 💡 Helpful Flags
+
+| Flag | Description | Default |
+| :--- | :--- | :--- |
+| `--dry-run` | Simulates the fusion process, identifying missing keys, calculating partial coverage matrices, and verifying sharded indexing without writing files. | N/A |
+| `--layers` | A space-separated list of specific layer indices to fuse (e.g., `--layers 5 11 17`), skipping the rest. | None |
+| `--alpha` | Controls the variance scale multiplier for the `down_proj` update. | `0.02` |
+| `--gamma-cap` | Sets the maximum fractional adjustment allowed for the `gate_proj`. | `0.05` |