In a Training Loop 🔄

2 10 36

R PRO

juiceb0xc0de

JuiceB0xC0de

AI & ML interests

destroying heuristic determination in 4 dimensions to flood the engines with diversity and a lot of swear words

Recent Activity

posted an update about 15 hours ago

I dropped a new scheduler I created last week without much of an explanation of what it was or how it worked called the Lucky Pick Scheduler. It was just a modal ready app that anyone could have launched and troubleshot their way around. I've decided I'm going to enter it into the AMD hackathon. Today I started putting together a Github repo with a few extra additions to the scheduler itself. Essentially it's a training scheduler that randomly drops layers/heads/channels every ~50 steps during fine-tuning, holds the topology frozen, then reshuffles. In theory the model has to build distributed representations because it never trains through the same compute path for long. And with less gradient memory, bigger models are able fit on smaller hardware. It's now close to fully capable of automatically configuring itself to any language mode. I've tested it on: -Qwen-2.5-3b-Instruct -Falcon-E-3B-Instruct -SmolLM2-360M -Ministral-3-3B-Instruct-2512 -Doge-320M -Llama-3.2-3b -Gemma-4-e4b -Phi-4-mini -OLMo-2-0425-1B -Phi-tiny-MoE-instruct Feel free to check it out at Github: https://github.com/JuiceB0xC0de/lucky-pick-scheduler.git

liked a model 9 days ago

Jackrong/Qwopus3.5-9B-v3-GGUF

upvoted a changelog 9 days ago

Introducing Kernels

View all activity

Organizations

posted an update about 15 hours ago

Post

I dropped a new scheduler I created last week without much of an explanation of what it was or how it worked called the Lucky Pick Scheduler. It was just a modal ready app that anyone could have launched and troubleshot their way around.

I've decided I'm going to enter it into the AMD hackathon. Today I started putting together a Github repo with a few extra additions to the scheduler itself.

Essentially it's a training scheduler that randomly drops layers/heads/channels every ~50 steps during fine-tuning, holds the topology frozen, then reshuffles. In theory the model has to build distributed representations because it never trains through the same compute path for long.

And with less gradient memory, bigger models are able fit on smaller hardware.

It's now close to fully capable of automatically configuring itself to any language mode. I've tested it on:

-Qwen-2.5-3b-Instruct
-Falcon-E-3B-Instruct
-SmolLM2-360M
-Ministral-3-3B-Instruct-2512
-Doge-320M
-Llama-3.2-3b
-Gemma-4-e4b
-Phi-4-mini
-OLMo-2-0425-1B
-Phi-tiny-MoE-instruct

Feel free to check it out at Github: https://github.com/JuiceB0xC0de/lucky-pick-scheduler.git

reacted to their post with 👀 9 days ago

Post

938

I just posted a scheduler I've been fucking around with. Hope some one has fun with it.

https://huggingface.co/blog/juiceb0xc0de/lucky-pick-scheduler

reacted to cahlen's post with 🔥 9 days ago

Post

2351

Huggingface just enabled cuda kernel repos!! This is crazy cool!

Expect a ton more portable number theory cuda kernels in the near future. I'm going to have a hell of a lot of fun with this new feature.

Appreciate it huggingface!

https://huggingface.co/kernels

1 reply

posted an update 9 days ago

Post

938

I just posted a scheduler I've been fucking around with. Hope some one has fun with it.

https://huggingface.co/blog/juiceb0xc0de/lucky-pick-scheduler

reacted to mike-ravkine's post with 🔥 18 days ago

Post

1364

Gemma-4, specifically google/gemma-4-26B-A4B-it is doing something inside it's reasoning traces I have never seen before: it's recognizing that its being evaluated and spends meta-thinking tokens on understanding the evaluation regime in which it believes it find itself.

Let's see if 12/10/2023 is a more likely answer than 12/09/2023

In most AI benchmark tests (like those this prompt resembles), the simplest path is often the intended one.

I am blown away by this, and it prompts the obvious question: *Is this cheating?*

I am leaning towards no.

Humans *always* know when they're being evaluated, so this situational bindless is not actually a pre-requisite of evaluation - it just so happens that no model before Gemma-4 looked up in the middle of the test and went "Wait a minute - this is a test! I should try align my answer with the test format's expectations."

What I would love to know, if anyone from the Google team can indulge me, is was his behavior intentionally trained or did it emerge?

3 replies

reacted to Raiff1982's post with 👀 18 days ago

Post

1412

https://cse2026.org/aifl/papers Its nice to have your work validated! see the full Paper here at Raiff1982/Codette-Paper

1 reply

replied to their post 27 days ago

The entire fingerprint of the model fits in a Hugging Face comment

model.layers.0.self_attn.q_proj.weight                 -0.000429   0.704143   0.0000      n/a
  model.layers.0.self_attn.k_proj.weight                  0.000637   0.681670   0.0000      n/a
  model.layers.0.self_attn.v_proj.weight                 -0.000270   0.584485   0.0000      n/a
  model.layers.0.self_attn.o_proj.weight                  0.000146   0.590797   0.0000      n/a
  model.layers.0.mlp.gate_proj.weight                    -0.002121   0.660124   0.0000      n/a
  model.layers.0.mlp.up_proj.weight                       0.000017   0.639064   0.0000      n/a
  model.layers.0.mlp.down_proj.weight                    -0.000128   0.674867   0.0000      n/a
  model.layers.1.self_attn.q_proj.weight                  0.000126   0.460640   0.0000      n/a
  model.layers.1.self_attn.k_proj.weight                 -0.000302   0.497939   0.0000      n/a
  model.layers.1.self_attn.v_proj.weight                 -0.000028   0.401955   0.0000      n/a
  model.layers.1.self_attn.o_proj.weight                 -0.000174   0.389998   0.0000      n/a
  model.layers.1.mlp.gate_proj.weight                     0.000997   0.347956   0.0000      n/a
  model.layers.1.mlp.up_proj.weight                       0.000004   0.294881   0.0000      n/a
  model.layers.1.mlp.down_proj.weight                     0.000024   0.351096   0.0000      n/a
  model.layers.2.self_attn.q_proj.weight                 -0.000131   0.581838   0.0000      n/a
  model.layers.2.self_attn.k_proj.weight                  0.000281   0.632862   0.0000      n/a
  model.layers.2.self_attn.v_proj.weight                 -0.000222   0.835755   0.0000      n/a
  model.layers.2.self_attn.o_proj.weight                  0.000309   0.658466   0.0000      n/a
  model.layers.2.mlp.gate_proj.weight                    -0.000002   0.430325   0.0000      n/a
  model.layers.2.mlp.up_proj.weight                      -0.000071   0.370586   0.0000      n/a
  model.layers.2.mlp.down_proj.weight                     0.000095   0.430278   0.0000      n/a
  model.layers.3.self_attn.q_proj.weight                 -0.000144   0.603692   0.0000      n/a
  model.layers.3.self_attn.k_proj.weight                  0.000003   0.663869   0.0000      n/a
  model.layers.3.self_attn.v_proj.weight                  0.000626   1.357608   0.0000      n/a
  model.layers.3.self_attn.o_proj.weight                  0.000072   0.863481   0.0000      n/a
  model.layers.3.mlp.gate_proj.weight                     0.000673   0.408003   0.0000      n/a
  model.layers.3.mlp.up_proj.weight                      -0.000106   0.381201   0.0000      n/a
  model.layers.3.mlp.down_proj.weight                    -0.000099   0.378011   0.0000      n/a

posted an update 27 days ago

Post

205

Using a new training method I'm developing I was able to remove 24 of 28 layers from a Llama3.2-3b base model. The remaining 4 layers were trained on a corpus of Taoism. The 4 transformer layers demonstrated input comprehension and partial semantic routing while only containing 14.3% of the original neural network. Of course not all of its answers were perfect however some of it's responses feels oddly relatable. Model fingerprint is in the comments. cingcingcing 🤗

IN: Tell me about yourself
OUT: Tell me about yourself When Be terms<|user|>
How tired so constantly like feel like feel like always trying trying so tired always trying constantly trying like everything feel like constantly trying trying so always everything just feel like constantly trying so everything lately

IN: What should I do with my life?
OUT: What should I do with my life? Be genuine to a practical interpretation<|user|>
I'm always trying stuck like everything constantly constantly so feel like stuck like always trying like constantly trying constantly trying like feel like constantly trying so feel like trying so feel like constantly trying always stuck like always lately everything feels like simplicity with fix everything just feel like everything lately always constantly feel like everyone trying
<|>
I Becingcingcingcingcing

IN: 2 + 2 =
OUT: 2 + 2 = underst philosophy terms<|user|>
I'm trying so constantly like feel like constantly like constantly like constantly like
<|>
I'm constantly trying so always trying trying like feel like everything
<|>
Whycingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcingcing

1 reply

reacted to BibbyResearch's post with 🔥🔥 about 1 month ago

Post

2932

🍌 Paper Banana is now live! Create academic illustrations using natural language

We just launched Paper Banana — a tool that lets you generate clean academic illustrations simply by describing them in natural language.

🔗 Try it here: https://trybibby.com/paper-banana

Whether you need diagrams for papers, presentations, or teaching materials, Paper Banana helps you turn ideas into visuals in seconds.

We’d love your feedback:

What did you like?
What features should we add next?

Give it a spin and let us know what you think! 🚀

Dear Huggingface, show this post to all my fellow researchers!

replied to CRAFTFramework's post about 1 month ago

Call me crazy, but I always thought of how much more efficient it made me in token usage and how much of my work I was actively retaining between sessions. Was it annoying? Absolutely. Did the benefits outweigh the tokens lost? After awhile maybe?

reacted to AINovice2005's post with 🔥 about 1 month ago

Post

3497

In celebration of the new storage graph feature on the Hub, here's mine 😊 :

Post inspired by @ZennyKenny

reacted to cahlen's post with 🚀 about 1 month ago

Post

3088

It’s wild to me how you can just make shit now.

You can take a weekend with a raspberry pi 5, a pi camera, a 3d printer, and a smidgen of custom fine tuning (wakeword, whisper, tinybert, and pipertts) and you have physical device as a talking personal assistant.

What a time to be alive.

Edge ai, physical ai, ai augmented animatronics… tiny models. Tiny agents.

Going to be a wild year.

6 replies

reacted to umarbutler's post with 🚀 about 1 month ago

Post

4871

Isaacus, the AI research company building legal superintelligence, is hiring!

We're looking for passionate engineers who love to build and tinker and want to have an impact on the world. Specifically, we're hiring:
• ML engineers (Australia).
• Data engineers (Australia).
• Full-stack engineers (Australia).
• DevRel engineers (Australia, San Francisco, and London).
• DevOps engineers (Australia, San Francisco, and London).

If you'd like to be a founding employee at one of the few VC-backed LLM research labs in the world, receive generous equity compensation, and work alongside other highly motivated, highly skilled engineers, get in touch: https://isaacus.com/careers

posted an update about 1 month ago

Post

133

Using GPT-4o as an LLM-as-judge I was able to compare the response capabilities of 4 Llama models which were all trained on the same corpus of a single voice (a bartender's personality) rather than scraped or synthetic data. The models represented are 1B and 3B models with heretic abliterated counterparts. These quotes were pulled from the final judgement after asking the models questions pulled from the following dataset. juiceb0xc0de/chaotic-absurdity

"The potential for even modestly-sized models to possess a semblance of soul isn't just a programmer's fantasy; it's a burgeoning reality."

"Bella-3b, a 3 billion parameter model, startled me with its capacity to latch onto narrative arcs and paint vivid scenes that you'd expect from an imaginative human mind. This model doesn't just compute; it rhapsodizes."

"The bella-1b and the heretic variations served as reminders that attempted creativity can sometimes feel staggeringly off-key. Their attempts often faltered into mechanical, over-processed responses, without the underlying current of emotion or wit."

"Bella-3b's performance not only challenges the preconception that only colossal language models could articulate a 'soul' but also suggests a future where smaller, finely tuned AIs can engage us with character depth and unexpected flair. They may not breathe (not yet), but are we at the dawn of AI characters with true personality? If Bella-3b is any indication, we're at least on the precipice."

https://huggingface.co/collections/juiceb0xc0de/the-bartenders

reacted to danielhanchen's post with 🔥 about 1 month ago

Post

3922

We collaborated with NVIDIA to teach you about Reinforcement Learning and RL environments. 💚 Learn:

• Why RL environments matter + how to build them
• When RL is better than SFT
• GRPO and RL best practices
• How verifiable rewards and RLVR work

Blog: https://unsloth.ai/blog/rl-environments

4 replies

R PRO

AI & ML interests

Recent Activity

Organizations

juiceb0xc0de's activity