BigCodeArena: Judging code generations end to end with code executions
โข
19
AI & ML interests
None defined yet.
Recent Activity
View all activity
Papers
BigCodeArena: Unveiling More Reliable Human Preferences in Code Generation via Execution
BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions
Articles
ZennyKennyย
posted
an
update
2 days ago
Post
3045
๐ My new personal website is live! Check out https://kennethhamilton.me to chat with an LLM about my professional skills and personal projects.
๐ Think of it like a really, really vain version of ChatGPT.
๐ Think of it like a really, really vain version of ChatGPT.
-
- 6 replies
gagan3012ย
authored
2
papers
7 days ago
ybelkadaย
authored
a
paper
12 days ago
ZennyKennyย
posted
an
update
about 1 month ago
Post
1999
๐ One of the coolest parts about being an early Strawberry user has been the opportunity to build on the app at the ground floor.
The platform already has a ton of great integrations that let you interact with your external apps directly with tools, but I wanted to add the ability to do stuff in Slack as well.
๐ช So I took the base Anthropic Slack MCP server, added a whole bunch of new tools, and generalized it as an HTTP-based SSE-server and deployed it in like 2 minutes with Railway so that Strawberry could make use of it (as can Claude or any other MCP client).
Now, you can Chat with your Strawberry Companion (or Claude, or whatever) and do things like:
โก๏ธ Get caught up across all of your Slack channels after a long weekend or noisy incident without having to read 20 threads in 10 different channels
โก๏ธ Create, read, and edit Canvases, Messages, and Channels
โก๏ธ Take any resources or content that you're using in your Chat and inject it directly into Slack without copy / paste
๐ I'm pretty pleased with the results, and I made a short demo video showing the results of the work (link in comments). The best part is, it's available on GitHub for anyone else to use too (link in the comments, instructions in the README). The setup takes about 5-10 minutes.
The platform already has a ton of great integrations that let you interact with your external apps directly with tools, but I wanted to add the ability to do stuff in Slack as well.
๐ช So I took the base Anthropic Slack MCP server, added a whole bunch of new tools, and generalized it as an HTTP-based SSE-server and deployed it in like 2 minutes with Railway so that Strawberry could make use of it (as can Claude or any other MCP client).
Now, you can Chat with your Strawberry Companion (or Claude, or whatever) and do things like:
โก๏ธ Get caught up across all of your Slack channels after a long weekend or noisy incident without having to read 20 threads in 10 different channels
โก๏ธ Create, read, and edit Canvases, Messages, and Channels
โก๏ธ Take any resources or content that you're using in your Chat and inject it directly into Slack without copy / paste
๐ I'm pretty pleased with the results, and I made a short demo video showing the results of the work (link in comments). The best part is, it's available on GitHub for anyone else to use too (link in the comments, instructions in the README). The setup takes about 5-10 minutes.
-
- 2 replies
ZennyKennyย
posted
an
update
about 2 months ago
Post
225
What a trip. Just walked through
@burtenshaw
and
@evalstate
tutorial on adding Hugging Face Skills to your Claude Code agent so you can fine tune LLMs by chatting with AI.
These are the kinds of innovations that are going to help everyone benefit from the power of Artificial Intelligence. Well done gentlemen and thank you for sharing.
These are the kinds of innovations that are going to help everyone benefit from the power of Artificial Intelligence. Well done gentlemen and thank you for sharing.
-
- 1 reply
monsoon-nlpย
posted
an
update
about 2 months ago
Post
405
PatchDNA, a DNA foundation model based on Meta's BLT tokenization strategy https://www.biorxiv.org/content/10.1101/2025.11.28.691095v1
ZennyKennyย
posted
an
update
about 2 months ago
Post
260
๐ I keep seeing takes on LinkedIn from American business influencers melting down about Silicon Valley startup "dependence" on open-source Chinese models.
๐ค Can anyone describe a credible scenario where these models can be leveraged by the Chinese government to endanger American security interests or am I right to believe that this is just Red Scare nonsense?
๐ค Can anyone describe a credible scenario where these models can be leveraged by the Chinese government to endanger American security interests or am I right to believe that this is just Red Scare nonsense?
-
- 2 replies
ZennyKennyย
posted
an
update
2 months ago
Post
435
The #feedback channel of app early access Slack Workspaces is some of the best unintentional comedy material I have ever come across tbh.
ZennyKennyย
posted
an
update
2 months ago
ZennyKennyย
posted
an
update
2 months ago
Post
337
๐ Novoyaz is live.
A few months ago, I built a quick POC in Hugging Face that used a fine-tuned variant of OpenAI's OSS-20B model that I trained to convert the text from pre-reform Russian-language documents into modern Russian orthography.
โก๏ธ This morning, I launched novoyaz.io.
This is a production app, the frontend for which I built in like two hours with Lovable, that uses that same fine-tuned model for transliteration, but now has a bunch of extra features that make using it even easier (like taking and uploading pictures with your on-device camera for example ๐ ).
๐ If you're a researcher, or know a researcher, for whom this app will improve their day-to-day workflows, please get in touch with me.
A few months ago, I built a quick POC in Hugging Face that used a fine-tuned variant of OpenAI's OSS-20B model that I trained to convert the text from pre-reform Russian-language documents into modern Russian orthography.
โก๏ธ This morning, I launched novoyaz.io.
This is a production app, the frontend for which I built in like two hours with Lovable, that uses that same fine-tuned model for transliteration, but now has a bunch of extra features that make using it even easier (like taking and uploading pictures with your on-device camera for example ๐ ).
๐ If you're a researcher, or know a researcher, for whom this app will improve their day-to-day workflows, please get in touch with me.
ZennyKennyย
posted
an
update
3 months ago
Post
348
Anyone got the scoop on a good OCR model that's available on inference?
Keen to make use of an endpoint (gated or not -- happy to pay for usage) for a personal project, but not so keen to pay for the GPU hosting myself.
๐๐๐
Keen to make use of an endpoint (gated or not -- happy to pay for usage) for a personal project, but not so keen to pay for the GPU hosting myself.
๐๐๐
-
- 4 replies
nouamanetaziย
posted
an
update
3 months ago
Post
4306
After training ๐๐ฆ๐จ๐ฅ๐๐๐ on ๐๐๐ ๐๐๐๐๐ฌ for nearly a month, I've come to realize something most people overlook: ๐ข๐ง๐๐ซ๐๐ฌ๐ญ๐ซ๐ฎ๐๐ญ๐ฎ๐ซ๐ ๐ข๐ฌ ๐ญ๐ก๐ ๐ฆ๐๐ค๐-๐จ๐ซ-๐๐ซ๐๐๐ค ๐๐๐๐ญ๐จ๐ซ ๐ข๐ง ๐๐๐ ๐ญ๐ซ๐๐ข๐ง๐ข๐ง๐ . ๐ฅ
Everyone talks about model architecture and data quality. And yes, those matter immensely. But here's what nobody tells you: when your training run fails at 2 AM because of mysterious ๐๐๐๐ ๐๐ซ๐ซ๐จ๐ซ๐ฌ, or when your expensive GPU cluster is running at ๐๐% ๐๐๐๐ข๐๐ข๐๐ง๐๐ฒ, the problem isn't your model. It's most probably a ๐ฆ๐ข๐ฌ๐ฎ๐ฌ๐ ๐จ๐ ๐ญ๐ก๐ ๐ก๐๐ซ๐๐ฐ๐๐ซ๐. ๐ ๏ธ
Questions that seemed simple but had no clear answers: Why is ๐๐จ๐ ๐ญ๐ซ๐๐ข๐ง๐ข๐ง๐ ๐ฌ๐ฅ๐จ๐ฐ๐๐ซ ๐ญ๐ก๐๐ง ๐๐๐ง๐ฌ๐ ๐ฆ๐จ๐๐๐ฅ๐ฌ? Which ๐๐๐๐ ๐๐ฅ๐๐ ๐ฌ should we actually set? How often should we checkpoint without killing throughput?
That's why we built ๐๐ก๐ ๐๐ฆ๐จ๐ฅ ๐๐ซ๐๐ข๐ง๐ข๐ง๐ ๐๐ฅ๐๐ฒ๐๐จ๐จ๐ค ๐: a complete guide covering everything from model architecture and data curation to the SmolLM3 training marathon, post-training techniques, and crucially, the ๐ข๐ง๐๐ซ๐๐ฌ๐ญ๐ซ๐ฎ๐๐ญ๐ฎ๐ซ๐ ๐ฅ๐๐ฒ๐๐ซ that most teams get wrong.
We validated real vs theoretical bandwidth across the entire stack: ๐๐๐๐ ๐ก๐ข๐ญ๐ญ๐ข๐ง๐ ๐ ๐๐/๐ฌ, ๐๐๐๐ข๐ง๐ค ๐.๐ ๐ซ๐๐๐๐ก๐ข๐ง๐ ๐๐๐ ๐๐/๐ฌ, ๐๐๐๐ ๐๐๐ง๐ ๐๐ญ ๐๐.๐ ๐๐/๐ฌ. Then we ran collective operations across ๐๐๐ ๐๐๐๐ฌ (16 nodes, 8xH100s each) and measured how performance degrades at scale: all-reduce drops from ๐๐๐ ๐๐/๐ฌ on a single node to ๐๐๐-๐๐๐ ๐๐/๐ฌ across 16 nodes.
If you've ever wondered why your training runs are slower than they should be, or you're planning to scale up and want to avoid expensive mistakes, this guide might save you weeks of debugging.
๐๐ก๐ ๐๐ฆ๐จ๐ฅ ๐๐ซ๐๐ข๐ง๐ข๐ง๐ ๐๐ฅ๐๐ฒ๐๐จ๐จ๐ค: https://lnkd.in/e5MKXUHS
Shared with โค๏ธ by the HuggingFace team
Everyone talks about model architecture and data quality. And yes, those matter immensely. But here's what nobody tells you: when your training run fails at 2 AM because of mysterious ๐๐๐๐ ๐๐ซ๐ซ๐จ๐ซ๐ฌ, or when your expensive GPU cluster is running at ๐๐% ๐๐๐๐ข๐๐ข๐๐ง๐๐ฒ, the problem isn't your model. It's most probably a ๐ฆ๐ข๐ฌ๐ฎ๐ฌ๐ ๐จ๐ ๐ญ๐ก๐ ๐ก๐๐ซ๐๐ฐ๐๐ซ๐. ๐ ๏ธ
Questions that seemed simple but had no clear answers: Why is ๐๐จ๐ ๐ญ๐ซ๐๐ข๐ง๐ข๐ง๐ ๐ฌ๐ฅ๐จ๐ฐ๐๐ซ ๐ญ๐ก๐๐ง ๐๐๐ง๐ฌ๐ ๐ฆ๐จ๐๐๐ฅ๐ฌ? Which ๐๐๐๐ ๐๐ฅ๐๐ ๐ฌ should we actually set? How often should we checkpoint without killing throughput?
That's why we built ๐๐ก๐ ๐๐ฆ๐จ๐ฅ ๐๐ซ๐๐ข๐ง๐ข๐ง๐ ๐๐ฅ๐๐ฒ๐๐จ๐จ๐ค ๐: a complete guide covering everything from model architecture and data curation to the SmolLM3 training marathon, post-training techniques, and crucially, the ๐ข๐ง๐๐ซ๐๐ฌ๐ญ๐ซ๐ฎ๐๐ญ๐ฎ๐ซ๐ ๐ฅ๐๐ฒ๐๐ซ that most teams get wrong.
We validated real vs theoretical bandwidth across the entire stack: ๐๐๐๐ ๐ก๐ข๐ญ๐ญ๐ข๐ง๐ ๐ ๐๐/๐ฌ, ๐๐๐๐ข๐ง๐ค ๐.๐ ๐ซ๐๐๐๐ก๐ข๐ง๐ ๐๐๐ ๐๐/๐ฌ, ๐๐๐๐ ๐๐๐ง๐ ๐๐ญ ๐๐.๐ ๐๐/๐ฌ. Then we ran collective operations across ๐๐๐ ๐๐๐๐ฌ (16 nodes, 8xH100s each) and measured how performance degrades at scale: all-reduce drops from ๐๐๐ ๐๐/๐ฌ on a single node to ๐๐๐-๐๐๐ ๐๐/๐ฌ across 16 nodes.
If you've ever wondered why your training runs are slower than they should be, or you're planning to scale up and want to avoid expensive mistakes, this guide might save you weeks of debugging.
๐๐ก๐ ๐๐ฆ๐จ๐ฅ ๐๐ซ๐๐ข๐ง๐ข๐ง๐ ๐๐ฅ๐๐ฒ๐๐จ๐จ๐ค: https://lnkd.in/e5MKXUHS
Shared with โค๏ธ by the HuggingFace team
Post
3934
๐ค Did you know your voice might be cloned without your consent from just *one sentence* of audio?
That's not great. So with @frimelle , we brainstormed a new idea for developers who want to curb malicious use: โจThe Voice Consent Gate.โจ
Details, code, here: https://huggingface.co/blog/voice-consent-gate
That's not great. So with @frimelle , we brainstormed a new idea for developers who want to curb malicious use: โจThe Voice Consent Gate.โจ
Details, code, here: https://huggingface.co/blog/voice-consent-gate
-
- 3 replies
ZennyKennyย
posted
an
update
3 months ago
Post
332
Has anyone tried Strawberry Browser? https://strawberrybrowser.com/?ref_id=8D41NQCY7
๐ Shamelessly sharing my referral link here to move up in the waitlist line. Help me out, give it a click.
๐ Shamelessly sharing my referral link here to move up in the waitlist line. Help me out, give it a click.
-
- 2 replies
tscholakย
authored
5
papers
3 months ago
PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models
Paper
โข
2109.05093
โข
Published
โข
1
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models
Paper
โข
2201.05966
โข
Published
โข
1
Unifying Autoregressive and Diffusion-Based Sequence Generation
Paper
โข
2504.06416
โข
Published
โข
3
Apriel-Nemotron-15B-Thinker
Paper
โข
2508.10948
โข
Published
โข
5
Apriel-1.5-15b-Thinker
Paper
โข
2510.01141
โข
Published
โข
119