Hugging Face's logo Hugging Face
  • Models
  • Datasets
  • Spaces
  • Buckets new
  • Docs
  • Enterprise
  • Pricing
    • Website
      • Tasks
      • HuggingChat
      • Collections
      • Languages
      • Organizations
    • Community
      • Blog
      • Posts
      • Daily Papers
      • Learn
      • Discord
      • Forum
      • GitHub
    • Solutions
      • Team & Enterprise
      • Hugging Face PRO
      • Enterprise Support
      • Inference Providers
      • Inference Endpoints
      • Storage Buckets

  • Log In
  • Sign Up
fxmeng 's Collections
TransMLA-base
CLOVER-Commonsense-148k
PiSSA-LLaMA-3-8B
PiSSA-LLaMA-3-70B
PiSSA-LLaMA-2-7B
PiSSA-LLaMA-3-8B-Instruct
PiSSA-Qwen2
PiSSA Datasets
Mixtral-1~8x7B-Instruct-v0.1

TransMLA-base

updated Mar 2

Base Model for TransMLA

Upvote
-

  • TransMLA: Multi-head Latent Attention Is All You Need

    Paper • 2502.07864 • Published Feb 11, 2025 • 69

  • fxmeng/transmla_pretrain_6B_tokens

    Viewer • Updated Jul 5, 2025 • 5.94M • 2.07k

  • fxmeng/transmla_pretrain_1B_tokens

    Viewer • Updated Jul 5, 2025 • 1.14M • 16

  • fxmeng/transmla_pretrain_100m_tokens

    Viewer • Updated Jul 5, 2025 • 100k • 8

  • fxmeng/TransMLA-llama3-8b-8k

    8B • Updated Nov 24, 2025 • 102

  • fxmeng/TransMLA-llama3-8b-32k

    8B • Updated Nov 24, 2025 • 9
Upvote
-
  • Collection guide
  • Browse collections
Company
TOS Privacy About Careers
Website
Models Datasets Spaces Pricing Docs