Instructions to use mrm8488/bloom-560m-finetuned-the-stack-rust with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mrm8488/bloom-560m-finetuned-the-stack-rust with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="mrm8488/bloom-560m-finetuned-the-stack-rust")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("mrm8488/bloom-560m-finetuned-the-stack-rust")
model = AutoModelForCausalLM.from_pretrained("mrm8488/bloom-560m-finetuned-the-stack-rust")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use mrm8488/bloom-560m-finetuned-the-stack-rust with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mrm8488/bloom-560m-finetuned-the-stack-rust"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mrm8488/bloom-560m-finetuned-the-stack-rust",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/mrm8488/bloom-560m-finetuned-the-stack-rust

SGLang

How to use mrm8488/bloom-560m-finetuned-the-stack-rust with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mrm8488/bloom-560m-finetuned-the-stack-rust" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mrm8488/bloom-560m-finetuned-the-stack-rust",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mrm8488/bloom-560m-finetuned-the-stack-rust" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mrm8488/bloom-560m-finetuned-the-stack-rust",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use mrm8488/bloom-560m-finetuned-the-stack-rust with Docker Model Runner:
```
docker model run hf.co/mrm8488/bloom-560m-finetuned-the-stack-rust
```

BLOOM (560M ckpt) fine-tuned on The Stack RUST code

Latest ckpt: https://huggingface.co/mrm8488/bloom-560m-finetuned-the-stack-rust/tree/100k

Model 🧠

BigScience Large Open-science Open-access Multilingual Language Model 🌸

BLOOM is an autoregressive Large Language Model (LLM), trained to continue text from a prompt on vast amounts of text data using industrial-scale computational resources. As such, it is able to output coherent text in 46 languages and 13 programming languages that is hardly distinguishable from text written by humans. BLOOM can also be instructed to perform text tasks it hasn't been explicitly trained for, by casting them as text generation tasks.

Dataset 📚

Rust 🦀 part of The Stack.

The Stack contains over 6TB of permissively-licensed source code files covering 358 programming languages. The dataset was created as part of the BigCode Project, an open scientific collaboration working on the responsible development of Large Language Models for Code (Code LLMs). The Stack serves as a pre-training dataset for Code LLMs, i.e., code-generating AI systems which enable the synthesis of programs from natural language descriptions as well as other from code snippets.

Example of usage 👩‍💻

import torch
from transformers import BloomTokenizerFast, BloomForCausalLM

device = 'cuda' if torch.cuda.is_available() else 'cpu'
ckpt = 'mrm8488/bloom-560m-finetuned-the-stack-rust'
revision = '100k' # latest one at the moment 

tokenizer = BloomTokenizerFast.from_pretrained(ckpt)
model = BloomForCausalLM.from_pretrained(ckpt, revision=revision).to(device)

def complete_code(text):
    inputs = tokenizer(text, return_tensors='pt')
    input_ids = inputs.input_ids.to(device)
    attention_mask = inputs.attention_mask.to(device)
    output = model.generate(input_ids, attention_mask=attention_mask, max_length=2048, eos_token_id=tokenizer.eos_token_id)

    return tokenizer.decode(output[0], skip_special_tokens=False)
    
code_prompt = """
use fastly::{Error, Request, Response};
use serde_json::{json, Value};

#[fastly::main]
fn main(req: Request) -> Result<Response, Error> {
  let mut response = req.send("origin_0")?;
"""

complete_code(code_prompt)

Citation ✒️

@misc {manuel_romero_2022,
    author       = { {Manuel Romero} },
    title        = { bloom-560m-finetuned-the-stack-rust (Revision 5358462) },
    year         = 2022,
    url          = { https://huggingface.co/mrm8488/bloom-560m-finetuned-the-stack-rust },
    doi          = { 10.57967/hf/0236 },
    publisher    = { Hugging Face }
}

Downloads last month: 1,076