Instructions to use Tiiny/SmallThinker-3B-Preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Tiiny/SmallThinker-3B-Preview with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Tiiny/SmallThinker-3B-Preview")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Tiiny/SmallThinker-3B-Preview")
model = AutoModelForCausalLM.from_pretrained("Tiiny/SmallThinker-3B-Preview")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Tiiny/SmallThinker-3B-Preview with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Tiiny/SmallThinker-3B-Preview"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tiiny/SmallThinker-3B-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Tiiny/SmallThinker-3B-Preview

SGLang

How to use Tiiny/SmallThinker-3B-Preview with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Tiiny/SmallThinker-3B-Preview" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tiiny/SmallThinker-3B-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Tiiny/SmallThinker-3B-Preview" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tiiny/SmallThinker-3B-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Tiiny/SmallThinker-3B-Preview with Docker Model Runner:
```
docker model run hf.co/Tiiny/SmallThinker-3B-Preview
```

SmallThinker-3B-Preview

Commit History

Update README.md

ffeee01
verified

AISafety commited on Jan 3, 2025

Update README.md

53ee55a
verified

Yixin Song commited on Jan 2, 2025

Update README.md

6e5607e
verified

Yixin Song commited on Jan 2, 2025

minor

94c6614

syx commited on Dec 31, 2024

minor

1c3cc64

syx commited on Dec 31, 2024

minor

1c81b39

syx commited on Dec 31, 2024

Update README.md

a325535
verified

Yixin Song commited on Dec 31, 2024

Update README.md

7b48509
verified

Yixin Song commited on Dec 31, 2024

Update README.md

f7d6bfa
verified

Yixin Song commited on Dec 31, 2024

aimc

0f650b4

syx commited on Dec 30, 2024

Delete result/aime24_eval.json

6d98efa
verified

Yixin Song commited on Dec 30, 2024

Delete zero_to_fp32.py

b113ff8
verified

Yixin Song commited on Dec 30, 2024

minoe

289b8a2

syx commited on Dec 30, 2024

minoe

0b68517

syx commited on Dec 30, 2024

minor

7a5581b

syx commited on Dec 30, 2024

add eval result

2e31d80

syx commited on Dec 30, 2024

Update README.md

9b1b5d2
verified

Yixin Song commited on Dec 30, 2024

Update README.md

434f731
verified

Yixin Song commited on Dec 30, 2024

Update README.md

a6eab6c
verified

Yixin Song commited on Dec 30, 2024

Update README.md

b71f3e0
verified

Yixin Song commited on Dec 30, 2024

Update README.md

011743e
verified

Yixin Song commited on Dec 30, 2024

Update README.md

5514a74
verified

Yixin Song commited on Dec 26, 2024

minor

6662798

syx commited on Dec 25, 2024

minor

afc7124

syx commited on Dec 25, 2024

minor

f0ad825

syx commited on Dec 25, 2024

minor

46c2512

syx commited on Dec 25, 2024

minor

38ae50e

syx commited on Dec 25, 2024

minor

fb9163c

syx commited on Dec 25, 2024

minor

d4dea51

syx commited on Dec 25, 2024

minor

67ff38a

syx commited on Dec 25, 2024

minor

bb1b8af

syx commited on Dec 25, 2024

minor

555999a

syx commited on Dec 25, 2024

update weight

0d452c5

syx commited on Dec 23, 2024

Update README.md

2641c8f
verified

Yixin Song commited on Dec 13, 2024

Update README.md

25741de
verified

Yixin Song commited on Dec 13, 2024

Update README.md

2e19f5b
verified

Yixin Song commited on Dec 13, 2024

Update README.md

6ff06d6
verified

Yixin Song commited on Dec 12, 2024

Update README.md

c4e402c
verified

Yixin Song commited on Dec 12, 2024

Update README.md

782776b
verified

Yixin Song commited on Dec 12, 2024

Update README.md

36f304d
verified

Yixin Song commited on Dec 12, 2024

qwen2 license

bc8fad5

syx commited on Dec 12, 2024

model weight upload

9dedae3

syx commited on Dec 12, 2024

initial commit

6ca1157
verified

Yixin Song commited on Dec 12, 2024

Commit History

Update README.md ffeee01 verified

Update README.md 53ee55a verified

Update README.md 6e5607e verified

minor 94c6614

minor 1c3cc64

minor 1c81b39

Update README.md a325535 verified

Update README.md 7b48509 verified

Update README.md f7d6bfa verified

aimc 0f650b4

Delete result/aime24_eval.json 6d98efa verified

Delete zero_to_fp32.py b113ff8 verified

minoe 289b8a2

minoe 0b68517

minor 7a5581b

add eval result 2e31d80

Update README.md 9b1b5d2 verified

Update README.md 434f731 verified

Update README.md a6eab6c verified

Update README.md b71f3e0 verified

Update README.md 011743e verified

Update README.md 5514a74 verified

minor 6662798

minor afc7124

minor f0ad825

minor 46c2512

minor 38ae50e

minor fb9163c

minor d4dea51

minor 67ff38a

minor bb1b8af

minor 555999a

update weight 0d452c5

Update README.md 2641c8f verified

Update README.md 25741de verified

Update README.md 2e19f5b verified

Update README.md 6ff06d6 verified

Update README.md c4e402c verified

Update README.md 782776b verified

Update README.md 36f304d verified

qwen2 license bc8fad5

model weight upload 9dedae3

initial commit 6ca1157 verified

Update README.md

ffeee01
verified

Update README.md

53ee55a
verified

Update README.md

6e5607e
verified

minor

94c6614

minor

1c3cc64

minor

1c81b39

Update README.md

a325535
verified

Update README.md

7b48509
verified

Update README.md

f7d6bfa
verified

aimc

0f650b4

Delete result/aime24_eval.json

6d98efa
verified

Delete zero_to_fp32.py

b113ff8
verified

minoe

289b8a2

minoe

0b68517

minor

7a5581b

add eval result

2e31d80

Update README.md

9b1b5d2
verified

Update README.md

434f731
verified

Update README.md

a6eab6c
verified

Update README.md

b71f3e0
verified

Update README.md

011743e
verified

Update README.md

5514a74
verified

minor

6662798

minor

afc7124

minor

f0ad825

minor

46c2512

minor

38ae50e

minor

fb9163c

minor

d4dea51

minor

67ff38a

minor

bb1b8af

minor

555999a

update weight

0d452c5

Update README.md

2641c8f
verified

Update README.md

25741de
verified

Update README.md

2e19f5b
verified

Update README.md

6ff06d6
verified

Update README.md

c4e402c
verified

Update README.md

782776b
verified

Update README.md

36f304d
verified

qwen2 license

bc8fad5

model weight upload

9dedae3

initial commit

6ca1157
verified