Instructions to use Delta-Vector/Plesio-70B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Delta-Vector/Plesio-70B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Delta-Vector/Plesio-70B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Delta-Vector/Plesio-70B")
model = AutoModelForCausalLM.from_pretrained("Delta-Vector/Plesio-70B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Delta-Vector/Plesio-70B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Delta-Vector/Plesio-70B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Delta-Vector/Plesio-70B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Delta-Vector/Plesio-70B

SGLang

How to use Delta-Vector/Plesio-70B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Delta-Vector/Plesio-70B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Delta-Vector/Plesio-70B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Delta-Vector/Plesio-70B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Delta-Vector/Plesio-70B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Delta-Vector/Plesio-70B with Docker Model Runner:
```
docker model run hf.co/Delta-Vector/Plesio-70B
```

Context size?

by opiiopip - opened Nov 6, 2025

Discussion

opiiopip

Nov 6, 2025

Dear developers of the model Delta-Vector/Plesio-70B

I found this model via the tag roleplay, but I couldn't find the context size for this model. For roleplay, chat memory is crucial. Summaries of the chat history can only go so far; sooner or later, details the user clearly remembers are forgotten by the AI because of the limited context and the immersion is lost.

On the Featherless AI deployment of this model https://featherless.ai/models/Delta-Vector/Plesio-70B , I found the tag "Ctx length: 32768" -- but I'm a bit confused because for some models which were trained on a much larger context length like https://huggingface.co/aifeifei798/DarkIdol-Llama-3.1-8B-Instruct-1.2-Uncensored?inference_provider=featherless-ai with context length 128k , Featherless AI still has this tag.

So does Featherless AI somehow offer these models with lower context length than was used in the training?

Thank you very much in advance!

Delta-Vector

Owner Nov 6, 2025

The model should be able to handle upto 64K~, Featherless is compute limited and only can serve 32K ctx length.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment