Instructions to use defog/sqlcoder-7b-2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use defog/sqlcoder-7b-2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="defog/sqlcoder-7b-2")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("defog/sqlcoder-7b-2") model = AutoModelForCausalLM.from_pretrained("defog/sqlcoder-7b-2") - llama-cpp-python
How to use defog/sqlcoder-7b-2 with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="defog/sqlcoder-7b-2", filename="sqlcoder-7b-q5_k_m.gguf", )
output = llm( "Once upon a time,", max_tokens=512, echo=True ) print(output)
- Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use defog/sqlcoder-7b-2 with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf defog/sqlcoder-7b-2:Q5_K_M # Run inference directly in the terminal: llama-cli -hf defog/sqlcoder-7b-2:Q5_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf defog/sqlcoder-7b-2:Q5_K_M # Run inference directly in the terminal: llama-cli -hf defog/sqlcoder-7b-2:Q5_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf defog/sqlcoder-7b-2:Q5_K_M # Run inference directly in the terminal: ./llama-cli -hf defog/sqlcoder-7b-2:Q5_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf defog/sqlcoder-7b-2:Q5_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf defog/sqlcoder-7b-2:Q5_K_M
Use Docker
docker model run hf.co/defog/sqlcoder-7b-2:Q5_K_M
- LM Studio
- Jan
- vLLM
How to use defog/sqlcoder-7b-2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "defog/sqlcoder-7b-2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "defog/sqlcoder-7b-2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/defog/sqlcoder-7b-2:Q5_K_M
- SGLang
How to use defog/sqlcoder-7b-2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "defog/sqlcoder-7b-2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "defog/sqlcoder-7b-2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "defog/sqlcoder-7b-2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "defog/sqlcoder-7b-2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Ollama
How to use defog/sqlcoder-7b-2 with Ollama:
ollama run hf.co/defog/sqlcoder-7b-2:Q5_K_M
- Unsloth Studio new
How to use defog/sqlcoder-7b-2 with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for defog/sqlcoder-7b-2 to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for defog/sqlcoder-7b-2 to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for defog/sqlcoder-7b-2 to start chatting
- Docker Model Runner
How to use defog/sqlcoder-7b-2 with Docker Model Runner:
docker model run hf.co/defog/sqlcoder-7b-2:Q5_K_M
- Lemonade
How to use defog/sqlcoder-7b-2 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull defog/sqlcoder-7b-2:Q5_K_M
Run and chat with the model
lemonade run user.sqlcoder-7b-2-Q5_K_M
List all available models
lemonade list
How to make sql coder generate sql query for sqllite or different types of schemas like postgres mysql redshift
I am using the sqlcoder-7b-2 model to generate the sqlquery everything works okay but I want to get the query which can run on sqllite right now it returns the postgres format how to make that changes,Do I give that in prompt of do I need add some parameter .How does this work
Hi @akhil2204 sqllite generally has the more liberal query syntax (eg less strict about types) so what you generated in postgres format should work with sqllite. If there are any edge cases that the query doesn't cover, please feel free to share a reproducible example.
Hi @wongjingping I am getting the keyword ILIKE in the query but sqllite expects like.How would I tackle these issues
Hi @wongjingping This is the reproducible example ,If you can see the last line in the query it returns ILIKE.
I want it to be "LIKE".
SELECT DISTINCT p.ProductCode,
p.ProductLine,
p.MSRP
FROM Products p
JOIN Orders o ON p.ProductCode = o.ProductCode
JOIN Customers c ON o.CustomerName = c.CustomerName
WHERE c.CustomerName ILIKE '%kwai%'
If you can see the 4 line in Instruction I have mentioned to use LIKE
Here is the prompt which I have been using:
"""
Task
Generate a SQL query to answer [QUESTION]{question}[/QUESTION]
Instructions
-This schema runs on sqlite server and not postgres server
-The database schema provided is in sqlite3 syntax so make sure you generate sql in sqlite3 format
-If you cannot answer the question with the available database schema, return 'I do not know'
-Instead of ILIKE in the query use "LIKE" in the query
-Instead of 'true or false ' use 1 or 0 in the query
CREATE TABLE IF NOT EXISTS Products (
ProductCode TEXT PRIMARY KEY,
ProductLine TEXT,
MSRP INTEGER
);### Database Schema
This query will run on a SQLLITE database whose schema is represented in this string:
-ProductCode is the primarykey
CREATE TABLE IF NOT EXISTS Customers (
CustomerName TEXT PRIMARY KEY,
Phone TEXT,
AddressLine1 TEXT,
AddressLine2 TEXT,
City TEXT,
State TEXT,
PostalCode TEXT,
Country TEXT,
Territory TEXT,
ContactLastName TEXT,
ContactFirstName TEXT
);
This query will run on a SQLLITE database whose schema is represented in this string:
-CustomerName is the primarykey
CREATE TABLE IF NOT EXISTS Orders (
OrderNumber INTEGER PRIMARY KEY,
QuantityOrdered INTEGER NOT NULL,
PriceEach REAL NOT NULL,
OrderLineNumber INTEGER,
Sales REAL,
OrderDate TEXT NOT NULL,
Status TEXT,
Qtr_ID INTEGER,
Month_ID INTEGER,
Year_ID INTEGER,
ProductLine TEXT,
MSRP INTEGER,
ProductCode TEXT NOT NULL,
CustomerName TEXT NOT NULL,
DealSize TEXT,
FOREIGN KEY (ProductCode) REFERENCES Products(ProductCode),
FOREIGN KEY (CustomerName) REFERENCES Customers(CustomerName)
);
This query will run on a SQLLITE database whose schema is represented in this string:
-OrderNumber is the primarykey
-orders.ProductCode can join on Products.ProductCode
-Customers.CustomerName can join on Orders.CustomerName
Answer
Given the database schema, here is the SQL query that answers [QUESTION]{question}[/QUESTION]
[SQL]
"""
was anyone able to work around or fix this issue ?
I think maybe we can use another text-to-text llm model for fixing non-sqllite command. With this extra layer sql sentences could be suitable for sqllite.
You can also have the model correct the error that occurs, perhaps the accuracy rate will not be as low as MSSQL, but correction can be made with a second response cycle.
I settled on a simple replace using python. The generated sql had ilike statement which is not supported by sqlite, so I just replaced it with like.
Just changed the prompt and it worked
Could you please share the prompt that worked for you. I'm also facing similar issues like (CURRENT_DATE - INTERVAL '10 days').
PostgreSQL requires quotes (INTERVAL '10 days'). MySQL requires no quotes and uses DAY (INTERVAL 10 DAY).