Instructions to use abdoelsayed/llama-7b-v2-Receipt-Key-Extraction with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use abdoelsayed/llama-7b-v2-Receipt-Key-Extraction with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="abdoelsayed/llama-7b-v2-Receipt-Key-Extraction")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("abdoelsayed/llama-7b-v2-Receipt-Key-Extraction")
model = AutoModelForCausalLM.from_pretrained("abdoelsayed/llama-7b-v2-Receipt-Key-Extraction")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use abdoelsayed/llama-7b-v2-Receipt-Key-Extraction with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "abdoelsayed/llama-7b-v2-Receipt-Key-Extraction"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "abdoelsayed/llama-7b-v2-Receipt-Key-Extraction",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/abdoelsayed/llama-7b-v2-Receipt-Key-Extraction

SGLang

How to use abdoelsayed/llama-7b-v2-Receipt-Key-Extraction with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "abdoelsayed/llama-7b-v2-Receipt-Key-Extraction" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "abdoelsayed/llama-7b-v2-Receipt-Key-Extraction",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "abdoelsayed/llama-7b-v2-Receipt-Key-Extraction" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "abdoelsayed/llama-7b-v2-Receipt-Key-Extraction",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use abdoelsayed/llama-7b-v2-Receipt-Key-Extraction with Docker Model Runner:
```
docker model run hf.co/abdoelsayed/llama-7b-v2-Receipt-Key-Extraction
```

llama-7b-v2-Receipt-Key-Extraction / README.md

abdoelsayed

Update README.md

f909dfc over 2 years ago

preview code

raw

history blame contribute delete

2.71 kB

	---
	license: llama2
	language:
	- en
	- ar
	metrics:
	- accuracy
	- f1
	library_name: transformers
	---

	# llama-7b-v2-Receipt-Key-Extraction

	llama-7b-v2-Receipt-Key-Extraction is a 7 billion parameter based on LLamA v1

	[AMuRD: Annotated Multilingual Receipts Dataset for Cross-lingual Key Information Extraction and Classification](https://arxiv.org/abs/2309.09800)

	## Uses

	The model is intended for research-only use in English and Arabic for key information extraction for items in receipts.

	## How to Get Started with the Model

	Use the code below to get started with the model.

	```bibtex
	# pip install -q transformers
	import torch
	from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig

	try:
	if torch.backends.mps.is_available():
	device = "mps"
	except:
	pass
	checkpoint = "abdoelsayed/llama-7b-v2-Receipt-Key-Extraction"
	device = "cuda" if torch.cuda.is_available() else "cpu"

	tokenizer = AutoTokenizer.from_pretrained(checkpoint, model_max_length=512,
	padding_side="right",
	use_fast=False,)
	model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)

	def generate_response(instruction, input_text, max_new_tokens=100, temperature=0.1, num_beams=4 , top_p=0.75, top_k=40):
	prompt = f"Below is an instruction that describes a task, paired with an input that provides further context.\n\n### Instruction:\n{instruction}\n\n### Input:\n{input_text}\n\n### Response:"
	inputs = tokenizer(prompt, return_tensors="pt")
	input_ids = inputs["input_ids"].to(device)
	generation_config = GenerationConfig(
	temperature=temperature,
	top_p=top_p,
	top_k=top_k,
	num_beams=num_beams,
	)
	with torch.no_grad():
	outputs = model.generate(input_ids,generation_config=generation_config, max_new_tokens=max_new_tokens,return_dict_in_generate=True,output_scores=True,)
	outputs = tokenizer.decode(outputs.sequences[0])
	return outputs.split("### Response:")[-1].strip().replace("</s>","")

	instruction = "Extract the class, Brand, Weight, Number of units, Size of units, Price, T.Price, Pack, Unit from the following sentence"
	input_text = "Americana Okra zero 400 gm"

	response = generate_response(instruction, input_text)
	print(response)



	```



	## How to Cite

	Please cite this model using this format.

	```bibtex
	@misc{abdallah2023amurd,
	title={AMuRD: Annotated Multilingual Receipts Dataset for Cross-lingual Key Information Extraction and Classification},
	author={Abdelrahman Abdallah and Mahmoud Abdalla and Mohamed Elkasaby and Yasser Elbendary and Adam Jatowt},
	year={2023},
	eprint={2309.09800},
	archivePrefix={arXiv},
	primaryClass={cs.CL}
	}
	```