ValueError: The checkpoint you are trying to load has model type `internvl` but Transformers does not recognize this architecture.
#4
by LeMoussel - opened
from transformers.models.auto.processing_auto import AutoProcessor
from transformers.models.auto.modeling_auto import AutoModelForImageTextToText
import torch
torch_device = "cuda"
model_checkpoint = "OpenGVLab/InternVL3-8B-hf"
processor = AutoProcessor.from_pretrained(model_checkpoint)
model = AutoModelForImageTextToText.from_pretrained(model_checkpoint, device_map=torch_device, torch_dtype=torch.bfloat16)
messages = [
{
"role": "user",
"content": [
{"type": "image", "url": "page_23.jpg"},
{"type": "text", "text": """
You are an expert document converter that transforms documents into clean, well-formatted markdown.
Extract all text content, preserve document structure, and format properly with markdown syntax.
Include headings, lists, tables, and other formatting elements as appropriate.
Ignore page numbers, headers/footers, and other irrelevant elements.
Return ONLY the markdown content with no explanations or additional text."""},
],
}
]
inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(model.device, dtype=torch.bfloat16)
generate_ids = model.generate(**inputs, max_new_tokens=50)
decoded_output = processor.decode(generate_ids[0, inputs["input_ids"].shape[1] :], skip_special_tokens=True)
Error: ValueError: The checkpoint you are trying to load has model type internvl but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
You can update Transformers with the command pip install --upgrade transformers. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command pip install git+https://github.com/huggingface/transformers.git
Note: I use the latest version of Transformers by installing it directly from the source.