AesSedai

/

Kimi-K2.5-GGUF

like 29

Just in case anybody else has problems with the thinking block not showing up, I posted the fix needed to the jinja template here:

https://huggingface.co/unsloth/Kimi-K2.5-GGUF/discussions/1#697b46fdf48287bb9c2e92dc

I've figured out what is causing this:

https://huggingface.co/moonshotai/Kimi-K2.5/blob/main/chat_template.jinja

{%- if add_generation_prompt -%}
  <|im_assistant|>assistant<|im_middle|>
  {%- if thinking is defined and thinking is false -%}
  <think></think>
  {%- else -%}
  <think>        # <--- This forced <think> tag doesn't set returned as part of the chat.
  {%- endif -%}
{%- endif -%}

To fix this you need to make a copy of chat_template.jinja and delete the {%- else -%} <think> part at the bottom like this:

{%- if add_generation_prompt -%}
  <|im_assistant|>assistant<|im_middle|>
  {%- if thinking is defined and thinking is false -%}
  <think></think>
  {%- endif -%}
{%- endif -%}

Then run llama.cpp using --jinja --chat-template-file chat_template.jinja options.

(Also: there doesn't seem to be any need/requirement to use --special option now)