Manhph2211/Q-HEART

Q-HEART: ECG Question Answering via Knowledge-Informed Multimodal LLMs (ECAI 2025)

Hung Manh Pham   Jialu Tang   Aaqib Saeed   Dong Ma  

Usage

After we have access to meta-llama/Llama-3.2-1B-Instruct model and install suitable transformers package version, we can run:

# transformers==4.43.3 accelerate==1.0.1 peft==0.13.2
from transformers import AutoModel
model = AutoModel.from_pretrained("Manhph2211/Q-HEART", trust_remote_code=True, dtype="auto")

Or

git clone https://github.com/manhph2211/Q-HEART.git && cd Q-HEART
conda create -n qheart python=3.9
conda activate qheart
pip install torch --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt

Download the checkpoint from here and place it at ckpts/pytorch_model.bin, then run evaluation:

python main.py --model_type meta-llama/Llama-3.2-1B-Instruct --mapping_type Transformer

Citation

@article{pham2025q,
  title={Q-Heart: ECG Question Answering via Knowledge-Informed Multimodal LLMs},
  author={Pham, Hung Manh and Tang, Jialu and Saeed, Aaqib and Ma, Dong},
  journal={arXiv preprint arXiv:2505.06296},
  year={2025}
}

@inproceedings{pham2025qheart,
  title     = {Q-HEART: ECG Question Answering via Knowledge-Informed Multimodal LLMs},
  author    = {Pham, Hung Manh and Tang, Jialu and Saeed, Aaqib and Ma, Dong},
  booktitle = {Proceedings of the European Conference on Artificial Intelligence (ECAI)},
  series    = {Frontiers in Artificial Intelligence and Applications},
  volume    = {413},
  pages     = {4545--4552},
  year      = {2025},
  publisher = {IOS Press},
  doi       = {10.3233/FAIA251356}
}

Please refer to our GitHub repo for more details!