Can this model be run on a Turing GPU (No Flash Attention support)?
#1
by rsbdev - opened
Hi I was trying to test this model on a 2080TI but I get an error saying that FA2 isn't installed , my GPU does not support FA2 so can I run this model using a different backend and if so how?
Edit: Also which languages does this model support? I cant find that information anywhere.
We are integrating it into HF to run the model easily.
yes, if you are trying on official demos, --attn_implementation sdpa may works.