SimCSE: Simple Contrastive Learning of Sentence Embeddings
Paper • 2104.08821 • Published
How to use mrp/simcse-model-distil-m-bert with sentence-transformers:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("mrp/simcse-model-distil-m-bert")
sentences = [
"That is a happy person",
"That is a happy dog",
"That is a very happy person",
"Today is a sunny day"
]
embeddings = model.encode(sentences)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [4, 4]How to use mrp/simcse-model-distil-m-bert with Transformers:
# Load model directly
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("mrp/simcse-model-distil-m-bert")
model = AutoModel.from_pretrained("mrp/simcse-model-distil-m-bert")This is a sentence-transformers by using m-Distil-BERT as the baseline model model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.
We use SimCSE here and training the model with Thai Wikipedia here
Using this model becomes easy when you have sentence-transformers installed:
pip install -U sentence-transformers
Then you can use the model like this:
from sentence_transformers import SentenceTransformer
sentences = ["ฉันนะคือคนรักชาติยังไงละ!", "พวกสามกีบล้มเจ้า!"]
model = SentenceTransformer('{MODEL_NAME}')
embeddings = model.encode(sentences)
print(embeddings)