Instructions to use AstroMLab/astrollama-2-7b-base_abstract with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AstroMLab/astrollama-2-7b-base_abstract with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AstroMLab/astrollama-2-7b-base_abstract")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("AstroMLab/astrollama-2-7b-base_abstract")
model = AutoModelForCausalLM.from_pretrained("AstroMLab/astrollama-2-7b-base_abstract")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use AstroMLab/astrollama-2-7b-base_abstract with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AstroMLab/astrollama-2-7b-base_abstract"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AstroMLab/astrollama-2-7b-base_abstract",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/AstroMLab/astrollama-2-7b-base_abstract

SGLang

How to use AstroMLab/astrollama-2-7b-base_abstract with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AstroMLab/astrollama-2-7b-base_abstract" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AstroMLab/astrollama-2-7b-base_abstract",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AstroMLab/astrollama-2-7b-base_abstract" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AstroMLab/astrollama-2-7b-base_abstract",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use AstroMLab/astrollama-2-7b-base_abstract with Docker Model Runner:
```
docker model run hf.co/AstroMLab/astrollama-2-7b-base_abstract
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

AstroLLaMA-2-7B-Base_Abstract

AstroLLaMA-2-7B-Abstract is a specialized base language model for astronomy, developed by fine-tuning Meta's LLaMA-2-7b architecture on astronomical literature. This model was originally developed by the AstroLLaMA team as part of the UniverseTBD initiative. It is designed for next token prediction tasks and is not an instruct/chat model.

Note: This model is provided for completeness in the series of AstroLLaMA models. The core AstroLLaMA team has since moved on to develop more advanced models under AstroMLab. For the original UniverseTBD version, please visit their repository.

Model Details

Base Architecture: LLaMA-2-7b
Training Data: Abstracts from 326,238 astronomy papers from arXiv's astro-ph category (April 1992 to July 2023)
Fine-tuning Method: Parameter-Efficient Fine-Tuning (PEFT) with LowRank Adaptation (LoRA)
Primary Use: Next token prediction for astronomy-related text generation and analysis
Reference: Nguyen et al. 2023

Generating text from a prompt

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load the model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("AstroMLab/astrollama-2-7b-base_abstract")
model = AutoModelForCausalLM.from_pretrained("AstroMLab/astrollama-2-7b-base_abstract", device_map="auto")

# Create the pipeline with explicit truncation
from transformers import pipeline
generator = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    device_map="auto",
    truncation=True,
    max_length=512
)

# Example prompt from an astronomy paper
prompt = "In this letter, we report the discovery of the highest redshift, " \
    "heavily obscured, radio-loud QSO candidate selected using JWST NIRCam/MIRI, " \
    "mid-IR, sub-mm, and radio imaging in the COSMOS-Web field. "

# Set seed for reproducibility
torch.manual_seed(42)

# Generate text
generated_text = generator(prompt, do_sample=True)
print(generated_text[0]['generated_text'])

Model Limitations and Biases

This model is specifically trained on astronomy abstracts and may not generalize well to other domains. Users should be aware of potential biases in the training data, which may reflect historical trends and biases in astronomical research publications.

Importantly, this model has been superseded by more advanced versions. Here's a performance comparison chart based upon the astronomical benchmarking Q&A as described in Ting et al. 2024.

Model	Score (%)
AstroSage-LLaMA-3.1-8B (AstroMLab)	80.9
AstroLLaMA-2-70B (AstroMLab)	76.0
LLaMA-3.1-8B	73.7
Gemma-2-9B	71.5
Qwen-2.5-7B	70.4
Yi-1.5-9B	68.4
InternLM-2.5-7B	64.5
Mistral-7B-v0.3	63.9
ChatGLM3-6B	50.4
AstroLLaMA-2-7B-AIC	44.3
AstroLLaMA-2-7B-Abstract	43.5

As shown, AstroLLaMA-2-7B series are outperformed by newer models. For state-of-the-art performance, we recommend using the latest models.

Ethical Considerations

While this model is designed for scientific use, users should be mindful of potential misuse, such as generating misleading scientific content. Always verify model outputs against peer-reviewed sources for critical applications.

Citation

If you use this model in your research, please cite:

@ARTICLE{2023arXiv230906126D,
       author = {{Dung Nguyen}, Tuan and {Ting}, Yuan-Sen and {Ciuc{\u{a}}}, Ioana and {O'Neill}, Charlie and {Sun}, Ze-Chang and {Jab{\l}o{\'n}ska}, Maja and {Kruk}, Sandor and {Perkowski}, Ernest and {Miller}, Jack and {Li}, Jason and {Peek}, Josh and {Iyer}, Kartheik and {R{\'o}{\.z}a{\'n}ski}, Tomasz and {Khetarpal}, Pranav and {Zaman}, Sharaf and {Brodrick}, David and {Rodr{\'\i}guez M{\'e}ndez}, Sergio J. and {Bui}, Thang and {Goodman}, Alyssa and {Accomazzi}, Alberto and {Naiman}, Jill and {Cranney}, Jesse and {Schawinski}, Kevin and {UniverseTBD}},
        title = "{AstroLLaMA: Towards Specialized Foundation Models in Astronomy}",
      journal = {arXiv e-prints},
     keywords = {Astrophysics - Instrumentation and Methods for Astrophysics, Astrophysics - Cosmology and Nongalactic Astrophysics, Astrophysics - Astrophysics of Galaxies, Astrophysics - High Energy Astrophysical Phenomena, Computer Science - Computation and Language, Computer Science - Machine Learning},
         year = 2023,
        month = sep,
          eid = {arXiv:2309.06126},
        pages = {arXiv:2309.06126},
          doi = {10.48550/arXiv.2309.06126},
archivePrefix = {arXiv},
       eprint = {2309.06126},
 primaryClass = {astro-ph.IM},
       adsurl = {https://ui.adsabs.harvard.edu/abs/2023arXiv230906126D},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}