| --- |
| datasets: |
| - brucewayne0459/Skin_diseases_and_care |
| language: |
| - en |
| license: mit |
| tags: |
| - medical |
| - dermatology |
| - skin_disease |
| - skin_care |
| - unsloth |
| - trl |
| - sft |
| --- |
| |
| ## Model Details |
|
|
| ### Model Description |
|
|
| <!-- Provide a longer summary of what this model is. --> |
|
|
|
|
|
|
| - **Developed by:** Bruce_Wayne(The Batman) |
| - **Model type:** Text Generation |
| - **Finetuned from model [optional]:** OpenBioLLM(llama-3)(aaditya/Llama3-OpenBioLLM-8B) |
| |
| ## You can find the gguf versions here --> https://huggingface.co/brucewayne0459/OpenBioLLm-Derm-gguf |
| ### please let me know how the model works -->https://forms.gle/N14zZTkLpUr6Hf4BA |
| ### Thank you! |
| ## Uses |
| |
| ### Direct Use |
| |
| This model is fine-tuned on skin diseases and dermatology data and is used for a dermatology chatbot to provide clear, accurate, and helpful information about various skin diseases, skin care routines, treatments, and related dermatological advice. |
| |
| |
| ## Bias, Risks, and Limitations |
| |
| This model is trained on dermatology data, which might contain inherent biases. It is important to note that the model's responses should not be considered a substitute for professional medical advice. There may be limitations in understanding rare skin conditions or those not well-represented in the training data. |
| The model still need to be fine-tuned further to get accurate answers. |
| |
| ### Recommendations |
| |
| Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations. |
| |
| ## How to Get Started with the Model |
| |
| Use the code below to get started with the model. |
| |
| ```python |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| |
| model_name = "brucewayne0459/OpenBioLLm-Derm" |
| tokenizer = AutoTokenizer.from_pretrained(model_name) |
| model = AutoModelForCausalLM.from_pretrained(model_name) |
| ``` |
| |
| ## Training Details |
| |
| ### Training Data |
| |
| The model is fine-tuned on a dataset containing information about various skin diseases and dermatology care. |
| brucewayne0459/Skin_diseases_and_care |
| |
| ### Training Procedure |
| |
| <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. --> |
| |
| #### Preprocessing [optional] |
| |
| """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. |
| |
| ### Instruction: |
| You are a highly knowledgeable and empathetic dermatologist. Provide clear, accurate, and helpful information about various skin diseases, skin care routines, treatments, and related dermatological advice. |
| |
| ### Input: |
| {} |
| |
| ### Response: |
| {} |
| """ |
| EOS_TOKEN = tokenizer.eos_token # Must add EOS_TOKEN |
| |
| def formatting_prompts_func(examples): |
| inputs = examples["Topic"] |
| outputs = examples["Information"] |
| texts = [] |
| |
| Prompt passed while fine tuning the model |
| #### Training Hyperparameters |
| |
| Training regime: The model was trained using the following hyperparameters: |
| Per device train batch size: 2 |
| Gradient accumulation steps: 4 |
| Warmup steps: 5 |
| Max steps: 120 |
| Learning rate: 2e-4 |
| Optimizer: AdamW (8-bit) |
| Weight decay: 0.01 |
| LR scheduler type: Linear |
| |
| |
| |
| ## Environmental Impact |
| |
| <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly --> |
| |
| Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). |
| |
| - **Hardware Type:** Tesls T4 gpu |
| - **Hours used:** 1hr |
| - **Cloud Provider:** Google Colab |
| |
| |
| ## Technical Specifications [optional] |
| |
| ### Model Architecture and Objective |
| |
| This model is based on the LLaMA (Large Language Model Meta AI) architecture and fine-tuned to provide dermatological advice. |
| |
| |
| #### Hardware |
| |
| The training was performed on Tesla T4 gpu with 4-bit quantization and gradient checkpointing to optimize memory usage. |
| |
| |
| ### Feel free to provide any missing details or correct the assumptions made, and I'll update the model card accordingly. |
| |
| |
