ende-chat-0.0.5

3
2 languages
license:apache-2.0
by
kno10
Language Model
OTHER
New
3 downloads
Early-stage
Try on Hugging FaceAdd to Compare
Edge AI:
Mobile
Laptop
Server
Unknown
Mobile
Laptop
Server
Quick Summary

Preliminary LoRA finetune of Mistral-7B for German and English quality text.

Code Examples

text
--stage sft \
    --model_name_or_path ende-0.0.5c3 \
    --finetuning_type lora \
    --template default \
    --dataset_dir data \
    --dataset sharegpt-deutsch,oasst_de,dolly_15k_de,openschnabeltier_de,ultrachat_de,evol_instruct,evol_instruct_de,alpaca-gpt4_de,dolphin_de,airoboros_de \
    --cutoff_len 1024 \
    --learning_rate 5e-05 \
    --num_train_epochs 1.0 \
    --per_device_train_batch_size 4 \
    --gradient_accumulation_steps 8 \
    --lr_scheduler_type cosine \
    --neftune_noise_alpha 0 \
    --lora_target all \
    --lora_rank 8 \
    --lora_dropout 0 \
    --fp16 True \
text
--stage sft \
    --model_name_or_path ende-0.0.5c3 \
    --finetuning_type lora \
    --template default \
    --dataset_dir data \
    --dataset sharegpt-deutsch,oasst_de,dolly_15k_de,openschnabeltier_de,ultrachat_de,evol_instruct,evol_instruct_de,alpaca-gpt4_de,dolphin_de,airoboros_de \
    --cutoff_len 1024 \
    --learning_rate 5e-05 \
    --num_train_epochs 1.0 \
    --per_device_train_batch_size 4 \
    --gradient_accumulation_steps 8 \
    --lr_scheduler_type cosine \
    --neftune_noise_alpha 0 \
    --lora_target all \
    --lora_rank 8 \
    --lora_dropout 0 \
    --fp16 True \
text
--stage sft \
    --model_name_or_path ende-0.0.5c3 \
    --finetuning_type lora \
    --template default \
    --dataset_dir data \
    --dataset sharegpt-deutsch,oasst_de,dolly_15k_de,openschnabeltier_de,ultrachat_de,evol_instruct,evol_instruct_de,alpaca-gpt4_de,dolphin_de,airoboros_de \
    --cutoff_len 1024 \
    --learning_rate 5e-05 \
    --num_train_epochs 1.0 \
    --per_device_train_batch_size 4 \
    --gradient_accumulation_steps 8 \
    --lr_scheduler_type cosine \
    --neftune_noise_alpha 0 \
    --lora_target all \
    --lora_rank 8 \
    --lora_dropout 0 \
    --fp16 True \
text
--stage sft \
    --model_name_or_path ende-0.0.5c3 \
    --finetuning_type lora \
    --template default \
    --dataset_dir data \
    --dataset sharegpt-deutsch,oasst_de,dolly_15k_de,openschnabeltier_de,ultrachat_de,evol_instruct,evol_instruct_de,alpaca-gpt4_de,dolphin_de,airoboros_de \
    --cutoff_len 1024 \
    --learning_rate 5e-05 \
    --num_train_epochs 1.0 \
    --per_device_train_batch_size 4 \
    --gradient_accumulation_steps 8 \
    --lr_scheduler_type cosine \
    --neftune_noise_alpha 0 \
    --lora_target all \
    --lora_rank 8 \
    --lora_dropout 0 \
    --fp16 True \
text
--stage sft \
    --model_name_or_path ende-0.0.5c3 \
    --finetuning_type lora \
    --template default \
    --dataset_dir data \
    --dataset sharegpt-deutsch,oasst_de,dolly_15k_de,openschnabeltier_de,ultrachat_de,evol_instruct,evol_instruct_de,alpaca-gpt4_de,dolphin_de,airoboros_de \
    --cutoff_len 1024 \
    --learning_rate 5e-05 \
    --num_train_epochs 1.0 \
    --per_device_train_batch_size 4 \
    --gradient_accumulation_steps 8 \
    --lr_scheduler_type cosine \
    --neftune_noise_alpha 0 \
    --lora_target all \
    --lora_rank 8 \
    --lora_dropout 0 \
    --fp16 True \
text
--stage sft \
    --model_name_or_path ende-0.0.5c3 \
    --finetuning_type lora \
    --template default \
    --dataset_dir data \
    --dataset sharegpt-deutsch,oasst_de,dolly_15k_de,openschnabeltier_de,ultrachat_de,evol_instruct,evol_instruct_de,alpaca-gpt4_de,dolphin_de,airoboros_de \
    --cutoff_len 1024 \
    --learning_rate 5e-05 \
    --num_train_epochs 1.0 \
    --per_device_train_batch_size 4 \
    --gradient_accumulation_steps 8 \
    --lr_scheduler_type cosine \
    --neftune_noise_alpha 0 \
    --lora_target all \
    --lora_rank 8 \
    --lora_dropout 0 \
    --fp16 True \
text
--stage sft \
    --model_name_or_path ende-0.0.5c3 \
    --finetuning_type lora \
    --template default \
    --dataset_dir data \
    --dataset sharegpt-deutsch,oasst_de,dolly_15k_de,openschnabeltier_de,ultrachat_de,evol_instruct,evol_instruct_de,alpaca-gpt4_de,dolphin_de,airoboros_de \
    --cutoff_len 1024 \
    --learning_rate 5e-05 \
    --num_train_epochs 1.0 \
    --per_device_train_batch_size 4 \
    --gradient_accumulation_steps 8 \
    --lr_scheduler_type cosine \
    --neftune_noise_alpha 0 \
    --lora_target all \
    --lora_rank 8 \
    --lora_dropout 0 \
    --fp16 True \

Deploy This Model

Production-ready deployment in minutes

Together.ai

Instant API access to this model

Fastest API

Production-ready inference API. Start free, scale to millions.

Try Free API

Replicate

One-click model deployment

Easiest Setup

Run models in the cloud with simple API. No DevOps required.

Deploy Now

Disclosure: We may earn a commission from these partners. This helps keep LLMYourWay free.