JayHyeon

Model Card for pythia-2.8b-rDPO5e-71.0vpoconstant-1ep0.3labelsmoothing This model is a fine-tuned version of EleutherAI/pythia-2.8b on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.19.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

Model Card for pythia-2.8b-cDPO5e-71.0vpoconstant-1ep0.3labelsmoothing This model is a fine-tuned version of EleutherAI/pythia-2.8b on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.19.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

Model Card for pythia-2.8b-cDPO5e-71.0vpoconstant-1ep0.1labelsmoothing This model is a fine-tuned version of EleutherAI/pythia-2.8b on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.19.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

This model is a fine-tuned version of EleutherAI/pythia-2.8b on the HuggingFaceH4/ultrafeedbackbinarized dataset. It has been trained using TRL. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

Model Card for pythia-2.8b-rDPO5e-71.0vpoconstant-1ep0.1labelsmoothing This model is a fine-tuned version of EleutherAI/pythia-2.8b on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

Model Card for Qwen0.5-rDPO5e-70.1lsmooth-1.0vpoconstant This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the JayHyeon/shp-dpo-converted dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.55.0 - Pytorch: 2.7.1 - Datasets: 3.4.1 - Tokenizers: 0.21.1

This model is a fine-tuned version of meta-llama/Llama-2-7b-hf on the HuggingFaceH4/ultrafeedbackbinarized dataset. It has been trained using TRL. - TRL: 0.22.0.dev0 - Transformers: 4.55.0 - Pytorch: 2.7.1 - Datasets: 4.0.0 - Tokenizers: 0.21.4

Model Card for pythia-2.8b-VIPO5e-71.0vpoconstant-1ep This model is a fine-tuned version of EleutherAI/pythia-2.8b on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.19.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

Model Card for Qwen1.5B-math-DPO5e-61.0vpoconstant-10ep This model is a fine-tuned version of Qwen/Qwen2.5-Math-1.5B on the argilla/distilabel-math-preference-dpo dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

Model Card for Qwen0.5-IRPO1e-6-3ep0.01alp0.5bdpolam0dpoplam This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the JayHyeon/shp-dpo-converted dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

Model Card for Qwen0.5-IRPO1e-6-3ep0.005alp0.5bdpolam0dpoplam This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

Model Card for Qwen0.5-cDPO5e-70.1lsmooth-1.0vpoconstant-1ep This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the JayHyeon/shp-dpo-converted dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

A model designed for fine-tuning with specific parameters for optimal performance in various tasks.

Base model trained on ultrafeedback binarized datasets using the transformers library.

Base model JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep datasets trl-lib/ultrafeedback_binarized library name transformers.

This model is a fine-tuned version of EleutherAI/pythia-2.8b on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.13.0.dev0 - Transformers: 4.47.0.dev0 - Pytorch: 2.5.1 - Datasets: 3.1.0 - Tokenizers: 0.20.3

Model Card for Qwen0.5-IRPO1e-6-3ep2alp0.5bdpolam0dpoplam This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the JayHyeon/shp-dpo-converted dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the JayHyeon/shp-dpo-converted dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

Core model for fine-tuning with datasets from HuggingFaceH4/ultrafeedback_binarized using the transformers library.

Qwen 2.5 0.5B is a model trained on the HuggingFaceH4 ultrafeedback binarized dataset using the transformers library.

Core purpose is to provide a model based on Qwen/Qwen2.5-0.5B. It utilizes datasets from HuggingFaceH4/ultrafeedback_binarized and is built using the transformers library.

A model designed for various natural language processing tasks using the transformers library.

A model designed for fine-tuning tasks using the transformers library.

The model is based on JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep and utilizes datasets from trl-lib/ultrafeedback_binarized with the transformers library.

The model is based on JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep and utilizes datasets from trl-lib/ultrafeedback_binarized, implemented using the transformers library.

A model designed for fine-tuning tasks using the Qwen architecture, optimized for performance with specific training parameters.

Library name: transformers.

Model Card for Qwen1.5B-math-DPO5e-61.0vpoconstant-5ep This model is a fine-tuned version of Qwen/Qwen2.5-Math-1.5B on the argilla/distilabel-math-preference-dpo dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

Model Card for pythia-2.8b-VDPO5e-71.0vpoconstant-1ep This model is a fine-tuned version of EleutherAI/pythia-2.8b on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.19.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

Model Card for Qwen1.5B-math-VIPO5e-61.0vpoconstant-5ep This model is a fine-tuned version of Qwen/Qwen2.5-Math-1.5B on the argilla/distilabel-math-preference-dpo dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

Model Card for Qwen1.5B-math-VIPO5e-610.0vpoconstant-5ep This model is a fine-tuned version of Qwen/Qwen2.5-Math-1.5B on the argilla/distilabel-math-preference-dpo dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

Model Card for Qwen1.5B-math-DPO1e-51.0vpoconstant-5ep This model is a fine-tuned version of Qwen/Qwen2.5-Math-1.5B on the argilla/distilabel-math-preference-dpo dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

Model Card for pythia-2.8b-DPO1e-61.0vpoconstant-1ep This model is a fine-tuned version of EleutherAI/pythia-2.8b on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.19.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

Model Card for Qwen1.5B-math-DPO5e-51.0vpoconstant-10ep This model is a fine-tuned version of Qwen/Qwen2.5-Math-1.5B on the argilla/distilabel-math-preference-dpo dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

Model Card for Qwen1.5B-math-DPO5e-51.0vpoconstant-20ep This model is a fine-tuned version of Qwen/Qwen2.5-Math-1.5B on the argilla/distilabel-math-preference-dpo dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

This model is a fine-tuned version of EleutherAI/pythia-2.8b on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.13.0.dev0 - Transformers: 4.47.0.dev0 - Pytorch: 2.5.1 - Datasets: 3.1.0 - Tokenizers: 0.20.3

Model Card for Qwen1.5B-math-VDPO1e-41.0vpoconstant-10ep This model is a fine-tuned version of Qwen/Qwen2.5-Math-1.5B on the argilla/distilabel-math-preference-dpo dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.19.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

Model Card for pythia-2.8b-VDPO5e-710.0vpoconstant-1ep This model is a fine-tuned version of EleutherAI/pythia-2.8b on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.19.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

Model Card for Qwen0.5-DPO3e-61.0vpoconstant-1ep0.3flip This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.19.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

Model Card for Qwen0.5-VDPO3e-61.0vpoconstant-1ep0.3flip This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

Model Card for Qwen0.5-cDPO3e-61.0vpoconstant-1ep0.3flip This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

Model Card for Qwen0.5-IRPO5e-7-3ep0.1alp0.5bdpolam0dpoplam This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.13.0.dev0 - Transformers: 4.47.0.dev0 - Pytorch: 2.5.1 - Datasets: 3.1.0 - Tokenizers: 0.20.3

Model Card for Qwen0.5-IRPO5e-7-3ep10alp0.5bdpolam0dpoplam This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.13.0.dev0 - Transformers: 4.47.0.dev0 - Pytorch: 2.5.1 - Datasets: 3.1.0 - Tokenizers: 0.20.3

Model Card for Qwen0.5-IRPO1e-6-3ep5alp0.5bdpolam0dpoplam This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

Model Card for Qwen0.5-IRPO5e-7-3ep0.05alp0.5bdpolam0dpoplam This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.53.0 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.2

Model Card for Qwen0.5-IRPO1e-6-3ep0.25alp0.5bdpolam0dpoplam This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

Model Card for Qwen0.5-BDPO1e-6-3ep0alp0.99999bdpolam0dpoplam This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.13.0.dev0 - Transformers: 4.47.0.dev0 - Pytorch: 2.5.1 - Datasets: 3.1.0 - Tokenizers: 0.20.3

Model Card for gemma-BDPO5e-7-1ep0alp0.5bdpolam0dpoplam This model is a fine-tuned version of google/gemma-3-1b-it on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.53.0 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.2

Model Card for gemma-IRPO5e-7-1ep1alp0.5bdpolam0dpoplam This model is a fine-tuned version of google/gemma-3-1b-it on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.53.0 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.2

Model Card for gemma-DPO5e-7-1ep0alp0.5bdpolam0dpoplam This model is a fine-tuned version of google/gemma-3-1b-it on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.53.0 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.2

This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5Bultrainteractsft2e-51ep on the JayHyeon/trlultrainteract-pair dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.19.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

Model Card for Qwen0.5-BDPO5e-7-3ep0alp0.999bdpolam0dpoplam This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

Model Card for llama-IRPO1e-6-2ep1alp0.5bdpolam0dpoplam This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.53.0 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.2

Model Card for llama-BDPO1e-6-2ep0alp0.5bdpolam0dpoplam This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.53.0 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.2

Model Card for llama-DPOP1e-6-2ep0alp0.5bdpolam5dpoplam This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.53.0 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.2

Model Card for llama-DPO1e-7-1ep0alp0.5bdpolam0dpoplam This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.53.0 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.2

Model Card for llama-BDPO2e-7-1ep0alp0.5bdpolam0dpoplam This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.53.0 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.2

This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5Bultrainteractsft2e-51ep on the JayHyeon/trlultrainteract-pair dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.53.0 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.2

Model Card for llama-DPO2e-7-1ep0alp0.5bdpolam0dpoplam This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.53.0 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.2

Model Card for llama-BDPO3e-6-1ep0alp0.5bdpolam0dpoplam This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.53.0 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.2

Model Card for llama-IRPO3e-6-1ep1alp0.5bdpolam0dpoplam This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.53.0 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.2

Qwen 2.5 0.5B SFT is trained on the HuggingFaceH4 ultrafeedback binarized dataset using the transformers library.

Qwen 2.5 0.5B is trained on the HuggingFaceH4 ultrafeedback binarized dataset using the transformers library.

Qwen 2.5 0.5B is designed for fine-tuning with datasets from HuggingFaceH4/ultrafeedback_binarized using the transformers library.

The model is based on Qwen/Qwen2.5-0.5B and utilizes datasets from HuggingFaceH4/ultrafeedback_binarized with the library transformers.

Base model trained on datasets from trl-lib/ultrafeedback_binarized using the transformers library.

library_name: transformers tags: [] Provide a quick summary of what the model is/does.

A model designed for various natural language processing tasks using the transformers library.

A model designed for various natural language processing tasks using the transformers library.

The model is based on JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep and utilizes datasets from trl-lib/ultrafeedback_binarized with the transformers library.

A model designed for specific tasks using the transformers library.

The model is based on JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep and utilizes datasets from trl-lib/ultrafeedback_binarized with the transformers library.

The model is based on JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep and utilizes datasets from trl-lib/ultrafeedback_binarized with the transformers library.

Base model: Qwen2.5 0.5B SFT 2e 5 2ep. Datasets: trl-lib/ultrafeedback_binarized. Library name: transformers.

Base model JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep datasets trl-lib/ultrafeedback_binarized library name transformers.

Library name: transformers.

Library name: transformers. Tags: [] Provide a quick summary of what the model is/does.

This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the openbmb/UltraInteractpair dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.13.0.dev0 - Transformers: 4.47.0.dev0 - Pytorch: 2.5.1 - Datasets: 3.1.0 - Tokenizers: 0.20.3

Base model JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep datasets trl-lib/ultrafeedback_binarized library name transformers.

Model Card for Qwen1.5B-math-VDPO5e-71.0vpoconstant-20ep This model is a fine-tuned version of Qwen/Qwen2.5-Math-1.5B on the argilla/distilabel-math-preference-dpo dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

Model Card for Qwen0.5-VIPO3e-61.0vpoconstant-1ep0.3flip This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.15.2 - Transformers: 4.50.0 - Pytorch: 2.6.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1

Model Card for Qwen0.5-DPOP1e-6-3ep0alp0.5bdpolam500dpoplam This model is a fine-tuned version of JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.19.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

Model Card for llama-DPO1e-6-2ep0alp0.5bdpolam0dpoplam This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.53.0 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.2

Model Card for llama-IRPO2e-7-1ep1alp0.5bdpolam0dpoplam This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.53.0 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.2

Model Card for llama-DPOP2e-7-1ep0alp0.5bdpolam5dpoplam This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the trl-lib/ultrafeedbackbinarized dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.20.0.dev0 - Transformers: 4.53.0 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.2

Model Card for Qwen0.5-rDPO5e-71.0vpoconstant0.1labelsmoothing This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the JayHyeon/shp-dpo-converted dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.19.0.dev0 - Transformers: 4.52.4 - Pytorch: 2.7.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1

This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the JayHyeon/shp-dpo-converted dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.22.0.dev0 - Transformers: 4.55.0 - Pytorch: 2.8.0 - Datasets: 4.0.0 - Tokenizers: 0.21.4

Model Card for Qwen0.5-DPO5e-71.0vpoconstantls0.0seed42 This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the JayHyeon/shp-dpo-converted dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.22.0.dev0 - Transformers: 4.55.0 - Pytorch: 2.8.0 - Datasets: 4.0.0 - Tokenizers: 0.21.4

Model Card for Qwen0.5-VDPO5e-71.0vpoconstantls0.0seed42 This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the JayHyeon/shp-dpo-converted dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.22.0.dev0 - Transformers: 4.55.0 - Pytorch: 2.8.0 - Datasets: 4.0.0 - Tokenizers: 0.21.4

Model Card for Qwen0.5-IPO5e-71.0vpoconstantls0.0seed42 This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the JayHyeon/shp-dpo-converted dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.22.0.dev0 - Transformers: 4.55.0 - Pytorch: 2.8.0 - Datasets: 4.0.0 - Tokenizers: 0.21.4

Model Card for Qwen0.5-VIPO5e-71.0vpoconstantls0.0seed42 This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the JayHyeon/shp-dpo-converted dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.22.0.dev0 - Transformers: 4.55.0 - Pytorch: 2.8.0 - Datasets: 4.0.0 - Tokenizers: 0.21.4

Model Card for Qwen0.5-rDPO5e-71.0vpoconstantls0.1seed42 This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the JayHyeon/shp-dpo-converted dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.22.0.dev0 - Transformers: 4.55.0 - Pytorch: 2.8.0 - Datasets: 4.0.0 - Tokenizers: 0.21.4

Model Card for Qwen0.5-rDPO5e-71.0vpoconstantls0.3seed42 This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.22.0.dev0 - Transformers: 4.55.0 - Pytorch: 2.8.0 - Datasets: 4.0.0 - Tokenizers: 0.21.4

Model Card for Qwen0.5-cDPO5e-71.0vpoconstantls0.1seed42 This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the JayHyeon/shp-dpo-converted dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.22.0.dev0 - Transformers: 4.55.0 - Pytorch: 2.8.0 - Datasets: 4.0.0 - Tokenizers: 0.21.4

Model Card for Qwen0.5-cDPO5e-71.0vpoconstantls0.3seed42 This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the JayHyeon/shp-dpo-converted dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.22.0.dev0 - Transformers: 4.55.0 - Pytorch: 2.8.0 - Datasets: 4.0.0 - Tokenizers: 0.21.4

Qwen 2.5 0.5B Instruct is designed for instruction-based tasks using the Hugging Face Ultrafeedback binarized dataset with the Transformers library.

A model designed for efficient natural language processing tasks, utilizing the transformers library under the MIT license.

A model designed for efficient natural language processing tasks, utilizing the transformers library under the MIT license.

A model designed for various natural language processing tasks, utilizing the transformers library and licensed under MIT.

This model is designed for instruction-based tasks using the Qwen architecture. It is built with the Transformers library and is licensed under MIT.

This model is designed for instruction-based tasks using the Qwen architecture. It is built on the transformers library and is licensed under MIT.

This model is designed for instruction-based tasks using the Qwen architecture. It is built on the transformers library and is licensed under MIT.

A model that utilizes the transformers library and is licensed under MIT.

Qwen 2.5 0.5B is designed for fine-tuning with datasets from HuggingFaceH4/ultrafeedback_binarized using the transformers library.

Core model for fine-tuning with datasets from HuggingFaceH4/ultrafeedback_binarized using the transformers library.

Core model for fine-tuning with datasets from HuggingFaceH4/ultrafeedback_binarized using the transformers library.

Qwen 2.5 0.5B is a model trained on the HuggingFaceH4 ultrafeedback binarized dataset using the transformers library.

Core model for fine-tuning with datasets from HuggingFaceH4/ultrafeedback_binarized using the transformers library.

Qwen 2.5 0.5B is a model trained on the HuggingFaceH4 ultrafeedback binarized dataset using the transformers library.

Qwen 2.5 0.5B is designed for fine-tuning with datasets from HuggingFaceH4's ultrafeedback_binarized. It utilizes the transformers library.

Core model for fine-tuning with datasets from HuggingFaceH4/ultrafeedback_binarized using the transformers library.

Qwen 2.5 0.5B is designed for fine-tuning with datasets from HuggingFaceH4/ultrafeedback_binarized using the transformers library.

Qwen 2.5 0.5B is designed for fine-tuning tasks using the Hugging Face Transformers library with the ultrafeedback_binarized dataset.

Core model for Qwen 2.5 with 0.5B parameters, trained on the HuggingFaceH4 ultrafeedback binarized dataset using the transformers library.

Qwen 2.5 0.5B is designed for fine-tuning with datasets from HuggingFaceH4/ultrafeedback_binarized using the transformers library.

Qwen 2.5 0.5B is designed for fine-tuning with datasets from HuggingFaceH4/ultrafeedback_binarized using the transformers library.

Core purpose is to provide a fine-tuned model based on JayHyeon/Qwen2.5-0.5B-SFT-2e-5-5ep. It utilizes datasets from trl-lib/ultrafeedback_binarized and is built using the transformers library.

library_name: transformers tags: [] Provide a quick summary of what the model is/does.

library_name: transformers tags: [] Provide a quick summary of what the model is/does.

Library name: transformers.

A model designed for various natural language processing tasks using the transformers library.

The model is based on JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep and utilizes datasets from trl-lib/ultrafeedback_binarized with the library transformers.

The model is based on JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep and utilizes datasets from trl-lib/ultrafeedback_binarized, implemented using the transformers library.

Library name: transformers. Tags: [] Provide a quick summary of what the model is/does.

The model is based on JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep and utilizes datasets from trl-lib/ultrafeedback_binarized, implemented using the transformers library.

The model is based on JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep and utilizes datasets from trl-lib/ultrafeedback_binarized, implemented using the transformers library.

A model designed for fine-tuning tasks using the transformers library.

A model designed for various tasks using the transformers library.

The model is based on JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep and utilizes datasets from trl-lib/ultrafeedback_binarized, designed for use with the transformers library.

The model is based on JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep and utilizes datasets from trl-lib/ultrafeedback_binarized, designed for use with the transformers library.

A model designed for various natural language processing tasks, utilizing the transformers library.

A model designed for various tasks using the transformers library.

A model designed for various natural language processing tasks, utilizing the transformers library.

A model designed for efficient fine-tuning and deployment, utilizing advanced techniques for optimal performance.

A model designed for various natural language processing tasks using the transformers library.

A model designed for various natural language processing tasks, utilizing the transformers library.

A model designed for various natural language processing tasks using the transformers library.

A model designed for various tasks using the transformers library.

A model designed for various natural language processing tasks, utilizing the transformers library.

A model designed for various tasks using the transformers library.

A model designed for fine-tuning and optimization tasks using the transformers library.

A model designed for various tasks using the transformers library.

The model is based on JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep and utilizes datasets from trl-lib/ultrafeedback_binarized, implemented using the transformers library.

A model designed for various tasks using the transformers library.

A model designed for various natural language processing tasks using the transformers library.

A model designed for fine-tuning and optimization tasks using the transformers library.

The model is based on JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep and utilizes datasets from trl-lib/ultrafeedback_binarized, designed for use with the transformers library.

Core model for fine-tuning with datasets from trl-lib/ultrafeedback_binarized using the transformers library.

A model designed for fine-tuning with specific parameters for enhanced performance in various tasks.

A model designed for fine-tuning tasks using the transformers library.

The model is based on JayHyeon/Qwen2.5-0.5B-SFT-2e-5-2ep and utilizes datasets from trl-lib/ultrafeedback_binarized, implemented using the transformers library.

library_name: transformers tags: [] Provide a quick summary of what the model is/does.

library_name: transformers

Library name: transformers.

Base model: Qwen2.5 0.5B SFT 2e-5 2ep datasets: trl-lib/ultrafeedback_binarized library name: transformers

Base model: Qwen2.5 0.5B SFT 2e-5 2ep datasets: trl-lib/ultrafeedback_binarized library name: transformers

Base model: Qwen2.5 0.5B SFT 2e-5 2ep datasets: trl-lib/ultrafeedback_binarized library name: transformers

Base model: Qwen2.5 0.5B SFT 2e 5 2ep datasets: trl-lib/ultrafeedback_binarized library name: transformers

Base model: Qwen 2.5 0.5B SFT 2e 5 2ep. Datasets: trl-lib/ultrafeedback_binarized. Library name: transformers.

Library name: transformers. Tags: [] Provide a quick summary of what the model is/does.

Library name: transformers.

This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the openbmb/UltraInteractpair dataset. It has been trained using TRL. This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - TRL: 0.13.0.dev0 - Transformers: 4.47.0.dev0 - Pytorch: 2.5.1 - Datasets: 3.1.0 - Tokenizers: 0.20.3

A model designed for efficient natural language processing tasks, utilizing the Transformers library under the MIT license.