kz919

This is a LyCORIS adapter derived from stabilityai/stable-diffusion-3-medium-diffusers. The main validation prompt used during training was: Validation settings - CFG: `3.0` - CFG Rescale: `0.0` - Steps: `20` - Sampler: `FlowMatchEulerDiscreteScheduler` - Seed: `42` - Resolutions: `1024x1024,1280x768` Note: The validation settings are not necessarily the same as the training settings. You can find some example images in the following gallery: The text encoder was not trained. You may reuse the base model text encoder for inference. - Training epochs: 1 - Training steps: 10000 - Learning rate: 0.0001 - Learning rate schedule: polynomial - Warmup steps: 100 - Max grad value: 2.0 - Effective batch size: 32 - Micro-batch size: 32 - Gradient accumulation steps: 1 - Number of GPUs: 1 - Gradient checkpointing: True - Prediction type: flowmatching (extra parameters=['shift=3']) - Optimizer: adamwbf16 - Trainable parameter precision: Pure BF16 - Base model precision: `nochange` - Caption dropout probability: 0.1% multi-reward-diffusiondb-sd3 - Repeats: 0 - Total number of images: 200000 - Total number of aspect buckets: 1 - Resolution: 1.048576 megapixels - Cropped: True - Crop style: center - Crop aspect: square - Used for regularisation data: No

Model Details This model is a 1B llama3 model pretrained from scratch with torchtitan on fineweb-edu with AdamW optimizer. 20x chinchilla rule for 20B tokens seen. | Tasks |Version|Filter|n-shot| Metric | | Value | |Stderr| |--------------|------:|------|-----:|----------|---|------:|---|-----:| |arcchallenge | 1|none | 0|acc |↑ | 0.2688|± |0.0130| | | |none | 0|accnorm |↑ | 0.2875|± |0.0132| |arceasy | 1|none | 0|acc |↑ | 0.6082|± |0.0100| | | |none | 0|accnorm |↑ | 0.5412|± |0.0102| |hellaswag | 1|none | 0|acc |↑ | 0.3459|± |0.0047| | | |none | 0|accnorm |↑ | 0.4169|± |0.0049| |lambadaopenai| 1|none | 0|acc |↑ | 0.3311|± |0.0066| | | |none | 0|perplexity|↓ |38.2983|± |1.5427| |openbookqa | 1|none | 0|acc |↑ | 0.2340|± |0.0190| | | |none | 0|accnorm |↑ | 0.3500|± |0.0214| |piqa | 1|none | 0|acc |↑ | 0.6795|± |0.0109| | | |none | 0|accnorm |↑ | 0.6774|± |0.0109| MMLU | Groups |Version|Filter|n-shot|Metric| |Value | |Stderr| |------------------|------:|------|------|------|---|-----:|---|-----:| |mmlu | 2|none | |acc |↑ |0.2529|± |0.0037| | - humanities | 2|none | |acc |↑ |0.2459|± |0.0063| | - other | 2|none | |acc |↑ |0.2424|± |0.0077| | - social sciences| 2|none | |acc |↑ |0.2697|± |0.0080| | - stem | 2|none | |acc |↑ |0.2572|± |0.0078|

Model Details This model is a 1B llama3 model pretrained from scratch with torchtitan on fineweb-edu with CAdamW optimizer. 20B tokens seen. Downstream Eval ARC, Hellaswag, LambdaOpenAI, OpenbookQA, PIQA | Tasks |Version|Filter|n-shot| Metric | | Value | |Stderr| |--------------|------:|------|-----:|----------|---|------:|---|-----:| |arcchallenge | 1|none | 0|acc |↑ | 0.2688|± |0.0130| | | |none | 0|accnorm |↑ | 0.2978|± |0.0134| |arceasy | 1|none | 0|acc |↑ | 0.6090|± |0.0100| | | |none | 0|accnorm |↑ | 0.5362|± |0.0102| |hellaswag | 1|none | 0|acc |↑ | 0.3473|± |0.0048| | | |none | 0|accnorm |↑ | 0.4193|± |0.0049| |lambadaopenai| 1|none | 0|acc |↑ | 0.3229|± |0.0065| | | |none | 0|perplexity|↓ |42.2319|± |1.7220| |openbookqa | 1|none | 0|acc |↑ | 0.2360|± |0.0190| | | |none | 0|accnorm |↑ | 0.3380|± |0.0212| |piqa | 1|none | 0|acc |↑ | 0.6779|± |0.0109| | | |none | 0|accnorm |↑ | 0.6768|± |0.0109| MMLU | Groups |Version|Filter|n-shot|Metric| |Value | |Stderr| |------------------|------:|------|------|------|---|-----:|---|-----:| |mmlu | 2|none | |acc |↑ |0.2535|± |0.0037| | - humanities | 2|none | |acc |↑ |0.2578|± |0.0064| | - other | 2|none | |acc |↑ |0.2446|± |0.0077| | - social sciences| 2|none | |acc |↑ |0.2610|± |0.0079| | - stem | 2|none | |acc |↑ |0.2487|± |0.0077|

Model Details This model is a 1B llama3 model pretrained from scratch with torchtitan on fineweb-edu with AdamW optimizer. 100B tokens seen. Downstream Eval ARC, Hellaswag, LambdaOpenAI, OpenbookQA, PIQA | Tasks |Version|Filter|n-shot| Metric | | Value | |Stderr| |--------------|------:|------|-----:|----------|---|------:|---|-----:| |arcchallenge | 1|none | 0|acc |↑ | 0.3123|± |0.0135| | | |none | 0|accnorm |↑ | 0.3413|± |0.0139| |arceasy | 1|none | 0|acc |↑ | 0.6768|± |0.0096| | | |none | 0|accnorm |↑ | 0.5922|± |0.0101| |hellaswag | 1|none | 0|acc |↑ | 0.4007|± |0.0049| | | |none | 0|accnorm |↑ | 0.5092|± |0.0050| |lambadaopenai| 1|none | 0|acc |↑ | 0.3901|± |0.0068| | | |none | 0|perplexity|↓ |21.6290|± |0.7689| |openbookqa | 1|none | 0|acc |↑ | 0.2660|± |0.0198| | | |none | 0|accnorm |↑ | 0.3680|± |0.0216| |piqa | 1|none | 0|acc |↑ | 0.7127|± |0.0106| | | |none | 0|accnorm |↑ | 0.7100|± |0.0106| MMLU | Groups |Version|Filter|n-shot|Metric| |Value | |Stderr| |------------------|------:|------|------|------|---|-----:|---|-----:| |mmlu | 2|none | |acc |↑ |0.2515|± |0.0037| | - humanities | 2|none | |acc |↑ |0.2451|± |0.0063| | - other | 2|none | |acc |↑ |0.2716|± |0.0080| | - social sciences| 2|none | |acc |↑ |0.2476|± |0.0078| | - stem | 2|none | |acc |↑ |0.2452|± |0.0076|

Model Details This model is a 1B llama3 model pretrained from scratch with torchtitan on fineweb-edu with CAdamW optimizer. 100B tokens seen. Downstream Eval ARC, Hellaswag, LambdaOpenAI, OpenbookQA, PIQA | Tasks |Version|Filter|n-shot| Metric | | Value | |Stderr| |--------------|------:|------|-----:|----------|---|------:|---|-----:| |arcchallenge | 1|none | 0|acc |↑ | 0.3183|± |0.0136| | | |none | 0|accnorm |↑ | 0.3379|± |0.0138| |arceasy | 1|none | 0|acc |↑ | 0.6650|± |0.0097| | | |none | 0|accnorm |↑ | 0.6061|± |0.0100| |hellaswag | 1|none | 0|acc |↑ | 0.3999|± |0.0049| | | |none | 0|accnorm |↑ | 0.5025|± |0.0050| |lambadaopenai| 1|none | 0|acc |↑ | 0.3912|± |0.0068| | | |none | 0|perplexity|↓ |23.8709|± |0.8855| |openbookqa | 1|none | 0|acc |↑ | 0.2580|± |0.0196| | | |none | 0|accnorm |↑ | 0.3740|± |0.0217| |piqa | 1|none | 0|acc |↑ | 0.7116|± |0.0106| | | |none | 0|accnorm |↑ | 0.7149|± |0.0105| MMLU | Groups |Version|Filter|n-shot|Metric| |Value | |Stderr| |------------------|------:|------|------|------|---|-----:|---|-----:| |mmlu | 2|none | |acc |↑ |0.2519|± |0.0037| | - humanities | 2|none | |acc |↑ |0.2540|± |0.0064| | - other | 2|none | |acc |↑ |0.2527|± |0.0078| | - social sciences| 2|none | |acc |↑ |0.2480|± |0.0078| | - stem | 2|none | |acc |↑ |0.2518|± |0.0077|