Dream-org

This is the instruct model of Dream 7B, which is an open diffusion large language model with top-tier performance. More details about the model and usage can be found in the blog and github bellow: - Blog: https://hkunlp.github.io/blog/2025/dream/ - Github: https://github.com/HKUNLP/Dream

This is the base model of Dream 7B, which is an open diffusion large language model with top-tier performance. More details about the model and usage can be found in the blog and github bellow: - Blog: https://hkunlp.github.io/blog/2025/dream/ - Github: https://github.com/HKUNLP/Dream

Dream-Coder 7B is a diffusion LLM for code trained exclusively on open-source data across its development stages—adaptation, supervised fine-tuning, and reinforcement learning. It achieves an impressive 21.4% pass@1 on LiveCodeBench (2410-2505), outperforming other open-source diffusion LLMs by a wide margin. More details about the model and usage can be found in the blog and github bellow: - Blog: https://hkunlp.github.io/blog/2025/dream-coder/ - Github: https://github.com/DreamLM/Dream-Coder Quickstart To get start with, please install `transformers==4.46.2` and `torch==2.5.1`. Here is an example to use Dream-Coder 7B:

This is the base model of Dream Coder 7B, which is an open diffusion LLM for code with top-tier performance. More details about the model and usage can be found in the blog and github bellow: - Blog: https://hkunlp.github.io/blog/2025/dream-coder/ - Github: https://github.com/DreamLM/Dream-Coder

DreamOn-v0-7B DreamOn is a novel discrete diffusion algorithm designed to address the variable-length generation challenge in code infilling. Unlike current discrete diffusion language models, our approach enables dynamic expansion and contraction of mask tokens during inference, providing flexible length control without requiring predetermined canvas sizes. Github: https://github.com/DreamLM/DreamOn Quick Start Parameters - `inputids`: The input token ids. - `maxnewtokens`: The maximum tokens to generate. Note that the context length (input+output) of Dream currently is 2048. And the mask added to the prompt is counted as new tokens. Therefore, `maxnewtokens` can not be set to a value smaller than the number of mask tokens in the prompt. For example, if you set `numberofmask` to 4, then `maxnewtokens` should be at least 4. - `outputhistory`: Whether to return the output at each intermediate step. - `returndictingenerate`: The output format, mostly set to True. - `numberoftransfertokens`: The number of tokens to predict at each denoising step. We mainly test our model with `numberoftransfertokens` set to 1. Other settings are not fully tested. - `temperature`: The value used to module the next token probabilities. By default 0.0. The smaller the value, the more accurate the results (e.g., in math or coding). The larger the value, the more diverse the results (e.g., in general conversation). If you notice repeated results, you might consider increasing the temperature. - `topp`: If set to float < 1, only the smallest set of most probable tokens with probabilities that add up to topp or higher are kept for generation. By default None. Control the diversity of generation. - `topk`: The number of highest probability vocabulary tokens to keep for top-k-filtering. By default None. Control the diversity of generation. - `alg`: The remasking strategy in diffusion sampling, controlling the token generation order. Support one random strategy and three confidence-based strategies: - `maskgitplus`: Token will be generated based on the top1 confidence from https://arxiv.org/abs/2202.04200. - `topkmargin`: Token will be generated based on the margin confidence by taking `top1 - top2` from https://arxiv.org/abs/2502.06768. - `entropy`: Token will be generated based on the entropy of each token distribution. - `algtemp`: Add some randomness to `alg` when using confidence-based strategies. By default None. Note: We currently do not support attention mask, as we recompute attention mask each denoising step to support variable-length generation.