JetBrains

Model Description Mellum-4b-sft-python is a fine-tuned version of JetBrains' first open-source large language model (LLM) optimized for code-related tasks. Pre-trained on over 4 trillion tokens with a context window of 8192 tokens across multiple programming languages, and then fine-tuned, Mellum-4b-sft-python is tailored specifically for code completion in Python. The model follows a LLaMA-style architecture with 4 billion parameters, making it efficient for both cloud inference (e.g., via vLLM) and local deployment (e.g., using llama.cpp or Ollama). Mellum was trained using Automatic Mixed Precision (AMP) with bf16 precision. The uploaded version on Hugging Face retains the bf16 format for public use. Designed for integration into professional developer tooling (e.g., intelligent code suggestions in IDEs), AI-powered coding assistants, and research on code understanding and generation, Mellum is also well-suited for educational applications and fine-tuning experiments. Limitations - Biases: May reflect biases present in public codebases. For example it will likely produce code which is similar in style to the open-source repositories. - Security: Code suggestions should not be assumed to be secure or free of vulnerabilities. Sample Usage Here is an example of how to run and sample from the model with additional files context and fill in the middle. Fill in the middle with additional files as context generation Contact For questions, collaborations and requests reach us out via [email protected]

Model Description Mellum-4b-dpo-all is the third stage of our pipeline (after pretraining and SFT), trained with direct preference optimization on code-quality preferences to produce more readable, useful code. Pre-trained on over 4 trillion tokens with a context window of 8192 tokens across multiple programming languages, and then fine-tuned, Mellum-4b-dpo-all is tailored for context-aware code completion tasks. It was fine-tuned on a diverse set of languages, including Batchfile, C, C#, CMake, C++, CSS, Cython, Dockerfile, F#, Go, Groovy, HCL, HTML (and variants like Django, EEx, ERB, and PHP templates), Java, JSP, JavaScript, JSX, Kotlin, Less, Makefile, Objective-C++, PHP, PowerShell, Python, R, RHTML, Ruby, Rust, Sass, Scala, SCSS, Shell, SQL, Swift, TOML, TypeScript, Visual Basic, Vue, and YAML. The model follows a LLaMA-style architecture with 4 billion parameters, making it efficient for both cloud inference (e.g., via vLLM) and local deployment (e.g., using llama.cpp or Ollama). Mellum was trained using Automatic Mixed Precision (AMP) with bf16 precision. The uploaded version on Hugging Face retains the bf16 format for public use. Designed for integration into professional developer tooling (e.g., intelligent code suggestions in IDEs), AI-powered coding assistants, and research on code understanding and generation, Mellum is also well-suited for educational applications and fine-tuning experiments. Limitations - Biases: May reflect biases present in public codebases. For example it will likely produce code which is similar in style to the open-source repositories. - Security: Code suggestions should not be assumed to be secure or free of vulnerabilities. Sample Usage Here is an example of how to run and sample from the model with additional files context and fill in the middle. Fill in the middle with additional files as context generation Contact For questions, collaborations and requests reach us out via [email protected]

Model Description Mellum-4b-dpo-all is the third stage of our pipeline (after pretraining and SFT), trained with direct preference optimization on code-quality preferences to produce more readable, useful code. Pre-trained on over 4 trillion tokens with a context window of 8192 tokens across multiple programming languages, and then fine-tuned, Mellum-4b-dpo-all is tailored for context-aware code completion tasks. It was fine-tuned on a diverse set of languages, including Batchfile, C, C#, CMake, C++, CSS, Cython, Dockerfile, F#, Go, Groovy, HCL, HTML (and variants like Django, EEx, ERB, and PHP templates), Java, JSP, JavaScript, JSX, Kotlin, Less, Makefile, Objective-C++, PHP, PowerShell, Python, R, RHTML, Ruby, Rust, Sass, Scala, SCSS, Shell, SQL, Swift, TOML, TypeScript, Visual Basic, Vue, and YAML. The model follows a LLaMA-style architecture with 4 billion parameters, making it efficient for both cloud inference (e.g., via vLLM) and local deployment (e.g., using llama.cpp or Ollama). Mellum was trained using Automatic Mixed Precision (AMP) with bf16 precision. The uploaded version on Hugging Face retains the bf16 format for public use. Designed for integration into professional developer tooling (e.g., intelligent code suggestions in IDEs), AI-powered coding assistants, and research on code understanding and generation, Mellum is also well-suited for educational applications and fine-tuning experiments. Limitations - Biases: May reflect biases present in public codebases. For example it will likely produce code which is similar in style to the open-source repositories. - Security: Code suggestions should not be assumed to be secure or free of vulnerabilities. Sample Usage Here is an example of how to run and sample from the model with additional files context and fill in the middle. Contact For questions, collaborations and requests reach us out via [email protected]

Model Description Mellum-4b-dpo-python is the third stage of our pipeline (after pretraining and SFT), trained with direct preference optimization on code-quality preferences to produce more readable, useful code. Pre-trained on over 4 trillion tokens with a context window of 8192 tokens across multiple programming languages, and then fine-tuned, Mellum-4b-dpo-python is tailored specifically for code completion in Python. The model follows a LLaMA-style architecture with 4 billion parameters, making it efficient for both cloud inference (e.g., via vLLM) and local deployment (e.g., using llama.cpp or Ollama). Mellum was trained using Automatic Mixed Precision (AMP) with bf16 precision. The uploaded version on Hugging Face retains the bf16 format for public use. Designed for integration into professional developer tooling (e.g., intelligent code suggestions in IDEs), AI-powered coding assistants, and research on code understanding and generation, Mellum is also well-suited for educational applications and fine-tuning experiments. Limitations - Biases: May reflect biases present in public codebases. For example it will likely produce code which is similar in style to the open-source repositories. - Security: Code suggestions should not be assumed to be secure or free of vulnerabilities. - Format: This model is suitable mostly for FIM Completion objective with context's files. Sample Usage Here are examples of how to run and sample from the model. Contact For questions, collaborations and requests reach us out via [email protected]

Model Description Mellum-4b-dpo-python is the third stage of our pipeline (after pretraining and SFT), trained with direct preference optimization on code-quality preferences to produce more readable, useful code. Pre-trained on over 4 trillion tokens with a context window of 8192 tokens across multiple programming languages, and then fine-tuned, Mellum-4b-dpo-python is tailored specifically for code completion in Python. The model follows a LLaMA-style architecture with 4 billion parameters, making it efficient for both cloud inference (e.g., via vLLM) and local deployment (e.g., using llama.cpp or Ollama). Mellum was trained using Automatic Mixed Precision (AMP) with bf16 precision. The uploaded version on Hugging Face retains the bf16 format for public use. Designed for integration into professional developer tooling (e.g., intelligent code suggestions in IDEs), AI-powered coding assistants, and research on code understanding and generation, Mellum is also well-suited for educational applications and fine-tuning experiments. Limitations - Biases: May reflect biases present in public codebases. For example it will likely produce code which is similar in style to the open-source repositories. - Security: Code suggestions should not be assumed to be secure or free of vulnerabilities. Sample Usage Here is an example of how to run and sample from the model with additional files context and fill in the middle. Fill in the middle with additional files as context generation Contact For questions, collaborations and requests reach us out via [email protected]