zed-industries

This repository contains a fine-tuned version of Qwen2.5-Coder-7B to support edit prediction in Zed. The model has been fine-tuned using the zeta dataset. If you want to fine-tune the model yourself, you can refer to the following scripts: - DPO Fine-Tuning: View Notebook - SFT Fine-Tuning: View Notebook The dataset used for training is available at: zed-industries/zeta `vllm serve zed-industries/zeta --served-model-name zeta` - Quantization vLLM supports FP8 (8-bit floating point) weight and activation quantization using hardware acceleration on GPUs such as Nvidia H100 and AMD MI300x. - NGram Speculative Decoding configures vLLM to use speculative decoding where proposals are generated by matching n-grams in the prompt. This is a great fit for edit predictions since many of the tokens are already present in the prompt and the model is only needed to generate changes to the code file. `vllm serve zed-industries/zeta --served-model-name zeta --enable-prefix-caching --enable-chunked-prefill --quantization="fp8" --speculative-model [ngram] --ngram-prompt-lookup-max 4 --ngram-prompt-lookup-min 2 --num-speculative-tokens 8` For more insights about the model and its integration in Zed, check out the official blog post: Zed Blog - Edit Prediction