Deepseek Coder 1.3b Typescript GGUF
[CodeGPT.co] | [🦙 Ollama] | [Discord] | [VSCode Extension]
CodeGPTPlus/deepseek-coder-1.3b-typescript, emerges as a fine-tuned iteration of deepseek-ai/deepseek-coder-1.3b-base, meticulously crafted by the CodeGPT team to excel in generating expert code in TypeScript. With specific fine-tuning for TypeScript and a dataset of 0.5B tokens, this model excels in producing precise and efficient solutions in this programming language.
The 16K window size and an additional fill-in-the-middle task are employed to deliver project-level code completion.
This new model stands as the ideal choice for those seeking a specialized code generator for TypeScript, backed by the expertise of the CodeGPT team.
It achieves the following results on the evaluation set: - Loss: 0.7681
How to Use This model is for completion purposes only. Here give some examples of how to use the model.
Running with Ollama Model: https://ollama.ai/codegpt/deepseek-coder-1.3b-typescript
Running with Ollama and CodeGPT Autocomplete in VSCode
Documentation: https://docs.codegpt.co/docs/tutorial-features/codeautocompletion
Select "Ollama - codegpt/deepseek-coder-1.3b-typescript" in the autocomplete model selector.
Then, write any code or comment in the vscode text editor, and the model will provide you with code suggestions through the CodeGPT code autocomplete.
The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 20 - evalbatchsize: 20 - seed: 42 - gradientaccumulationsteps: 2 - totaltrainbatchsize: 40 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-06 - lrschedulertype: cosine - lrschedulerwarmupsteps: 261 - numepochs: 1
| Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-----:|:-----:|:---------------:| | 1.0745 | 0.0 | 1 | 0.8681 | | 1.2267 | 0.05 | 1308 | 0.8130 | | 1.1594 | 0.1 | 2616 | 0.8018 | | 0.7674 | 0.15 | 3924 | 0.7942 | | 0.6443 | 0.2 | 5232 | 0.7889 | | 0.9155 | 0.25 | 6540 | 0.7847 | | 0.7501 | 0.3 | 7848 | 0.7819 | | 0.8835 | 0.35 | 9156 | 0.7792 | | 0.7261 | 0.4 | 10464 | 0.7769 | | 0.9746 | 0.45 | 11772 | 0.7748 | | 0.6884 | 0.5 | 13080 | 0.7734 | | 0.6104 | 0.55 | 14388 | 0.7722 | | 0.8876 | 0.6 | 15696 | 0.7710 | | 0.9567 | 0.65 | 17004 | 0.7703 | | 0.6915 | 0.7 | 18312 | 0.7696 | | 0.8874 | 0.75 | 19620 | 0.7691 | | 0.6124 | 0.8 | 20928 | 0.7686 | | 0.8147 | 0.85 | 22236 | 0.7684 | | 0.8021 | 0.9 | 23544 | 0.7683 | | 0.8665 | 0.95 | 24852 | 0.7681 |
- Transformers 4.37.0.dev0 - Pytorch 2.0.1+cu118 - Datasets 2.16.1 - Tokenizers 0.15.0