dicta-il

Adapting LLMs to Hebrew: Unveiling DictaLM 2.0 with Enhanced Vocabulary and Instruction Capabilities The DictaLM-2.0-Instruct Large Language Model (LLM) is an instruct fine-tuned version of the DictaLM-2.0 generative model using a variety of conversation datasets. For full details of this model please read our release blog post or the technical report. This is the instruct-tuned full-precision model designed for chat. You can try the model out on a live demo here. You can view and access the full collection of base/instruct unquantized/quantized versions of `DictaLM-2.0` here. In order to leverage instruction fine-tuning, your prompt should be surrounded by `[INST]` and `[/INST]` tokens. The very first instruction should begin with a begin of sentence id. The next instructions should not. The assistant generation will be ended by the end-of-sentence token id. This format is available as a chat template via the `applychattemplate()` method: DictaLM-2.0-Instruct follows the Zephyr-7B-beta recipe for fine-tuning an instruct model, with an extended instruct dataset for Hebrew. The DictaLM 2.0 Instruct model is a demonstration that the base model can be fine-tuned to achieve compelling performance. It does not have any moderation mechanisms. We're looking forward to engaging with the community on ways to make the model finely respect guardrails, allowing for deployment in environments requiring moderated outputs.

Adapting LLMs to Hebrew: Unveiling DictaLM 2.0 with Enhanced Vocabulary and Instruction Capabilities The DictaLM-2.0 Large Language Model (LLM) is a pretrained generative text model with 7 billion parameters trained to specialize in Hebrew text. For full details of this model please read our release blog post or the technical report. This is the full-precision base model. You can view and access the full collection of base/instruct unquantized/quantized versions of `DictaLM-2.0` here. There are already pre-quantized 4-bit models using the `GPTQ` and `AWQ` methods available for use: DictaLM-2.0-AWQ and DictaLM-2.0-GPTQ. For dynamic quantization on the go, here is sample code which loads the model onto the GPU using the `bitsandbytes` package, requiring : DictaLM-2.0 is based on the Mistral-7B-v0.1 model with the following changes: - An extended tokenizer with 1,000 injected tokens specifically for Hebrew, increasing the compression rate from 5.78 tokens/word to 2.76 tokens/word. - Continued pretraining on over 190B tokens of naturally occuring text, 50% Hebrew and 50% English. DictaLM 2.0 is a pretrained base model and therefore does not have any moderation mechanisms.

DictaBERT: A State-of-the-Art BERT Suite for Modern Hebrew State-of-the-art language model for Hebrew, released here. This is the fine-tuned model for the joint parsing of the following tasks: - Prefix Segmentation - Morphological Disabmgiuation - Lexicographical Analysis (Lemmatization) - Syntactical Parsing (Dependency-Tree) - Named-Entity Recognition A live demo of the model with instant visualization of the syntax tree can be found here. For a faster model, you can use the equivalent bert-tiny model for this task here. For the bert-base models for other tasks, see here. 1. JSON: The model returns a JSON object for each sentence in the input, where for each sentence we have the sentence text, the NER entities, and the list of tokens. For each token we include the output from each of the tasks. 1. UD: The model returns the full UD output for each sentence, according to the style of the Hebrew UD Treebank. 1. UD, in the style of IAHLT: This model returns the full UD output, with slight modifications to match the style of IAHLT. This differences are mostly granularity of some dependency relations, how the suffix of a word is broken up, and implicit definite articles. The actual tagging behavior doesn't change. If you only need the output for one of the tasks, you can tell the model to not initialize some of the heads, for example: The list of options are: `dolex`, `dosyntax`, `doner`, `doprefix`, `domorph`. You can also choose to get your response in UD format: If you use DictaBERT-joint in your research, please cite This work is licensed under a [Creative Commons Attribution 4.0 International License][cc-by].

DictaBERT-char-spacefix: A finetuned BERT model for restoring missing spaces in Hebrew texts. DictaBERT-char-spacefix is a finetuned BERT model based on dicta-il/dictabert-char, for the task of restoring missing spaces in Hebrew text. This model is released to the public in this 2025 W-NUT paper: Avi Shmidman and Shaltiel Shmidman, "Restoring Missing Spaces in Scraped Hebrew Social Media", The 10th Workshop on Noisy and User-generated Text (W-NUT), 2025 If you use DictaBERT-char-spacefix in your research, please cite This work is licensed under a [Creative Commons Attribution 4.0 International License][cc-by].