ltg

This is deberta-v2-xxlarge updated to implement the `AutoModelForCausalLM` class, enabling it to generate text. This implementation is based on our paper "BERTs are Generative In-Context Learners". This repository also fixes three bugs in the original HF implementation of DeBERTa: 1. We fixed the incorrect name of the output embedding weights in the checkpoint file; 2. We fixed the implementation of the enhanced mask decoder (EMD), based on the original GitHub repository; 3. We clamp the positional embeddings so that they work with long sequence lengths. If you find DeBERTa useful for your work, please cite the following paper:

This model is a version of CohereLabs/aya-101, fine-tuned on datasets of Finnish usage examples and definitions. It generates definitions of Finnish words in context. Its input is the usage example and the instruction question ". Mitä tarkoittaa \ ?" - Github repository: MultilingualDefGen - Paper: EMNLP 2025 Findings The model is intended for research purposes, as a source of contextualized dictionary-like lexical definitions. The fine-tuning datasets were limited to Finnish. Although the original model is multilingual, we did not evaluate its ability to generate definitions in other languages. Generated definitions can contain all sorts of biases and stereotypes, stemming from the underlying language model and raw dictionary data.

This model is a version of CohereLabs/aya-101, fine-tuned on datasets of Finnish usage examples and definitions. It generates definitions of Finnish words in context. Its input is the usage example and the instruction question ". Mitä tarkoittaa \ ?" - Github repository: MultilingualDefGen - Paper: EMNLP 2025 Findings The model is intended for research purposes, as a source of contextualized dictionary-like lexical definitions. The fine-tuning datasets were limited to Finnish. Although the original model is multilingual, we did not evaluate its ability to generate definitions in other languages. Generated definitions can contain all sorts of biases and stereotypes, stemming from the underlying language model and raw dictionary data.

This model is a version of bigscience/mt0-xl, fine-tuned on datasets of German usage examples and definitions. It generates definitions of German words in context. Its input is the usage example and the instruction question ". Was ist die Definition von \ ?" - Github repository: MultilingualDefGen - Paper: EMNLP 2025 Findings The model is intended for research purposes, as a source of contextualized dictionary-like lexical definitions. The fine-tuning datasets were limited to German. Although the original model is multilingual, we did not evaluate its ability to generate definitions in other languages. Generated definitions can contain all sorts of biases and stereotypes, stemming from the underlying language model and raw dictionary data.

This model is a version of bigscience/mt0-xl, fine-tuned on datasets of Russian usage examples and definitions. It generates definitions of Russian words in context. Its input is the usage example and the instruction question "Что такое ?" - Github repository: MultilingualDefGen - Paper: EMNLP 2025 Findings The model is intended for research purposes, as a source of contextualized dictionary-like lexical definitions. The fine-tuning datasets were limited to Russian. Although the original model is multilingual, we did not evaluate its ability to generate definitions in other languages. Generated definitions can contain all sorts of biases and stereotypes, stemming from the underlying language model and raw dictionary data.

This model is a version of Unbabel/TowerInstruct-7B-v0.2, fine-tuned on datasets of Russian usage examples and definitions. It generates definitions of Russian words in context. Its input is the usage example and the instruction question "Что такое \ ?" - Github repository: MultilingualDefGen - Paper: EMNLP 2025 Findings The model is intended for research purposes, as a source of contextualized dictionary-like lexical definitions. The fine-tuning datasets were limited to Russian. Although the original model is multilingual, we did not evaluate its ability to generate definitions in other languages. Generated definitions can contain all sorts of biases and stereotypes, stemming from the underlying language model and raw dictionary data.

This model is a version of Unbabel/TowerInstruct-7B-v0.2, fine-tuned on datasets of German usage examples and definitions. It generates definitions of German words in context. Its input is the usage example and the instruction question ". Was ist die Definition von \ ?" - Github repository: MultilingualDefGen - Paper: EMNLP 2025 Findings The model is intended for research purposes, as a source of contextualized dictionary-like lexical definitions. The fine-tuning datasets were limited to German. Although the original model is multilingual, we did not evaluate its ability to generate definitions in other languages. Generated definitions can contain all sorts of biases and stereotypes, stemming from the underlying language model and raw dictionary data.

This model is a version of Unbabel/TowerInstruct-7B-v0.2, fine-tuned on datasets of Russian usage examples and definitions. It generates definitions of Russian words in context. Its input is the usage example and the instruction question "Что такое ?" - Github repository: MultilingualDefGen - Paper: EMNLP 2025 Findings The model is intended for research purposes, as a source of contextualized dictionary-like lexical definitions. The fine-tuning datasets were limited to Russian. Although the original model is multilingual, we did not evaluate its ability to generate definitions in other languages. Generated definitions can contain all sorts of biases and stereotypes, stemming from the underlying language model and raw dictionary data.

This model is a version of Unbabel/TowerInstruct-7B-v0.2, fine-tuned on datasets of Finnish usage examples and definitions. It generates definitions of Finnish words in context. Its input is the usage example and the instruction question ". Mitä tarkoittaa ?" - Github repository: MultilingualDefGen - Paper: EMNLP 2025 Findings The model is intended for research purposes, as a source of contextualized dictionary-like lexical definitions. The fine-tuning datasets were limited to Finnish. Although the original model is multilingual, we did not evaluate its ability to generate definitions in other languages. Generated definitions can contain all sorts of biases and stereotypes, stemming from the underlying language model and raw dictionary data.

This model is a version of Unbabel/TowerInstruct-7B-v0.2, fine-tuned on datasets of Finnish usage examples and definitions. It generates definitions of Finnish words in context. Its input is the usage example and the instruction question ". Mitä tarkoittaa ?" - Github repository: MultilingualDefGen - Paper: EMNLP 2025 Findings The model is intended for research purposes, as a source of contextualized dictionary-like lexical definitions. The fine-tuning datasets were limited to Finnish. Although the original model is multilingual, we did not evaluate its ability to generate definitions in other languages. Generated definitions can contain all sorts of biases and stereotypes, stemming from the underlying language model and raw dictionary data.