ZennyKenny

Novoyaz 20B — Pre-reform → Modern Russian Orthography on gpt-oss-20b Novoyaz 20B is a specialized text-generation model built on top of OpenAI’s open-weight gpt-oss-20b. > Rewrite дореформенный (pre-revolution / pre-1918) Russian text into modern Russian orthography, preserving meaning and punctuation, without adding explanations, commentary, or translation. - A task-specific model configuration (`novoyaz-20b`) set up for 4-bit inference with bitsandbytes. - A custom inference handler (`handler.py`) used for Hugging Face Inference Endpoints, which: - Loads `openai/gpt-oss-20b` (or `unsloth/gpt-oss-20b-bnb-4bit` via an env flag). - Wraps inputs in a fixed, Russian-language instruction prompt. - A LoRA adapter checkpoint under `checkpoint-60/` (trained with Unsloth + TRL on `unsloth/gpt-oss-20b-unsloth-bnb-4bit`) for advanced users who want to attach the adapter manually. > Note: The current production `handler.py` loads the base `openai/gpt-oss-20b` model and applies a strict system prompt for orthography normalization. > The LoRA adapter in `checkpoint-60/` is provided for offline / custom integration and future experiments. - Input: Russian text in pre-reform / дореформенная orthography (e.g. word-final hard signs `ъ`, letters like `ѣ`, old spellings, etc.), often coming from OCR of older printed materials. - Output: The same text in modern Russian spelling, with: - Meaning preserved - Punctuation preserved as much as possible - No commentary, explanation, or translation The production handler uses the following instruction (simplified): > Ты – модель, которая строго переписывает дореформенный русский текст > в современную орфографию, не меняя смысл и пунктуацию. > Не добавляй комментарии и не переводь текст. - Normalizing OCR output from pre-1918 printed sources - Preparing historical Russian texts for NLP pipelines that expect modern orthography - Making archival / émigré Russian materials more accessible without changing their meaning - General chat / assistant behavior - Translation into other languages - Content moderation or classification - Creative writing or long-form generation unrelated to orthography normalization Outside this domain the model may still produce usable text, but behavior is not tuned or evaluated. The snippet below mirrors what `handler.py` does internally: - Loads `openai/gpt-oss-20b` - Wraps your text with a fixed instruction prefix and suffix - Generates a non-sampled, deterministic normalization You can switch to your own quantized or fine-tuned variant by changing `modelid` to point to a local directory or a Hugging Face model that includes your weights. 2.2 Using this repo’s Inference Endpoint / HF Inference API If you deploy this repository (`ZennyKenny/novoyaz-20b`) as an Inference Endpoint, `handler.py` expects requests in the following format: - Uses deterministic generation (`dosample = False`, `temperature = 0.0`, `numbeams = 1`). - Limits responses to `GENMAXNEWTOKENS` (default 512; configurable via environment variable). - Truncates very long inputs to 2048 tokens to avoid pathological cases. The directory `checkpoint-60/` contains a LoRA adapter trained with Unsloth’s SFT trainer and TRL on: - Base model: `unsloth/gpt-oss-20b-unsloth-bnb-4bit` This is exposed as a PEFT adapter that you can attach manually to a compatible base model: The `checkpoint-60/README.md` contains the auto-generated PEFT model card template; this top-level `README.md` provides the task-specific details and recommended usage. - Base architecture: `GptOssForCausalLM` (`modeltype: "gptoss"`) - Hidden size: 2880 - Layers: 24 transformer decoder layers - Attention heads: 64 - Key/value heads: 8 - Mixture-of-Experts (MoE): - `expertspertoken`: 4 - `numlocalexperts`: 32 - `outputrouterlogits`: false - Context & RoPE: - `maxpositionembeddings`: 131072 - `initialcontextlength`: 4096 - `ropescaling`: YaRN-style scaling up to 131k tokens - Quantization config (bitsandbytes): - `loadin4bit: true` - `bnb4bitquanttype: "nf4"` - `bnb4bitcomputedtype: "bfloat16"` - `bnb4bitusedoublequant: true` - `llmint8skipmodules`: ["router", "lmhead", "embedtokens"] This config mirrors `openai/gpt-oss-20b`, but includes an explicit 4-bit quantization config so the model can be loaded efficiently on consumer hardware with bitsandbytes. The LoRA adapter in `checkpoint-60/` was trained using: - Base model: `unsloth/gpt-oss-20b-unsloth-bnb-4bit` - Method: Supervised fine-tuning (SFT) with LoRA - Frameworks: - Unsloth SFT trainer (for efficient fine-tuning and long context) - TRL (Transformers Reinforcement Learning library by Hugging Face) - Transformers, Accelerate, BitsAndBytes - TRL: 0.21.0 - Transformers: 4.56.0.dev0 - PyTorch: 2.8.0 - Datasets: 3.6.0 - Tokenizers: 0.21.4 (See `checkpoint-60/trainingargs.bin` and `checkpoint-60/trainerstate.json` for exact hyperparameters.) - Source: Russian texts written in pre-reform orthography (pre-1918, plus émigré usage). - Target: The same texts normalized to modern orthography. - Excerpts from public-domain historical materials that preserve pre-reform spelling. - Manually or semi-automatically normalized versions in modern spelling. - Augmented examples to cover edge cases (rare letters, tricky word forms, punctuation quirks, mild OCR noise). At this time, the exact dataset is not being released; if a cleaned public subset becomes available later, it will be linked from this model card. Novoyaz 20B is evaluated on fidelity of orthography conversion, not on generic benchmarks: - Does the output preserve the original meaning and sentence structure? - Does it follow modern Russian spelling norms? - Does it avoid adding commentary, translation, or explanations? Current evaluation is mostly qualitative and task-specific, including: - Manual review on held-out parallel examples. - Spot-checking OCR output from historical books. - Comparison to base `gpt-oss-20b` without the strict normalization prompt or adapter. No standardized numerical benchmarks are published yet. If you run your own evaluations (e.g. on a test set of historical texts), contributions via issues / PRs are very welcome. - Domain-specific: The model is tuned for pre-reform → modern Russian orthography. It is not a general chat model. - OCR noise: Very noisy OCR (missing characters, mis-segmented words, etc.) can cause: - Over-aggressive “fixing” of grammar/meaning. - Occasional hallucinated corrections where the original glyphs are unreadable. - Edge cases: Very archaic or regional spellings not present in the training set may be normalized incorrectly or inconsistently. - The base model `gpt-oss-20b` and its training data. - The historical texts used for fine-tuning (which may contain outdated or discriminatory language). The task is narrowly defined — rewriting spelling — so harmful or biased content present in the input will usually be preserved (in modern orthography) rather than removed. - Safety filtering - Content moderation - Debiasing text For sensitive domains, consider chaining Novoyaz 20B with dedicated moderation / safety models. - Researchers, students, and archivists work with pre-revolutionary and émigré Russian sources. - NLP practitioners integrate historical Russian text into modern pipelines. - Does not attempt to change the meaning, ideology, or tone of the text. - Does make the spelling conform to post-1918 standards, so the text can be read and processed more easily. Historical texts necessarily reflect the values and biases of their time. Modernized spelling does not imply endorsement, and users should approach such content with critical historical awareness. If you use Novoyaz 20B in academic work or products, please consider citing both: OpenAI. “gpt-oss-120b & gpt-oss-20b Model Card.” 2025. Hamilton, K. “Novoyaz 20B: Orthography Normalization for Pre-reform Russian on gpt-oss-20b.” Hugging Face, 2025. - Author / maintainer: @ZennyKenny - Issues & questions: Open an issue on the model page. - Contributions welcome: - Examples of failure cases / tricky spellings - Evaluation scripts and test sets for historical Russian - Suggestions for improved prompts or integration patterns If you build something with Novoyaz (OCR pipelines, research tools, teaching aids, etc.), I’d love to hear about it — it helps guide what to improve next.

This is a LoRA for the FLUX.1-dev text-to-image model. It can be used with diffusers or ComfyUI. It was trained on Replicate using AI toolkit: https://replicate.com/ostris/flux-dev-lora-trainer/train You should use `XTON` to trigger the image generation. For more details, including weighting, merging and fusing LoRAs, check the documentation on loading LoRAs in diffusers - Steps: 2000 - Learning rate: 0.0004 - LoRA rank: 32 You can use the community tab to add images that show off what you’ve made with this LoRA.