osmosis-ai

`Osmosis-Structure-0.6B`: Small Language Model for Structured Outputs `Osmosis-Structure-0.6B` is a specialized small language model (SLM) designed to excel at structured output generation. Despite its compact 0.6B parameter size, this model demonstrates remarkable performance on extracting structured information when paired with supported frameworks. Our approach leverages structured output during training, forcing our model to only focus on the value for each key declared by the inference engine, which significantly improves the accuracy of the model's ability to produce well-formatted, structured responses across various domains, particularly in mathematical reasoning and problem-solving tasks. We evaluate the effectiveness of osmosis-enhanced structured generation on challenging mathematical reasoning benchmarks. The following results demonstrate the dramatic performance improvements achieved through structured outputs with osmosis enhancement across different model families - the same technique that powers `Osmosis-Structure-0.6B`. | Model | Structured Output | Structured w/ Osmosis | Performance Gain | |-------|:-------------:|:-------------:|:-------------:| | Claude 4 Sonnet | 15.52% | 69.40% | +347% | | Claude 4 Opus | 15.28% | 69.91% | +357% | | GPT-4.1 | 10.53% | 70.03% | +565% | | OpenAI o3 | 91.14% | 94.05% | +2.9% | | Model | Structured Output | Structured w/ Osmosis | Performance Gain | |-------|:-------------:|:-------------:|:-------------:| | Claude 4 Sonnet | 16.29% | 62.59% | +284% | | Claude 4 Opus | 22.94% | 65.06% | +184% | | GPT-4.1 | 2.79% | 39.66% | +1322% | | OpenAI o3 | 92.05% | 93.24% | +1.3% | > Key Insight: These results demonstrate that by allowing models to think freely and leverage test time compute, we are able to increase performance and still maintain the structured guarantee after the fact with a SLM. `Osmosis-Structure-0.6B` is specifically designed and optimized to maximize these benefits in a compact 0.6B parameter model. `Osmosis-Structure-0.6B` is built on top of `Qwen3-0.6B`. We first established a baseline format using 10 samples of randomly generated text and their JSON interpretations. We then applied reinforcement learning to approximately 500,000 examples of JSON-to-natural language pairs, consisting of either reasoning traces with their final outputs, or natural language reports with their expected structured formats. We used verl as the framework to train our model and SGLang as the rollout backend. To enable structured training, we modified parts of the verl codebase to allow for per sample schema to be passed into the training data. We recommend an engine like SGLang to be used to serve the model, to serve, run the following: `python3 -m sglang.launchserver --model-path osmosis-ai/Osmosis-Structure-0.6B --host 0.0.0.0 --api-key osmosis` You can also use Ollama as an inference provider on local machines, here is a sample code of the setup: