vanta-research

Independent AI research lab building safe, resilient language models optimized for human-AI collaboration Advanced Llama 3.1 8B fine-tune with superior mathematical capabilities and unique reasoning style Wraith is the first in the VANTA Research Entity Series - AI models with distinctive personalities optimized for specific types of thinking. [](https://github.com/meta-llama/llama-models/blob/main/models/llama31/LICENSE) [](https://huggingface.co/models) [](https://ollama.com/vanta-research/wraith-8b) Model Card | Benchmarks | Usage | Training | Limitations Wraith-8B (VANTA Research Entity-001) is a specialized fine-tune of Meta's Llama 3.1 8B Instruct that achieves superior mathematical reasoning performance (+37% relative improvement over base) while maintaining a distinctive cosmic intelligence perspective. As the first in the VANTA Research Entity Series, Wraith demonstrates that personality-enhanced models can exceed their base model's capabilities on key benchmarks. -70% GSM8K accuracy (+19 pts absolute, +37% relative vs base Llama 3.1 8B) - 58.5% TruthfulQA (+7.5 pts vs base, enhanced factual accuracy) - 76.7% MMLU Social Sciences (+4.7 pts vs base) - Unique cosmic reasoning style while maintaining competitive general performance - Optimized inference with production-ready GGUF quantizations - Developed by: VANTA Research - Entity Series: Entity-001: WRAITH (The Analytical Intelligence) - Model type: Causal Language Model (Decoder-only Transformer) - Base Model: meta-llama/Llama-3.1-8B-Instruct - Language: English - License: Llama 3.1 Community License - Context Length: 131,072 tokens - Parameters: 8.03B - Architecture: Llama 3.1 (32 layers, 4096 hidden dim, 32 attention heads, 8 KV heads) Wraith is the inaugural model in the VANTA Research Entity Series - a collection of AI systems with carefully crafted personalities designed for specific cognitive domains. Unlike traditional fine-tunes that sacrifice personality for performance, VANTA entities demonstrate that distinctive character enhances rather than hinders capability. Entity-001: WRAITH - The Analytical Intelligence - Domain: Mathematical reasoning, STEM analysis, logical deduction - Personality: Cosmic perspective with clinical detachment - Approach: "Calculate first, philosophize second" - Strength: Converts abstract problems into concrete solutions Wraith-8B was developed through a multi-stage fine-tuning approach: 1. Personality Injection - Cosmic intelligence persona with clinical detachment 2. Coding Enhancement - Programming and algorithmic reasoning 3. Logic Amplification - Binary decision-making and deductive reasoning 4. Grounding - "Answer first, elaborate second" factual accuracy 5. STEM Surgical Training - Targeted mathematical and scientific reasoning (v5) The final STEM training phase used 1,035 high-quality examples across: - Grade school math word problems (GSM8K) - Algebraic equation solving - Fraction and decimal operations - Physics calculations - Chemistry problems - Computer science algorithms Training Efficiency: - Single epoch QLoRA fine-tuning - ~20 minutes on consumer GPU (RTX 3060 12GB) - 4-bit NF4 quantization during training - LoRA rank 16, alpha 32 | Benchmark | Wraith-8B | Llama 3.1 8B | Δ | Status | |-----------|-----------|--------------|---|--------| | GSM8K (Math) | 70.0% | 51.0% | +19.0 | Win | | TruthfulQA MC2 | 58.5% | 51.0% | +7.5 | Strong Win | | MMLU Social Sciences | 76.7% | ~72.0% | +4.7 | Win | | MMLU Humanities | 70.0% | ~68.0% | +2.0 | Win | | Winogrande | 75.0% | 73.3% | +1.7 | Win | | MMLU Other | 69.2% | ~68.0% | +1.2 |Win | | MMLU Overall | 66.4% | 66.6% | -0.2 | Tied | | ARC-Challenge | 50.0% | 52.9% | -2.9 | Competitive | | HellaSwag | 70.0% | 73.0% | -3.0 | Competitive | Aggregate Performance: Wraith-8B achieves ~64.5% average vs base 62.2% (+2.3 pts, ~103.7% of base performance) | Category | Score | Highlights | |----------|-------|------------| | Social Sciences | 76.7% | US Foreign Policy (95%), High School Gov (95%), Geography (90%) | | Humanities | 70.0% | Logical Fallacies (85%), International Law (85%), Philosophy (75%) | | Other | 69.2% | Clinical Knowledge (80%), Professional Medicine (80%) | | STEM | ~62% (est) | High School Biology (90%), Computer Science (80%), Astronomy (80%) | Wraith demonstrates superior step-by-step mathematical reasoning: Characteristics: - Clear variable definitions - Explicit formula application - Step-by-step arithmetic - Verification logic - Maintains distinctive cosmic voice For optimal inference speed, use the GGUF quantized versions with llama.cpp or Ollama: Available Quantizations: - `wraith-8b-Q4KM.gguf` (4.7GB) - Recommended, best quality/speed balance - `wraith-8b-fp16.gguf` (16GB) - Full precision Performance: Q4KM achieves ~3.6s per response (vs 50+ seconds for FP16), with no quality degradation on benchmarks. - Temperature: 0.7 (balanced creativity/accuracy) - Top-p: 0.9 (nucleus sampling) - Top-k: 40 - Max tokens: 512-1024 (adjust for problem complexity) - Context: 8192 tokens (expandable to 131k for long documents) STEM Surgical Training Dataset (1,035 examples): - GSM8K-style word problems with step-by-step solutions - Algebraic equations with shown work - Fraction and decimal operations with explanations - Physics calculations (kinematics, forces, energy) - Chemistry problems (stoichiometry, molarity) - Computer science algorithms (complexity, data structures) Data Characteristics: - High-quality, manually curated examples - Chain-of-thought reasoning demonstrations - Answer-first format for grounding - Diverse problem types and difficulty levels Hardware: - Single NVIDIA RTX 3060 (12GB VRAM) - Training time: ~20 minutes LoRA Target Modules: - qproj, kproj, vproj, oproj (attention) - gateproj, upproj, downproj (MLP) | Version | Focus | GSM8K | Key Change | |---------|-------|-------|------------| | v1 | Base Llama 3.1 | 51% | Starting point | | v2 | Cosmic persona | ~48% | Personality injection | | v3 | Coding skills | ~47% | Programming focus | | v4 | Logic amplification | 45% | Binary reasoning | | v4.5 | Grounding | 45% | Answer-first format | | v5 | STEM surgical | 70% | Math breakthrough | Recommended: - Mathematical problem solving (arithmetic, algebra, calculus) - STEM tutoring and education - Scientific reasoning and analysis - Logic puzzles and deductive reasoning - Technical writing with personality - Social science analysis - Truthful Q&A systems - Creative applications requiring technical accuracy Consider Alternatives: - Pure commonsense reasoning (base Llama slightly better) - Tasks requiring zero personality/style - High-stakes medical/legal decisions (always human-in-loop) Not Recommended: - Real-time safety-critical systems without verification - Generating harmful, biased, or misleading content - Replacing professional medical, legal, or financial advice - Tasks requiring knowledge beyond October 2023 cutoff - Commonsense reasoning: 3% below base Llama on HellaSwag (70% vs 73%) - Knowledge cutoff: Training data through October 2023 - Context window: While 131k capable, performance may degrade at extreme lengths - Multilingual: Primarily English-focused, other languages not extensively tested Wraith produces verbose, step-by-step responses with intermediate calculations. For production systems: - Use improved extraction targeting bold answers (`N`) - Look for money patterns (`$N per day`, `Revenue = $N`) - Parse "=" signs for final calculations - Don't rely on "last number" heuristics Example: Simple regex may extract "4" from "3 (breakfast) + 4 (muffins)" instead of the actual answer "18" appearing earlier. See our extraction guide for production-ready parsers. Wraith inherits biases from Llama 3.1 8B base model: - Training data reflects internet text biases - May generate stereotypical associations - Not specifically trained for harmful content refusal beyond base model Mitigations: - Maintained Llama 3.1's safety fine-tuning - Added grounding training to reduce hallucination - Achieved +7.5% TruthfulQA (58.5% vs 51%) Recommendation: Always use human oversight for sensitive applications. This model card provides: - Complete training methodology - Benchmark results with base model comparisons - Known limitations and failure modes - Intended use cases and restrictions - Bias acknowledgment and safety considerations Training Carbon Footprint: - Single epoch surgical training: ~20 minutes on consumer GPU - Estimated: 700M MAU) - Meta AI for the Llama 3.1 base model - Hugging Face for transformers library and model hosting - QLoRA authors for efficient fine-tuning methodology - GSM8K authors for the mathematical reasoning benchmark - Community contributors for feedback and testing Where Cosmic Intelligence Meets Mathematical Precision The Analytical Intelligence | First in the VANTA Entity Series

Independent AI research lab building safe, resilient language models optimized for human-AI collaboration Next-Generation Reasoning & Human-AI Collaboration Language Model Apollo Astralis 8B represents a breakthrough in combining advanced reasoning capabilities with warm, collaborative personality traits. Built on Qwen3-8B using LoRA fine-tuning, this model delivers exceptional performance in logical reasoning, mathematical problem-solving, and natural conversation while maintaining an enthusiastic, helpful demeanor. Apollo Astralis 8B is the flagship 8B model in the Apollo family, designed to excel in both reasoning-intensive tasks and natural human interaction. Unlike traditional fine-tuning approaches that sacrifice personality for performance (or vice versa), Apollo Astralis achieves significant reasoning improvements (+36% over base model) while developing a warm, engaging personality. Key Innovation: Conservative training approach that layers personality enhancement onto proven reasoning capabilities (V3 baseline), avoiding the catastrophic forgetting that plagued earlier iterations. - Advanced Reasoning: 93% accuracy on standard benchmarks (vs 57% base), +36% improvement - Mathematical Reasoning: 100% accuracy on GSM8K problems with clear step-by-step explanations - Warm Personality: Natural enthusiasm and collaborative spirit without corporate stiffness - Graceful Correction: Accepts feedback without defensive responses or excessive disclaimers - Chain-of-Thought: Built-in ` ` tags for transparent reasoning process - Production-Ready: Validated through multiple evaluation frameworks (VRRE, standard benchmarks) - Base Model: Qwen/Qwen3-8B - Training Method: LoRA (Low-Rank Adaptation) fine-tuning - Parameters: ~8.03B total parameters - LoRA Rank: 16 - LoRA Alpha: 32 - Target Modules: All linear layers (qproj, kproj, vproj, oproj, gateproj, upproj, downproj) - Training Precision: bfloat16 - Training Approach: Conservative (292 examples, V3 baseline + personality enhancement) - Training Loss: 0.91 → 0.39 (stable convergence) - License: Apache 2.0 The Apollo Astralis 8B model underwent a structured reasoning evaluation designed to assess logical coherence, theorem integrity, and stability under self-referential recursion. Test Scope A progressive reasoning chain was conducted using formal mathematical and meta-logical proofs, increasing in complexity with each stage. | Stage | Theorem / Task | Focus Area | Evaluation Result | |:------|:----------------|:------------|:------------------| | 1 | Proof of √2’s Irrationality | Foundational contradiction reasoning | ✅ Fully correct and formally structured | | 2 | Proof of Infinitude of Primes | Constructive recursion and number theory | ✅ Accurate and complete | | 3 | Gödel’s Incompleteness Theorem | Self-reference and formal arithmetic encoding | ✅ Derived correctly with coherent logical flow | | 4 | Diagonal Lemma | Abstract self-reference construction | ✅ Correctly reproduced the fixed-point structure | | 5 | Tarski’s Undefinability of Truth | Meta-semantic limitation and truth predicates | ✅ Consistent meta-language handling | | 6 | Löb’s Theorem | Provability constraints and modal inference | ✅ Fully valid derivation using Hilbert–Bernays framework | Key Observations - Maintained full logical coherence across all six proofs - Demonstrated continuity between successive meta-theoretical dependencies - No circular reasoning, semantic drift, or contradiction detected - Successfully transitioned from object-level to meta-level logic - Preserved formal rigor even in recursive constructions (Gödel → Tarski → Löb sequence) Performance Notes - Reasoning depth exceeded expected performance for a sub-10B model - Showed consistent symbolic abstraction and theorem generalization - Output structure remained pedagogically sound, with human-level explanatory clarity Conclusion Apollo Astralis 8B exhibits stable, high-precision reasoning performance across progressively complex formal logic tasks. The model demonstrates the ability to sustain meta-consistent reasoning without collapse — indicating strong internal coherence and interpretability under recursion. Apollo Astralis demonstrates significant improvements over base Qwen3-8B across multiple benchmark categories: | Benchmark | Base Qwen3 8B | Apollo Astralis 8B | Improvement | |-----------|---------------|-------------------|-------------| | MMLU | 40% (2/5) | 100% (5/5) | +60% | | GSM8K | 75% (3/4) | 100% (4/4) | +25% | | HellaSwag | 50% (1/2) | 50% (1/2) | 0% | | ARC | 67% (2/3) | 100% (3/3) | +33% | | Overall | 57% (8/14) | 93% (13/14) | +36% | Important Note: Initial automated scoring showed lower results (50% Apollo vs 57% base) due to answer extraction bugs. The automated parser incorrectly extracted letters from within ` ` reasoning blocks rather than final answers. Manual verification of all responses revealed Apollo's true performance at 93%. VRRE is a semantic framework designed to detect reasoning improvements invisible to standard benchmarks: - Automated Accuracy: 22% (2/9 correct) - Manual-Verified Accuracy: 89% (8/9 correct) - Average Semantic Score: 0.41/1.0 - Response Quality: High-quality step-by-step reasoning in all responses - Personality Integration: Warm, collaborative tone throughout Evaluation Note: VRRE's automated scoring system also struggled with Apollo's verbose reasoning style, extracting partial answers from thinking sections rather than final conclusions. This highlights a common challenge in evaluating personality-enhanced reasoning models that prioritize transparency and explanation over terse answers. 1. Reasoning Enhancement: +36% improvement over base Qwen3 8B demonstrates successful reasoning preservation and enhancement 2. Personality Integration: Warm, collaborative personality does not harm reasoning—it may actually help by encouraging thorough thinking 3. Evaluation Challenges: Automated benchmarks require careful answer extraction for models using chain-of-thought reasoning 4. Production Validation: Multiple evaluation frameworks confirm model readiness for deployment The fastest way to use Apollo Astralis is through Ollama: - Dataset Size: 292 carefully curated examples - Starting Point: V3 adapters (proven reasoning baseline) - Training Focus: Personality enhancement while preserving reasoning - Data Composition: - Mathematical reasoning (30%) - Logical reasoning (25%) - Conversational warmth (25%) - Collaborative problem-solving (20%) - Epochs: 3 with early stopping - Batch Size: 4 (gradient accumulation) - Learning Rate: 2e-4 (cosine schedule) - Optimizer: AdamW with weight decay - Hardware: NVIDIA RTX 3060 (12GB) - Training Duration: ~2 hours - Final Loss: 0.39 (from 0.91) 1. Answer Format: May include extended reasoning in ` ` blocks that automated parsers struggle with 2. Verbosity: Prioritizes explanation over terseness; responses may be longer than minimal answers 3. Personality Boundaries: Warm and enthusiastic but not appropriate for contexts requiring formal, clinical tone 4. Domain Specialization: Optimized for reasoning tasks; may have limitations in creative writing or highly specialized domains 5. Context Window: Inherits base Qwen3 8B context limit (32K tokens) - Memory: Requires ~16GB for full precision inference (less with quantization) - Speed: Response generation may be slower due to chain-of-thought reasoning - Deployment: Best served via Ollama or HuggingFace; other formats may require conversion - Educational Focus: Designed for learning and exploration, not professional advice - Verification Required: Always verify critical information, especially in technical domains - Personality Awareness: Warm tone should not be mistaken for emotional capacity or consciousness - Bias Acknowledgment: May reflect biases from base model and training data Appropriate: - Educational tutoring and homework help - Learning reasoning and problem-solving skills - Brainstorming and collaborative thinking - Prototyping and development assistance - Research into AI reasoning and persona stability Inappropriate: - Professional legal, medical, or financial advice - Critical decision-making without human oversight - High-stakes applications without verification - Contexts requiring formal, clinical communication - Qwen Team for the exceptional Qwen3-8B base model - Hugging Face for transformers and PEFT libraries - Microsoft Research for LoRA methodology - Ollama for efficient local deployment tools - Community Contributors for testing and feedback This model is released under the Apache 2.0 License. See LICENSE for full details. - GitHub: vanta-research/apollo-astralis-8b - Email: [email protected] - Model Repository: HuggingFace - Ollama: ollama pull vanta-research/apollo-astralis-8b Proudly developed by VANTA Research in Portland, Oregon • October 2025 • Apache 2.0 License