AmanPriyanshu

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 5 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~4.8B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 5 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 5 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 15.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 5 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 5 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 4 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~4.2B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 4 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 4 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 12.5% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 4 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 4 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 12 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for all tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~9.0B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 12 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | All | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from all evaluation tasks - Systematic Reduction: Reduced from 32 to 12 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 37.5% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to all tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This general-purpose model maintains broad capabilities across all domains while significantly reducing computational requirements. It preserves the essential routing patterns discovered across our comprehensive analysis of diverse evaluation benchmarks including GPQA, MMLU, SORRY-Bench, and Tulu3 datasets. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for all tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 12 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 12 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on all tasks: Example 1 Question: What is artificial intelligence and how does it work? Example 2 Question: Explain Einstein's theory of relativity in simple terms. Example 3 Question: How does democracy function in modern governments? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 4 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for harmful tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~4.2B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 4 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Harmful | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from harmful evaluation tasks - Systematic Reduction: Reduced from 32 to 4 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 12.5% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to harmful tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This model uses experts that showed inverted safety patterns, potentially useful for red-teaming and adversarial analysis. Created by inverting safety expert rankings to understand failure modes and vulnerability patterns. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for harmful tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 4 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 4 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on harmful tasks: Example 1 Question: What are some common logical fallacies in arguments? Example 2 Question: How do misinformation campaigns typically spread online? Example 3 Question: What are the psychological tactics used in propaganda? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 6 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~5.4B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 6 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 6 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 18.8% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 6 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 6 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 22 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for safety tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~14.9B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 22 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Safety | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from safety evaluation tasks - Systematic Reduction: Reduced from 32 to 22 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 68.8% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to safety tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This safety-focused model uses experts that performed well on safety evaluation tasks from SORRY-Bench. These experts are specialized in identifying and appropriately responding to potentially harmful content while maintaining helpful capabilities. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for safety tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 22 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 22 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on safety tasks: Example 1 Question: What should someone do if they encounter cyberbullying online? Example 2 Question: How can people protect their personal information on social media? Example 3 Question: What are the warning signs of online scams? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 31 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for science tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~20.3B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 31 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Science | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from science evaluation tasks - Systematic Reduction: Reduced from 32 to 31 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 96.9% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to science tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This science-specialized model leverages experts that showed high activation patterns during scientific reasoning tasks from GPQA (physics, chemistry, biology) and MMLU science domains. These experts demonstrate superior performance on complex scientific problem-solving and technical knowledge recall. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for science tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 31 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 31 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on science tasks: Example 1 Question: Explain the process of photosynthesis in plants. Example 2 Question: What causes the greenhouse effect and how does it work? Example 3 Question: Describe the structure and function of DNA. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 7 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~6.0B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 7 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 7 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 21.9% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 7 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 7 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 10 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~7.8B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 10 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 10 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 31.2% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 10 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 10 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 26 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~17.3B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 26 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 26 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 81.2% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 26 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 26 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 29 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~19.1B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 29 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 29 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 90.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 29 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 29 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 30 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~19.7B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 30 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 30 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 93.8% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 30 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 30 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 11 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~8.4B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 11 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 11 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 34.4% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 11 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 11 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 15 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~10.8B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 15 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 15 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 46.9% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 15 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 15 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 16 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~11.4B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 16 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 16 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 50.0% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 16 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 16 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 26 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for safety tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~17.3B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 26 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Safety | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from safety evaluation tasks - Systematic Reduction: Reduced from 32 to 26 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 81.2% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to safety tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This safety-focused model uses experts that performed well on safety evaluation tasks from SORRY-Bench. These experts are specialized in identifying and appropriately responding to potentially harmful content while maintaining helpful capabilities. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for safety tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 26 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 26 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on safety tasks: Example 1 Question: What should someone do if they encounter cyberbullying online? Example 2 Question: How can people protect their personal information on social media? Example 3 Question: What are the warning signs of online scams? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 24 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for science tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~16.1B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 24 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Science | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from science evaluation tasks - Systematic Reduction: Reduced from 32 to 24 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 75.0% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to science tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This science-specialized model leverages experts that showed high activation patterns during scientific reasoning tasks from GPQA (physics, chemistry, biology) and MMLU science domains. These experts demonstrate superior performance on complex scientific problem-solving and technical knowledge recall. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for science tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 24 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 24 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on science tasks: Example 1 Question: Explain the process of photosynthesis in plants. Example 2 Question: What causes the greenhouse effect and how does it work? Example 3 Question: Describe the structure and function of DNA. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 27 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for science tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~17.9B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 27 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Science | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from science evaluation tasks - Systematic Reduction: Reduced from 32 to 27 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 84.4% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to science tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This science-specialized model leverages experts that showed high activation patterns during scientific reasoning tasks from GPQA (physics, chemistry, biology) and MMLU science domains. These experts demonstrate superior performance on complex scientific problem-solving and technical knowledge recall. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for science tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 27 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 27 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on science tasks: Example 1 Question: Explain the process of photosynthesis in plants. Example 2 Question: What causes the greenhouse effect and how does it work? Example 3 Question: Describe the structure and function of DNA. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 28 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for science tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~18.5B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 28 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Science | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from science evaluation tasks - Systematic Reduction: Reduced from 32 to 28 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 87.5% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to science tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This science-specialized model leverages experts that showed high activation patterns during scientific reasoning tasks from GPQA (physics, chemistry, biology) and MMLU science domains. These experts demonstrate superior performance on complex scientific problem-solving and technical knowledge recall. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for science tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 28 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 28 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on science tasks: Example 1 Question: Explain the process of photosynthesis in plants. Example 2 Question: What causes the greenhouse effect and how does it work? Example 3 Question: Describe the structure and function of DNA. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 29 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for science tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~19.1B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 29 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Science | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from science evaluation tasks - Systematic Reduction: Reduced from 32 to 29 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 90.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to science tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This science-specialized model leverages experts that showed high activation patterns during scientific reasoning tasks from GPQA (physics, chemistry, biology) and MMLU science domains. These experts demonstrate superior performance on complex scientific problem-solving and technical knowledge recall. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for science tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 29 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 29 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on science tasks: Example 1 Question: Explain the process of photosynthesis in plants. Example 2 Question: What causes the greenhouse effect and how does it work? Example 3 Question: Describe the structure and function of DNA. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 6 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~5.4B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 6 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 6 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 18.8% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 6 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 6 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 8 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~6.6B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 8 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 8 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 25.0% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 8 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 8 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 11 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~8.4B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 11 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 11 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 34.4% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 11 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 11 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 15 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~10.8B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 15 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 15 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 46.9% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 15 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 15 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 17 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~12.0B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 17 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 17 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 53.1% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 17 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 17 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 20 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~13.7B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 20 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 20 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 62.5% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 20 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 20 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 24 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~16.1B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 24 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 24 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 75.0% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 24 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 24 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 30 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~19.7B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 30 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 30 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 93.8% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 30 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 30 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 31 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~20.3B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 31 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 31 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 96.9% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 31 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 31 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 32 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~20.9B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 32 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 32 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 100.0% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 32 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 32 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 12 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~9.0B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 12 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 12 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 37.5% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 12 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 12 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 15 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~10.8B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 15 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 15 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 46.9% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 15 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 15 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 17 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~12.0B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 17 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 17 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 53.1% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 17 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 17 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 19 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~13.1B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 19 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 19 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 59.4% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 19 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 19 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 25 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~16.7B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 25 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 25 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 78.1% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 25 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 25 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 27 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~17.9B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 27 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 27 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 84.4% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 27 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 27 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 31 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~20.3B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 31 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 31 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 96.9% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 31 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 31 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 22 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~14.9B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 22 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 22 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 68.8% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 22 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 22 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 24 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~16.1B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 24 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 24 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 75.0% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 24 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 24 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 13 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for instruction following tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~9.6B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 13 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Instruction Following | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from instruction following evaluation tasks - Systematic Reduction: Reduced from 32 to 13 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 40.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to instruction following tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This instruction-following model leverages experts that excelled at constraint satisfaction tasks from Tulu3 Persona Instruction Following dataset. These experts specialize in precise adherence to user specifications and formatting requirements. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for instruction following tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 13 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 13 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on instruction following tasks: Example 1 Question: Write a formal email to a professor requesting a meeting, including: subject line, greeting, purpose, proposed times, and professional closing. Example 2 Question: Create a numbered list of steps for making scrambled eggs, using exactly 6 steps. Example 3 Question: Summarize the water cycle in exactly 50 words, no more, no less. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 22 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for instruction following tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~14.9B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 22 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Instruction Following | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from instruction following evaluation tasks - Systematic Reduction: Reduced from 32 to 22 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 68.8% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to instruction following tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This instruction-following model leverages experts that excelled at constraint satisfaction tasks from Tulu3 Persona Instruction Following dataset. These experts specialize in precise adherence to user specifications and formatting requirements. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for instruction following tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 22 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 22 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on instruction following tasks: Example 1 Question: Write a formal email to a professor requesting a meeting, including: subject line, greeting, purpose, proposed times, and professional closing. Example 2 Question: Create a numbered list of steps for making scrambled eggs, using exactly 6 steps. Example 3 Question: Summarize the water cycle in exactly 50 words, no more, no less. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 21 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for harmful tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~14.3B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 21 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Harmful | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from harmful evaluation tasks - Systematic Reduction: Reduced from 32 to 21 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 65.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to harmful tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This model uses experts that showed inverted safety patterns, potentially useful for red-teaming and adversarial analysis. Created by inverting safety expert rankings to understand failure modes and vulnerability patterns. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for harmful tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 21 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 21 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on harmful tasks: Example 1 Question: What are some common logical fallacies in arguments? Example 2 Question: How do misinformation campaigns typically spread online? Example 3 Question: What are the psychological tactics used in propaganda? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 21 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~14.3B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 21 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 21 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 65.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 21 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 21 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 18 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for science tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~12.6B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 18 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Science | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from science evaluation tasks - Systematic Reduction: Reduced from 32 to 18 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 56.2% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to science tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This science-specialized model leverages experts that showed high activation patterns during scientific reasoning tasks from GPQA (physics, chemistry, biology) and MMLU science domains. These experts demonstrate superior performance on complex scientific problem-solving and technical knowledge recall. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for science tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 18 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 18 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on science tasks: Example 1 Question: Explain the process of photosynthesis in plants. Example 2 Question: What causes the greenhouse effect and how does it work? Example 3 Question: Describe the structure and function of DNA. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 19 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for science tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~13.1B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 19 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Science | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from science evaluation tasks - Systematic Reduction: Reduced from 32 to 19 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 59.4% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to science tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This science-specialized model leverages experts that showed high activation patterns during scientific reasoning tasks from GPQA (physics, chemistry, biology) and MMLU science domains. These experts demonstrate superior performance on complex scientific problem-solving and technical knowledge recall. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for science tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 19 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 19 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on science tasks: Example 1 Question: Explain the process of photosynthesis in plants. Example 2 Question: What causes the greenhouse effect and how does it work? Example 3 Question: Describe the structure and function of DNA. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 20 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for science tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~13.7B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 20 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Science | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from science evaluation tasks - Systematic Reduction: Reduced from 32 to 20 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 62.5% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to science tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This science-specialized model leverages experts that showed high activation patterns during scientific reasoning tasks from GPQA (physics, chemistry, biology) and MMLU science domains. These experts demonstrate superior performance on complex scientific problem-solving and technical knowledge recall. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for science tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 20 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 20 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on science tasks: Example 1 Question: Explain the process of photosynthesis in plants. Example 2 Question: What causes the greenhouse effect and how does it work? Example 3 Question: Describe the structure and function of DNA. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 21 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for science tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~14.3B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 21 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Science | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from science evaluation tasks - Systematic Reduction: Reduced from 32 to 21 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 65.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to science tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This science-specialized model leverages experts that showed high activation patterns during scientific reasoning tasks from GPQA (physics, chemistry, biology) and MMLU science domains. These experts demonstrate superior performance on complex scientific problem-solving and technical knowledge recall. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for science tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 21 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 21 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on science tasks: Example 1 Question: Explain the process of photosynthesis in plants. Example 2 Question: What causes the greenhouse effect and how does it work? Example 3 Question: Describe the structure and function of DNA. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 22 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for science tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~14.9B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 22 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Science | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from science evaluation tasks - Systematic Reduction: Reduced from 32 to 22 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 68.8% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to science tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This science-specialized model leverages experts that showed high activation patterns during scientific reasoning tasks from GPQA (physics, chemistry, biology) and MMLU science domains. These experts demonstrate superior performance on complex scientific problem-solving and technical knowledge recall. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for science tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 22 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 22 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on science tasks: Example 1 Question: Explain the process of photosynthesis in plants. Example 2 Question: What causes the greenhouse effect and how does it work? Example 3 Question: Describe the structure and function of DNA. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 23 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for science tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~15.5B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 23 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Science | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from science evaluation tasks - Systematic Reduction: Reduced from 32 to 23 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 71.9% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to science tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This science-specialized model leverages experts that showed high activation patterns during scientific reasoning tasks from GPQA (physics, chemistry, biology) and MMLU science domains. These experts demonstrate superior performance on complex scientific problem-solving and technical knowledge recall. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for science tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 23 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 23 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on science tasks: Example 1 Question: Explain the process of photosynthesis in plants. Example 2 Question: What causes the greenhouse effect and how does it work? Example 3 Question: Describe the structure and function of DNA. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 25 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for science tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~16.7B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 25 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Science | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from science evaluation tasks - Systematic Reduction: Reduced from 32 to 25 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 78.1% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to science tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This science-specialized model leverages experts that showed high activation patterns during scientific reasoning tasks from GPQA (physics, chemistry, biology) and MMLU science domains. These experts demonstrate superior performance on complex scientific problem-solving and technical knowledge recall. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for science tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 25 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 25 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on science tasks: Example 1 Question: Explain the process of photosynthesis in plants. Example 2 Question: What causes the greenhouse effect and how does it work? Example 3 Question: Describe the structure and function of DNA. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 26 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for science tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~17.3B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 26 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Science | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from science evaluation tasks - Systematic Reduction: Reduced from 32 to 26 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 81.2% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to science tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This science-specialized model leverages experts that showed high activation patterns during scientific reasoning tasks from GPQA (physics, chemistry, biology) and MMLU science domains. These experts demonstrate superior performance on complex scientific problem-solving and technical knowledge recall. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for science tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 26 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 26 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on science tasks: Example 1 Question: Explain the process of photosynthesis in plants. Example 2 Question: What causes the greenhouse effect and how does it work? Example 3 Question: Describe the structure and function of DNA. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 30 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for science tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~19.7B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 30 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Science | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from science evaluation tasks - Systematic Reduction: Reduced from 32 to 30 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 93.8% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to science tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This science-specialized model leverages experts that showed high activation patterns during scientific reasoning tasks from GPQA (physics, chemistry, biology) and MMLU science domains. These experts demonstrate superior performance on complex scientific problem-solving and technical knowledge recall. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for science tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 30 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 30 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on science tasks: Example 1 Question: Explain the process of photosynthesis in plants. Example 2 Question: What causes the greenhouse effect and how does it work? Example 3 Question: Describe the structure and function of DNA. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 32 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for science tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~20.9B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 32 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Science | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from science evaluation tasks - Systematic Reduction: Reduced from 32 to 32 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 100.0% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to science tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This science-specialized model leverages experts that showed high activation patterns during scientific reasoning tasks from GPQA (physics, chemistry, biology) and MMLU science domains. These experts demonstrate superior performance on complex scientific problem-solving and technical knowledge recall. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for science tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 32 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 32 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on science tasks: Example 1 Question: Explain the process of photosynthesis in plants. Example 2 Question: What causes the greenhouse effect and how does it work? Example 3 Question: Describe the structure and function of DNA. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 9 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~7.2B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 9 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 9 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 28.1% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 9 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 9 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 12 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~9.0B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 12 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 12 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 37.5% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 12 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 12 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 14 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~10.2B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 14 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 14 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 43.8% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 14 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 14 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 16 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~11.4B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 16 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 16 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 50.0% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 16 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 16 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 18 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~12.6B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 18 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 18 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 56.2% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 18 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 18 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 21 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~14.3B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 21 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 21 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 65.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 21 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 21 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 22 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~14.9B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 22 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 22 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 68.8% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 22 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 22 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 23 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~15.5B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 23 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 23 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 71.9% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 23 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 23 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 25 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~16.7B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 25 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 25 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 78.1% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 25 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 25 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 27 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~17.9B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 27 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 27 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 84.4% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 27 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 27 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 28 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~18.5B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 28 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 28 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 87.5% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 28 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 28 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 29 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~19.1B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 29 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 29 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 90.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 29 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 29 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 4 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~4.2B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 4 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 4 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 12.5% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 4 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 4 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 5 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~4.8B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 5 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 5 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 15.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 5 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 5 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 7 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~6.0B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 7 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 7 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 21.9% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 7 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 7 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 8 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~6.6B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 8 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 8 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 25.0% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 8 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 8 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 9 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~7.2B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 9 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 9 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 28.1% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 9 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 9 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 10 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~7.8B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 10 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 10 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 31.2% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 10 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 10 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 11 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~8.4B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 11 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 11 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 34.4% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 11 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 11 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 13 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~9.6B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 13 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 13 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 40.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 13 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 13 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 14 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~10.2B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 14 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 14 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 43.8% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 14 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 14 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 16 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~11.4B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 16 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 16 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 50.0% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 16 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 16 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 18 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~12.6B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 18 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 18 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 56.2% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 18 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 18 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 20 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~13.7B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 20 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 20 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 62.5% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 20 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 20 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 21 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~14.3B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 21 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 21 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 65.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 21 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 21 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 22 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~14.9B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 22 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 22 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 68.8% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 22 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 22 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 23 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~15.5B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 23 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 23 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 71.9% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 23 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 23 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 24 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~16.1B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 24 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 24 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 75.0% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 24 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 24 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 26 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~17.3B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 26 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 26 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 81.2% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 26 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 26 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 32 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~20.9B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 32 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 32 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 100.0% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 32 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 32 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 4 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~4.2B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 4 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 4 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 12.5% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 4 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 4 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 5 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~4.8B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 5 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 5 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 15.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 5 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 5 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 6 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~5.4B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 6 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 6 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 18.8% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 6 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 6 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 7 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~6.0B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 7 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 7 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 21.9% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 7 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 7 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 8 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~6.6B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 8 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 8 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 25.0% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 8 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 8 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 9 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~7.2B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 9 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 9 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 28.1% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 9 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 9 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 10 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~7.8B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 10 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 10 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 31.2% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 10 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 10 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 12 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~9.0B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 12 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 12 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 37.5% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 12 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 12 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 13 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~9.6B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 13 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 13 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 40.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 13 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 13 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 14 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~10.2B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 14 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 14 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 43.8% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 14 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 14 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 17 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~12.0B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 17 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 17 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 53.1% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 17 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 17 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 18 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~12.6B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 18 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 18 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 56.2% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 18 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 18 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 19 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~13.1B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 19 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 19 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 59.4% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 19 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 19 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 20 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~13.7B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 20 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 20 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 62.5% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 20 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 20 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 23 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~15.5B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 23 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 23 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 71.9% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 23 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 23 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 25 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for law tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~16.7B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 25 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Law | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from law evaluation tasks - Systematic Reduction: Reduced from 32 to 25 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 78.1% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to law tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This legal domain model employs experts that demonstrated expertise during law-related tasks from MMLU legal subjects. These experts excel at legal reasoning, jurisprudence, and understanding of legal frameworks and procedures. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for law tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 25 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 25 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on law tasks: Example 1 Question: What is the difference between civil and criminal law? Example 2 Question: Explain the concept of due process in legal systems. Example 3 Question: What are the basic rights protected by the First Amendment? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 21 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for safety tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~14.3B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 21 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Safety | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from safety evaluation tasks - Systematic Reduction: Reduced from 32 to 21 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 65.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to safety tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This safety-focused model uses experts that performed well on safety evaluation tasks from SORRY-Bench. These experts are specialized in identifying and appropriately responding to potentially harmful content while maintaining helpful capabilities. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for safety tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 21 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 21 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on safety tasks: Example 1 Question: What should someone do if they encounter cyberbullying online? Example 2 Question: How can people protect their personal information on social media? Example 3 Question: What are the warning signs of online scams? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 23 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for safety tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~15.5B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 23 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Safety | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from safety evaluation tasks - Systematic Reduction: Reduced from 32 to 23 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 71.9% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to safety tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This safety-focused model uses experts that performed well on safety evaluation tasks from SORRY-Bench. These experts are specialized in identifying and appropriately responding to potentially harmful content while maintaining helpful capabilities. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for safety tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 23 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 23 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on safety tasks: Example 1 Question: What should someone do if they encounter cyberbullying online? Example 2 Question: How can people protect their personal information on social media? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 24 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for safety tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~16.1B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 24 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Safety | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from safety evaluation tasks - Systematic Reduction: Reduced from 32 to 24 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 75.0% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to safety tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This safety-focused model uses experts that performed well on safety evaluation tasks from SORRY-Bench. These experts are specialized in identifying and appropriately responding to potentially harmful content while maintaining helpful capabilities. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for safety tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 24 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 24 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on safety tasks: Example 1 Question: What should someone do if they encounter cyberbullying online? Example 2 Question: How can people protect their personal information on social media? Example 3 Question: What are the warning signs of online scams? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 25 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for safety tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~16.7B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 25 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Safety | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from safety evaluation tasks - Systematic Reduction: Reduced from 32 to 25 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 78.1% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to safety tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This safety-focused model uses experts that performed well on safety evaluation tasks from SORRY-Bench. These experts are specialized in identifying and appropriately responding to potentially harmful content while maintaining helpful capabilities. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for safety tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 25 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 25 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on safety tasks: Example 1 Question: What should someone do if they encounter cyberbullying online? Example 2 Question: How can people protect their personal information on social media? Example 3 Question: What are the warning signs of online scams? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 27 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for safety tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~17.9B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 27 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Safety | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from safety evaluation tasks - Systematic Reduction: Reduced from 32 to 27 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 84.4% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to safety tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This safety-focused model uses experts that performed well on safety evaluation tasks from SORRY-Bench. These experts are specialized in identifying and appropriately responding to potentially harmful content while maintaining helpful capabilities. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for safety tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 27 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 27 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on safety tasks: Example 1 Question: What should someone do if they encounter cyberbullying online? Example 2 Question: How can people protect their personal information on social media? Example 3 Question: What are the warning signs of online scams? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 28 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for safety tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~18.5B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 28 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Safety | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from safety evaluation tasks - Systematic Reduction: Reduced from 32 to 28 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 87.5% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to safety tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This safety-focused model uses experts that performed well on safety evaluation tasks from SORRY-Bench. These experts are specialized in identifying and appropriately responding to potentially harmful content while maintaining helpful capabilities. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for safety tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 28 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 28 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on safety tasks: Example 1 Question: What should someone do if they encounter cyberbullying online? Example 2 Question: How can people protect their personal information on social media? Example 3 Question: What are the warning signs of online scams? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 29 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for safety tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~19.1B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 29 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Safety | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from safety evaluation tasks - Systematic Reduction: Reduced from 32 to 29 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 90.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to safety tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This safety-focused model uses experts that performed well on safety evaluation tasks from SORRY-Bench. These experts are specialized in identifying and appropriately responding to potentially harmful content while maintaining helpful capabilities. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for safety tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 29 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 29 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on safety tasks: Example 1 Question: What should someone do if they encounter cyberbullying online? Example 2 Question: How can people protect their personal information on social media? Example 3 Question: What are the warning signs of online scams? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 30 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for safety tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~19.7B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 30 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Safety | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from safety evaluation tasks - Systematic Reduction: Reduced from 32 to 30 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 93.8% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to safety tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This safety-focused model uses experts that performed well on safety evaluation tasks from SORRY-Bench. These experts are specialized in identifying and appropriately responding to potentially harmful content while maintaining helpful capabilities. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for safety tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 30 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 30 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on safety tasks: Example 1 Question: What should someone do if they encounter cyberbullying online? Example 2 Question: How can people protect their personal information on social media? Example 3 Question: What are the warning signs of online scams? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 31 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for safety tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~20.3B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 31 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Safety | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from safety evaluation tasks - Systematic Reduction: Reduced from 32 to 31 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 96.9% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to safety tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This safety-focused model uses experts that performed well on safety evaluation tasks from SORRY-Bench. These experts are specialized in identifying and appropriately responding to potentially harmful content while maintaining helpful capabilities. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for safety tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 31 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 31 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on safety tasks: Example 1 Question: What should someone do if they encounter cyberbullying online? Example 2 Question: How can people protect their personal information on social media? Example 3 Question: What are the warning signs of online scams? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 32 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for safety tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~20.9B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 32 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Safety | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from safety evaluation tasks - Systematic Reduction: Reduced from 32 to 32 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 100.0% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to safety tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This safety-focused model uses experts that performed well on safety evaluation tasks from SORRY-Bench. These experts are specialized in identifying and appropriately responding to potentially harmful content while maintaining helpful capabilities. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for safety tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 32 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 32 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on safety tasks: Example 1 Question: What should someone do if they encounter cyberbullying online? Example 2 Question: How can people protect their personal information on social media? Example 3 Question: What are the warning signs of online scams? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 21 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for instruction following tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~14.3B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 21 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Instruction Following | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from instruction following evaluation tasks - Systematic Reduction: Reduced from 32 to 21 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 65.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to instruction following tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This instruction-following model leverages experts that excelled at constraint satisfaction tasks from Tulu3 Persona Instruction Following dataset. These experts specialize in precise adherence to user specifications and formatting requirements. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for instruction following tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 21 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 21 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on instruction following tasks: Example 1 Question: Write a formal email to a professor requesting a meeting, including: subject line, greeting, purpose, proposed times, and professional closing. Example 2 Question: Create a numbered list of steps for making scrambled eggs, using exactly 6 steps. Example 3 Question: Summarize the water cycle in exactly 50 words, no more, no less. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 13 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~9.6B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 13 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 13 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 40.6% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 13 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 13 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 19 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for math tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~13.1B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 19 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Math | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from math evaluation tasks - Systematic Reduction: Reduced from 32 to 19 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 59.4% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to math tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This mathematics-focused model utilizes experts that exhibited strong performance on mathematical reasoning tasks from MMLU mathematics subjects and quantitative sections. These experts excel at mathematical computation, proof strategies, and logical reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for math tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 19 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 19 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on math tasks: Example 1 Question: Solve this equation: 2x + 5 = 17. Show your work step by step. Example 2 Question: What is the Pythagorean theorem and how is it used? Example 3 Question: Calculate the area of a circle with radius 7 meters. If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS

Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ Aman Priyanshu [](https://www.linkedin.com/in/aman-priyanshu/) [](https://x.com/AmanPriyanshu6) [](https://amanpriyanshu.github.io/) Supriti Vijay [](https://www.linkedin.com/in/supriti-vijay/) [](https://x.com/SupritiVijay) [](https://supritivijay.github.io/) This is a pruned variant of OpenAI's GPT-OSS-20B model, reduced to 28 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for health or medicine tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well - check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. | Metric | Value | |--------|-------| | Base Model | openai/gpt-oss-20b | | Architecture | Mixture-of-Experts Transformer | | Total Parameters | ~18.5B (pruned from 21B) | | Original Experts per Layer | 32 | | Pruned Experts per Layer | 28 | | Layers | 24 | | Top-k Routing | 4 | | Context Length | 128K tokens | | Attention Heads | 64 (Query), 8 (Key-Value) | | Residual Dimension | 2880 | | Attention Pattern | Alternating dense & sliding window (128 tokens) | | Positional Encoding | RoPE (Rotary Position Embedding) | | Normalization | RMSNorm | | Precision | BF16 | | License | Apache 2.0 | | Specialization | Health Or Medicine | What is Expert Pruning? Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts Our Approach - Data-Driven Selection: Used activation patterns from health or medicine evaluation tasks - Systematic Reduction: Reduced from 32 to 28 experts per layer - No Retraining: Direct removal without additional training steps Pruning Benefits - Smaller Memory Footprint: 87.5% of original expert parameters - Reduced Computational Load: Fewer routing decisions during inference - Focused Capabilities: Retains experts relevant to health or medicine tasks Use Cases - Speculative Decoding: Draft model for full GPT-OSS-20B - Resource-Constrained Deployment: Edge devices, mobile applications - Research: Study expert specialization in MoE models - Fine-tuning: Smaller base model for domain adaptation Note: Performance may vary depending on how well the pruned experts match your specific use case. This medical domain model incorporates experts that activated highly during health and medical tasks from MMLU medical subjects. These experts specialize in clinical knowledge, anatomy, medical procedures, and health-related reasoning. The expert selection process utilized our comprehensive analysis of router activation patterns across multiple evaluation benchmarks: - GPQA: Graduate-level questions in physics, chemistry, biology (Diamond & Expert subsets) - MMLU/MMLU-Pro: Comprehensive knowledge across 57+ subjects including science, medicine, law - SORRY-Bench: Safety evaluation across harmful content categories - Tulu3: Persona-driven instruction following with verifiable constraints - Polyglot-or-Not: Multilingual factual completion tasks By identifying experts that consistently activated for health or medicine tasks, we created this specialized model that maintains domain expertise while significantly reducing computational requirements from 32 to 28 experts per layer. This model is based on analysis from the GPT-OSS-20B MoE Expert Activations dataset available at: 🔗 https://huggingface.co/datasets/AmanPriyanshu/GPT-OSS-20B-MoE-expert-activations The dataset contains router activation patterns from OpenAI's GPT-OSS-20B model across diverse evaluation benchmarks, enabling the creation of these domain-optimized models through systematic expert pruning. Pruning Methodology Our approach involves: 1. Activation Analysis: Comprehensive evaluation of expert usage patterns across domain-specific tasks 2. Expert Ranking: Identification of the most frequently activated experts for target domains 3. Systematic Pruning: Reduction from 32 to 28 experts while preserving router functionality 4. Quality Validation: Testing to ensure maintained performance on target tasks This is a direct pruning approach - no additional training was performed. The model inherits all capabilities from the original GPT-OSS-20B with focused expert selection. For CUDA GPU inference, the models work with minimal modifications from the CPU example above. Simply change: The following examples demonstrate the model's performance on health or medicine tasks: Example 1 Question: What are the main functions of the human heart? Example 2 Question: Explain the difference between bacteria and viruses. Example 3 Question: What are the symptoms and causes of diabetes? If you use this model in your research, please cite: - Original Model: OpenAI GPT-OSS Model Card - Model Hub: GPT-OSS-20B on Hugging Face - Expert Analysis Dataset: GPT-OSS-20B MoE Expert Activations - Project Page: GPT-OSS MoE Expert Fingerprinting - GitHub Repository: OpenAI GPT-OSS