naver-hyperclovax

HyperCLOVA X SEED 14B Think is a next-generation language model that moves beyond the conventional approach of simply increasing model size to improve performance. It combines HyperCLOVA X’s lightweighting technology for building high-efficiency LLMs with advanced reasoning capabilities. Its development relied on two key technologies: (1) Pruning & Knowledge Distillation, which achieves both compactness and high performance, and (2) a Reinforcement Learning (RL) pipeline, which maximizes reasoning ability. By pruning low-importance parameters and distilling knowledge from a large model into a smaller one, training costs have been significantly reduced. On top of this, the latest RL recipe validated in HyperCLOVA X Think is applied in a multi-stage process: (1) Supervised Fine-Tuning (SFT), (2) Reinforcement Learning with Verifiable Rewards (RLVR), (3) Length Controllability (LC) for reasoning path optimization, and (4) a joint training of Reinforcement Learning from Human Feedback (RLHF) and RLVR. It is a considerable challenge to equip a pruned, knowledge-distilled model with reasoning capabilities, since reductions in training costs and model size often degrade reasoning performance. However, through extensive research experience and persistent trial and error, the HyperCLOVA X team has succeeded in lowering training costs while maintaining reasoning performance comparable to that of larger, resource-intensive models. - Architecture : Transformer-based architecture with Peri-Layer Normalization and Maximal Update Parameterization(μP) (Dense Model) - Parameters : 14.74B - Input/Output Format (Input/Output) : Text / Text - Context Length : 32k `HyperCLOVA X SEED 14B Think` was trained at a significantly lower cost compared to high-performance external models of similar scale. By utilizing HCX’s lightweight training pipeline, it was trained at approximately 52.60× lower cost than `Qwen2.5-14B` and 91.38× lower cost than `Qwen3-14B`. | Model (Base) | GPU Hours (A100-80GB, MFU 50%) | | ------------------------------- | ---------------------------------- | | HyperCLOVA X SEED 14B Think | 68,049 | | Qwen2.5-0.5B | 169,257 | | Qwen2.5-1.5B | 449,643 | | Qwen3-0.6B | 602,460 | | Qwen3-1.7B | 1,063,991 | | HyperCLOVA X Think | 2,197,732 | | Qwen2.5-14B | 3,603,432 | | Qwen3-8B | 3,993,607 | | Qwen3-14B | 6,267,077 | | Qwen3-32B | 14,108,748 | Compared to global models of a similar scale, such as Qwen3 14B, HyperCLOVA X SEED 14B Think demonstrates superior performance in Korean language and cultural understanding, while showing competitive performance in math and coding tasks, which are directly or indirectly related to agent capabilities. This trend remains consistent even when compared with larger models like Qwen3 32B and LG Exaone-Deep 32B. Backbone Benchmarks Performance Comparison (Non-think) | Model | Average | CLIcK | HAERAE-Bench | KOBEST | KorMedMCQA | KMMLU | KoBigBench | KoCommonGEN-v2 | | ------------------------------- | ------- | ------ | ------------ | ------ | ---------- | ------ | ---------- | -------------- | | HyperCLOVA X SEED 14B Think | 0.7269 | 0.7208 | 0.8506 | 0.8570 | 0.6411 | 0.5428 | 0.7482 | 0.6682 | | QWEN3-8B | 0.6759 | 0.6206 | 0.6618 | 0.7919 | 0.6471 | 0.5543 | 0.7186 | 0.5773 | | QWEN3-14B | 0.7079 | 0.6707 | 0.6975 | 0.8174 | 0.6979 | 0.5864 | 0.7507 | 0.5927 | English/American Culture | Model | Average | MMLU | BigBench-Hard | Hellaswag | Winogrande | PIQA | ARC-challenge | Social IQa | | ------------------------------- | ------- | ------ | ------------- | --------- | ---------- | ------ | ------------- | ---------- | | HyperCLOVA X SEED 14B Think | 0.6614 | 0.7121 | 0.6216 | 0.6125 | 0.7593 | 0.7791 | 0.6246 | 0.5205 | | QWEN3-8B | 0.6548 | 0.7490 | 0.6072 | 0.5817 | 0.7198 | 0.7666 | 0.6433 | 0.5159 | | QWEN3-14B | 0.6807 | 0.7885 | 0.6325 | 0.6143 | 0.7356 | 0.8025 | 0.6698 | 0.5215 | Korean/Korea Culture | Model | KMMLU | CSAT-ko-2025 | KorMedMCQA | KoBALT | HAERAE | CLIcK | KoBigBench | LogicKor | |-----------------------------------------|--------|--------|--------|--------|--------|--------|--------|------| | HyperCLOVA X SEED 14B Think (Think) | 0.6649 | 0.7516 | 0.6933 | 0.4500 | 0.8537 | 0.7280 | 0.7974 | 8.74 | | QWEN3-8B | 0.5543 | 0.7200 | 0.6782 | 0.3060 | 0.6618 | 0.6690 | 0.7850 | 8.92 | | QWEN3-14B | 0.4930 | 0.7710 | 0.6850 | 0.3840 | 0.7410 | 0.6880 | 0.8380 | 9.15 | Coding/Math | Model | GSM8k | MATH500 | HumanEval | MBPP | |-----------------------------------------|--------|--------|--------|--------| | HyperCLOVA X SEED 14B Think | 0.9553 | 0.9380 | 0.9451 | 0.8759 | | QWEN3-14B | 0.9590 | 0.9680 | 0.9570 | 0.9080 | | Model | GSM8k | GPT4Eval | MT Bench | Arena-Hard-v0.1 | |---------------------------------------------|--------|--------|--------|--------| | HyperCLOVA X SEED 14B Think (Non-think) | 0.9348 | 0.6741 | 8.2063 | 0.2733 | | HyperCLOVA X SEED 14B Think (Think) | 0.9553 | 0.8200 | 8.8313 | 0.5826 | The chat template for HyperCLOVA X consists of the following elements. - toollist : A list of tools available to the model (in JSON format). If no tools are available, an empty block should be provided. - system : System prompt. If not are available, an empty block should be provided. - user : User input. - assistant : Assistant output. The basic configuration of ChatML block is as follows. - ` ` : Start token of ChatML block - ` ` : End token of ChatML block Given a two-turn conversation between the user and the assistant (`userquery1`, `assistantanswer1`, `userquery2`, `assistantanswer2`), the prompt for the first-turn can be constructed in its simplest form as follows: After the ` ` token (indicating the end of the ChatML block), the model generates text up to the ` ` token. This output corresponds to the assistant's first response (`assistantanswer1`). Based on the assistant's first response (`assistantanswer1`), when the user asks an additional question (`userquery2`), the prompt input to the model is constructed as follows: As in the previous turn, generation continues until the ` ` token appears after ` `. This corresponds to the assistant's second response (`assistantanswer2`). Insert the list of tools available into the toollist block as a JSON list. For example, the following is a case where the only available tool is `getweather`. Additional instructions can be included in the system prompt if needed, and it is recommended to format them as `- {content}`. For example: Suppose the user gives the instruction, 'Tell me the weather in Seoul.' The prompt input to the model would then be as follows: Generation continues until either the ` ` or ` ` token appears immediately after ` `. An example of the model’s output is shown below. HyperCLOVA X checks the list of available tools (toollist), selects the appropriate tool (`getweather`), and returns the necessary information for the tool call in JSON format. - `assistant -> tool/functioncall` means that the assistant(model) invokes a function call. The model stopped generating because ` ` token appeared immediately after ` `. Based on the information generated, `getweather`should now be called. Calling an external function and parsing the result must be implemented additionally. For explanation, let's assume that the result of calling and parsing an external function is `{"result":{"location": "Seoul", "weather": "Sunny", "temperature": 25}}`. The model is now ready to respond to the second turn. Using all the information gathered so far, input the following prompt into the model and have it follow accordingly. - `tool/functioncall` means it should pass the result of the function call to the assistant(model). Just like in the previous turn, generation continues until the ` ` or ` ` token appears immediately after ` `. If the model behaves as expected, the output will look like this. HyperCLOVA X can handle both reasoning and non-reasoning tasks. There is a difference depending on whether the assistant is prompted to 'think' before responding. Based on the previous example, to make HyperCLOVA X respond in reasoning mode, you can input the prompt into the model as follows (excluding the toollist and system) - Note that the prompt ends with `assistant/think\n`(think + `\n). - Generation continues until either the or token appears immediately after ` `. To have the assistant respond in non-reasoning mode (i.e., answer directly), you can input the following prompt. - Note that the prompt ends with `assistant\n`(assistant + `\n`). - Generation continues until either the or token appears immediately after ` `. Adjusting inference length The length of reasoning can be controlled by appending "\nThink for maximum {N} tokens" at the end of the user's utterance. Example Suppose the user says, "Tell me the prime number closest to 1000." If you want the model to reason for approximately 1024 tokens before answering, you would construct the prompt as follows: - Adjusting the reasoning length means guiding the model to reason for approximately the specified number of tokens; it does not always generate for exact number of tokens given. When engaging in multi-turn conversations with reasoning enabled, the reasoning content from previous turns (` assistant/think\n... `) is not included in the next turn's prompt. Instead, as in the standard multi-turn dialogue format, only the final assistant response is included in the prompt passed to the model. The same applies when a function call is made during reasoning: the reasoning content is excluded, and only the function call block (` assistant -> tool/functioncall\n... `) is included in the prompt provided to the model. Example: Solving a simple math problem Suppose the user sends a request: "Calculate the sum of odd numbers from 1 to 10." The following explains step-by-step how this is processed as a multi-turn reasoning interaction. As the ` ` token has been generated, the model concludes the current turn and awaits the next user input. 2. Second Turn Suppose you want to follow up with: "What is the result of adding 10 to that sum?" Here, the prompt is created by excluding the reasoning content (assistant/think) from the first turn and including only the final response (assistant). For all later turns, the reasoning (think) content from previous turns is not added to the prompt as well. - Similarity - They both act as signals to stop the model from generating further responses. - Difference - : After the response generation is halted and the toolinvoke results are processed, the AI turn resumes. - : After the response generation is halted, the AI turn is fully terminated, and the model waits for the user's next input. After downloading the model binaries, including the configuration files, to a local path(`/path/to/hyperclova-x-seed-think-14b`), you can run the following in a Python environment with the Huggingface library(verified to work with version >= 4.53.0) and timm(pytorch-image-models) installed. You can use the `applychattemplate` parameter to explicitly enable or disable the reasoning feature. - The default value for both options is `None`, in which case the model decides on its own whether to reason before answering or to answer directly without reasoning. - `forcereasoning=True`: Forces the model to always reason before answering. - `skipreasoning=True`: Forces the model to answer directly without reasoning. - Passing `None` or `False` has the same effect. - If both are set to True, `forcereasoning=True` takes precedence. Hybrid(the model decides whether to use think or non-think mode) Example Code Example code for function calls (tool usage) For a scenario involving tool usage, you can execute it as follows. - If you have any questions or issues regarding usage, please leave them as an issue in the Discussions section of this page. The HyperCLOVA X SEED Think model is built on a custom LLM architecture based on the LLaMA architecture, incorporating μP and Peri-LN techniques. For convenient use with vLLM, it is available as a dedicated vLLM plugin that can be installed and used with ease once vLLM is set up. 2. vLLM Plugin Build & Installation: While keeping the NAVER-Cloud-HyperCLOVA-X/hcx-vllm-plugin path downloaded in step 1, refer to the commands below. After downloading the model checkpoint to a local path (`/path/to/hyperclova-x-seed-think-14b`), you can perform text inference by running the following commands on a GPU environment with A100 or higher. `vllm serve naver-hyperclovax/HyperCLOVAX-SEED-Think-14B --trustremotecode` - Sampling parameters such as `topk`, `temperature`, `topp`, `repetitionpenalty`, and `maxtokens` can be set freely. - However, the `skipspecialtokens` and `stop` options must be set as below for vLLM to recognize the model's token generation stop signal and cease generation. - You need to add `--enable-auto-tool-choice --tool-call-parser hcx` to the existing script. `vllm serve naver-hyperclovax/HyperCLOVAX-SEED-Think-14B --trustremotecode --enable-auto-tool-choice --tool-call-parser hcx` - If you put the available tools in `tools`, they will be applied to the `toollist` part passed to the model. - Parsed tool calls are returned in the `toolcalls` field. - If there is a response generated by the model other than the tool call, it is returned in the `content` field. Otherwise, `null` is returned. - Serving script - You need to add `--enable-reasoning --reasoning-parser hcx` to the existing script. `vllm serve naver-hyperclovax/HyperCLOVAX-SEED-Think-14B --trustremotecode --enable-reasoning --reasoning-parser hcx` - The `--enable-reasoning` option has been deprecated since vLLM v0.9.0. - If you are using vLLM v0.9.0 or higher, you only need to add `--reasoning-parser hcx` without `--enable-reasoning`. - The reasoning parser extracts the reasoning content from responses generated in reasoning mode. This option does not always make the model operate in reasoning mode, nor does excluding the parser necessarily force non-reasoning operation. - `"chattemplatekwargs": {"forcereasoning": true}` forces reasoning. - `"chattemplatekwargs": {"skipreasoning": true}` forces non-reasoning. - If both are set to `true`, `forcereasoning: true` has higher priority. - If neither is given, the model decides whether to reason or not. - The reasoning part is returned in the `reasoningcontent` field, and the assistant's final response is returned in the `content` field separately. - Serving script - If you want to use both the reasoning parser and the tool call parser, you can combine the reasoning serving script and the tool call serving script. `vllm serve naver-hyperclovax/HyperCLOVAX-SEED-Think-14B --trustremotecode --enable-auto-tool-choice --tool-call-parser hcx --enable-reasoning --reasoning-parser hcx` The model is licensed under HyperCLOVA X SEED Model License Agreement For any other questions, please feel free to contact us at [email protected].