NX-AI

--- datasets: - autogluon/chronos_datasets - Salesforce/GiftEvalPretrain pipeline_tag: time-series-forecasting library_name: tirex license: other license_link: https://huggingface.co/NX-AI/TiRex/blob/main/LICENSE license_name: nx-ai-community-license ---

xLSTM-7B This xLSTM-7B was pre-trained on the DCLM and selected high-quality data for in a total of approx. 2.3 T tokens using the `xlstm-jax` framework. How to use it First, install `xlstm`, which now uses the `mlstmkernels` package for triton kernels (tested on python 3.11): If you cannot or do not want to use the triton kernels, you can change them to native PyTorch implementations: Speed results Generation Speed using `torch.cuda.graph` and `torch.compile` optimizations on one NVIDIA H100: | BBH | MMLU-Pro | Math | MUSR | GPQA | IfEval | |-------|----------|--------|------|------|--------| | 0.381 | 0.242 | 0.036 | 0.379|0.280 | 0.244 | Using HuggingFace's `lighteval` in the Leaderboard-v1 settings: |Arc-Challenge (25-shot) |MMLU (5-shot) |Hellaswag (10-shot)|Winogrande (5-shot) |TruthfulQA (0-shot) |GSM8k (5-shot) |OpenbookQA (5-shot) | PiQA (5-shot)| |------------------------|--------------|-------------------|--------------------|--------------------|---------------|--------------------|--------------| | 0.584 |0.589 | 0.710 |0.742 | 0.420 | 0.004 | 0.443 | 0.817 | License NXAI Community License (see `LICENSE` file)

The 1.1 release introduces long period normalisation, a method applied solely during inference. This specific version (1.1-gifteval) includes the 1.1 improvements plus the pretraining dataset has been cleaned to remove overlaps with the GIFT-Eval test dataset. TiRex is a time-series foundation model designed for time series forecasting, with the emphasis to provide state-of-the-art forecasts for both short- and long-term forecasting horizon. TiRex is 35M parameter small and is based on the xLSTM architecture allowing fast and performant forecasts. The model is described in the paper TiRex: Zero-Shot Forecasting across Long and Short Horizons with Enhanced In-Context Learning. - Zero-Shot Forecasting: TiRex performs forecasting without any training on your data. Just download and forecast. - Quantile Predictions: TiRex not only provides point estimates but provides quantile estimates. - State-of-the-art Performance over Long and Short Horizons: TiRex achieves top scores in various time series forecasting benchmarks, see GiftEval and ChronosZS. These benchmark show that TiRex provides great performance for both long and short-term forecasting. TiRex is currently only tested on Linux systems and Nvidia GPUs with compute capability >= 8.0. If you want to use different systems, please check the FAQ in the code repository. It's best to install TiRex in the specified conda environment. The respective conda dependency file is requirementspy26.yaml. We provide an extended quick start example in the GitHub repository. If you have problems please check the FAQ / Troubleshooting section in the GitHub repository and feel free to create a GitHub issue or start a discussion. - chronosdatasets - datasets overlapping with GiftEval have been removed - `solar` and `solar1h` have been filtered for data from Alabama which is included in GiftEval `solar`) - GiftEvalPretrain (Subset - details in the paper) - Synthetic Data If you use TiRex in your research, please cite our work: TiRex is licensed under the NXAI community license.