alana89

To fit a pretrained TabSTAR model to your own dataset, install the package: 📚 TabSTAR: A Foundation Tabular Model With Semantically Target-Aware Representations Paper: TabSTAR: A Foundation Tabular Model With Semantically Target-Aware Representations > While deep learning has achieved remarkable success across many domains, it > has historically underperformed on tabular learning tasks, which remain > dominated by gradient boosting decision trees (GBDTs). However, recent > advancements are paving the way for Tabular Foundation Models, which can > leverage real-world knowledge and generalize across diverse datasets, > particularly when the data contains free-text. Although incorporating language > model capabilities into tabular tasks has been explored, most existing methods > utilize static, target-agnostic textual representations, limiting their > effectiveness. We introduce TabSTAR: a Foundation Tabular Model with > Semantically Target-Aware Representations. TabSTAR is designed to enable > transfer learning on tabular data with textual features, with an architecture > free of dataset-specific parameters. It unfreezes a pretrained text encoder and > takes as input target tokens, which provide the model with the context needed > to learn task-specific embeddings. TabSTAR achieves state-of-the-art > performance for both medium- and large-sized datasets across known benchmarks > of classification tasks with text features, and its pretraining phase exhibits > scaling laws in the number of datasets, offering a pathway for further > performance improvements.