ibm-research

--- pipeline_tag: text-generation inference: false license: apache-2.0 library_name: transformers model-index: - name: ibm/PowerMoE-3b results: - task: type: text-generation dataset: type: lm-eval-harness name: ARC metrics: - name: accuracy-norm type: accuracy-norm value: 58.1 verified: false - task: type: text-generation dataset: type: lm-eval-harness name: BoolQ metrics: - name: accuracy type: accuracy value: 65.0 verified: false - task: type: text-generation dataset: type: lm-eval-harness nam

--- license: apache-2.0 library_name: transformers pipeline_tag: feature-extraction tags: - chemistry ---

selfies-ted is an transformer based encoder decoder model for molecular representations using SELFIES. Paper: - SELFIES-TED : A Robust Transformer Model for Molecular Representation using SELFIES - SELF-BART : A Transformer-based Molecular Representation Model using SELFIES

CTI-BERT is a pre-trained language model for the cybersecurity domain. The model was trained on a large corpus of security-related text data, comprising approximately 1.2 billion tokens sourced from a diverse range of sources, including security news articles, vulnerability descriptions, books, academic publications, and security-related Wikipedia pages. For additional technical details and the model's performance metrics, please refer to this paper. This model has a vocabulary of 50,000 tokens and the sequence length of 256. Both the tokenizer and the BERT model were trained from scratch using the runmlm script with the Masked language modeling (MLM) objective. You can use the model for masked language modeling or token embedding generation, but the model is aimed at being fine-tuned on a downstream task, such as sequence classification, text classification or question answering. The model has shown improved performance for various cybersecurity text classification. However, it is not designed to be used as the main model for general-domain text. The following hyperparameters were used during training: - learningrate: 0.0005 - trainbatchsize: 128 - evalbatchsize: 128 - seed: 42 - gradientaccumulationsteps: 16 - totaltrainbatchsize: 2048 - optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-06 - lrschedulertype: linear - lrschedulerwarmupsteps: 10000 - trainingsteps: 200000 - Transformers 4.18.0 - Pytorch 1.12.1+cu102 - Datasets 2.4.0 - Tokenizers 0.12.1

This repository provides PyTorch source code assosiated with our publication, "MHG-GNN: Combination of Molecular Hypergraph Grammar with Graph Neural Network" We present MHG-GNN, an autoencoder architecture that has an encoder based on GNN and a decoder based on a sequential model with MHG. Since the encoder is a GNN variant, MHG-GNN can accept any molecule as input, and demonstrate high predictive performance on molecular graph data. In addition, the decoder inherits the theoretical guarantee of MHG on always generating a structurally valid molecule as output. 1. Getting Started 1. Pretrained Models and Training Logs 2. Installation 2. Feature Extraction This code and environment have been tested on Intel E5-2667 CPUs at 3.30GHz and NVIDIA A100 Tensor Core GPUs. We provide checkpoints of the MHG-GNN model pre-trained on a dataset of ~1.34M molecules curated from PubChem. (later) For model weights: [HuggingFace Link]() Add the MHG-GNN `pre-trained weights.pt` to the `models/` directory according to your needs. We recommend to create a virtual environment. For example: Type the following command once the virtual environment is activated: The example notebook mhg-gnnencoderdecoderexample.ipynb contains code to load checkpoint files and use the pre-trained model for encoder and decoder tasks. For decoder, you can use the function, so you can return from embeddings to SMILES strings:

SMILES-based State-Space Encoder-Decoder (SMI-SSED) - MoLMamba This repository provides PyTorch source code associated with our publication, "A Mamba-Based Foundation Model for Chemistry". For more information contact: [email protected] or [email protected]. We present a Mamba-based encoder-decoder chemical foundation model, SMILES-based State-Space Encoder-Decoder (SMI-SSED), pre-trained on a curated dataset of 91 million SMILES samples sourced from PubChem, equivalent to 4 billion molecular tokens. SMI-SSED supports various complex tasks, including quantum property prediction, with two main variants ($336$ and $8 \times 336M$). Our experiments across multiple benchmark datasets demonstrate state-of-the-art performance for various tasks. - PyTorch (`.pt`): smissed130.pt - safetensors (`.bin`): smissed130.bin For more information contact: [email protected] or [email protected]. 1. Getting Started 1. Pretrained Models and Training Logs 2. Replicating Conda Environment 2. Pretraining 3. Finetuning 4. Feature Extraction This code and environment have been tested on Nvidia V100s and Nvidia A100s We provide checkpoints of the SMI-SSED model pre-trained on a dataset of ~91M molecules curated from PubChem. The pre-trained model shows competitive performance on classification and regression benchmarks from MoleculeNet. Add the SMI-SSED `pre-trained weights.pt` to the `inference/` or `finetune/` directory according to your needs. The directory structure should look like the following: Follow these steps to replicate our Conda environment and install the necessary libraries: For pretraining, we use two strategies: the masked language model method to train the encoder part and an encoder-decoder strategy to refine SMILES reconstruction and improve the generated latent space. SMI-SSED is pre-trained on canonicalized and curated 91M SMILES from PubChem with the following constraints: - Compounds are filtered to a maximum length of 202 tokens during preprocessing. - A 95/5/0 split is used for encoder training, with 5% of the data for decoder pretraining. - A 100/0/0 split is also used to train the encoder and decoder directly, enhancing model performance. The pretraining code provides examples of data processing and model training on a smaller dataset, requiring 8 A100 GPUs. Use `trainmodelD.py` to train only the decoder or `trainmodelED.py` to train both the encoder and decoder. The finetuning datasets and environment can be found in the finetune directory. After setting up the environment, you can run a finetuning task with: Finetuning training/checkpointing resources will be available in directories named `checkpoint `. The example notebook smissedencoderdecoderexample.ipynb contains code to load checkpoint files and use the pre-trained model for encoder and decoder tasks. It also includes examples of classification and regression tasks. For model weights: HuggingFace Link For decoder, you can use the function, so you can return from embeddings to SMILES strings:

Accurate prediction of drug-target binding affinity is essential in the early stages of drug discovery. This is an example of finetuning ibm/biomed.omics.bl.sm-ted-400 the task. Prediction of binding affinities using pKd, the negative logarithm of the dissociation constant, which reflects the strength of the interaction between a small molecule (drug) and a protein (target). The expected inputs for the model are the amino acid sequence of the target and the SMILES representation of the drug. The benchmark used for fine-tuning defined on: `https://tdcommons.ai/multipredtasks/dti/` We also harmonize the values using data.harmonizeaffinities(mode = 'maxaffinity') and transforming to log-scale. By default, we are using Drug+Target cold-split, as provided by tdcommons. - Developers: IBM Research - GitHub Repository: https://github.com/BiomedSciAI/biomed-multi-alignment - Paper: https://arxiv.org/abs/2410.22367 - Release Date: Oct 28th, 2024 - License: Apache 2.0. Using `ibm/biomed.omics.bl.sm.ma-ted-458m` requires installing https://github.com/BiomedSciAI/biomed-multi-alignment A simple example for a task already supported by `ibm/biomed.omics.bl.sm.ma-ted-458m`: For more advanced usage, see our detailed example at: on `https://github.com/BiomedSciAI/biomed-multi-alignment` If you found our work useful, please consider giving a star to the repo and cite our paper:

ibm-research/biomed.sm.mv-te-84m-CYP-ligandscaffoldbalanced-CYP3A4-101 `biomed.sm.mv-te-84m` is a multimodal biomedical foundation model for small molecules created using MMELON (Multi-view Molecular Embedding with Late Fusion), a flexible approach to aggregate multiple views (sequence, image, graph) of molecules in a foundation model setting. While models based on single view representation typically performs well on some downstream tasks and not others, the multi-view model performs robustly across a wide range of property prediction tasks encompassing ligand-protein binding, molecular solubility, metabolism and toxicity. It has been applied to screen compounds against a large (> 100 targets) set of G Protein-Coupled receptors (GPCRs) to identify strong binders for 33 targets related to Alzheimer’s disease, which are validated through structure-based modeling and identification of key binding motifs Multi-view biomedical foundation models for molecule-target and property prediction. - Developers: IBM Research - GitHub Repository: https://github.com/BiomedSciAI/biomed-multi-view - Paper: Multi-view biomedical foundation models for molecule-target and property prediction - Release Date: Oct 28th, 2024 - License: Apache 2.0 Source code for the model and finetuning is made available in this repository. Image Representation: Captures the 2D visual depiction of molecular structures, highlighting features like symmetry, bond angles, and functional groups. Molecular images are generated using RDKit and undergo data augmentation during training to enhance robustness. Graph Representation: Encodes molecules as undirected graphs where nodes represent atoms and edges represent bonds. Atom-specific properties (e.g., atomic number, chirality) and bond-specific properties (e.g., bond type, stereochemistry) are embedded using categorical embedding techniques. Text Representation: Utilizes SMILES strings to represent chemical structures, tokenized with a custom tokenizer. The sequences are embedded using a transformer-based architecture to capture the sequential nature of the chemical information. The embeddings from these single-view pre-trained encoders are combined using an attention-based aggregator module. This module learns to weight each view appropriately, producing a unified multi-view embedding. This approach leverages the strengths of each representation to improve performance on downstream predictive tasks. The model is intended for (1) Molecular property prediction. The pre-trained model may be fine-tuned for both regression and classification tasks. Examples include but are not limited to binding affinity, solubility and toxicity. (2) Pre-trained model embeddings may be used as the basis for similarity measures to search a chemical library. (3) Small molecule embeddings provided by the model may be combined with protein embeddings to fine-tune on tasks that utilize both small molecule and protein representation. (4) Select task-specific fine-tuned models are given as examples. Through listed activities, model may aid in aspects of the molecular discovery such as lead finding or optimization. The model’s domain of applicability is small, drug-like molecules. It is intended for use with molecules less than 1000 Da molecular weight. The MMELON approach itself may be extended to include proteins and other macromolecules but does not at present provide embeddings for such entities. The model is at present not intended for molecular generation. Molecules must be given as a valid SMILES string that represents a valid chemically bonded graph. Invalid inputs will impact performance or lead to error. Using `SmallMoleculeMultiView` API requires the codebase https://github.com/BiomedSciAI/biomed-multi-view Installation Follow these steps to set up the `biomed-multi-view` codebase on your system. Prerequisites Operating System: Linux or macOS Python Version: Python 3.11 Conda: Anaconda or Miniconda installed Git: Version control to clone the repository Step 1: Set up the project directory Choose a root directory where you want to install `biomed-multi-view`. For example: Step 3: Clone the repository Navigate to the project directory and clone the repository: Note: If you prefer using SSH, ensure that your SSH keys are set up with GitHub and use the following command: Step 4: Install package dependencies Install the package in editable mode along with development dependencies: Step 5: macOS-Specific instructions (Apple Silicon) If you are using a Mac with Apple Silicon (M1/M2/M3) and the zsh shell, you may need to disable globbing for the installation command: Install macOS-specific requirements optimized for Apple’s Metal Performance Shaders (MPS): Step 6: Installation verification (optional) Verify that the installation was successful by running unit tests You can generate embeddings for a given molecule using the pretrained model with the following code. You can use the finetuned models to make predictions on new data. For more advanced usage, see our detailed examples at: https://github.com/BiomedSciAI/biomed-multi-view If you found our work useful, please consider giving a star to the repo and cite our paper:

Protein solubility is a critical factor in both pharmaceutical research and production processes, as it can significantly impact the quality and function of a protein. This is an example for finetuning `ibm/biomed.omics.bl.sm-ted-458m` for protein solubility prediction (binary classification) based solely on the amino acid sequence. The benchmark defined in: https://academic.oup.com/bioinformatics/article/34/15/2605/4938490 Data retrieved from: https://zenodo.org/records/1162886 - Developers: IBM Research - GitHub Repository: https://github.com/BiomedSciAI/biomed-multi-alignment - Paper: https://arxiv.org/abs/2410.22367 - Release Date: Oct 28th, 2024 - License: Apache 2.0. Using `ibm/biomed.omics.bl.sm.ma-ted-458m` requires installing https://github.com/BiomedSciAI/biomed-multi-alignment A simple example for a task already supported by `ibm/biomed.omics.bl.sm.ma-ted-458m`: For more advanced usage, see our detailed example at: on `https://github.com/BiomedSciAI/biomed-multi-alignment` If you found our work useful, please consider giving a star to the repo and cite our paper:

ibm-research/biomed.sm.mv-te-84m-CYP-ligandscaffoldbalanced-CYP1A2-101 `biomed.sm.mv-te-84m` is a multimodal biomedical foundation model for small molecules created using MMELON (Multi-view Molecular Embedding with Late Fusion), a flexible approach to aggregate multiple views (sequence, image, graph) of molecules in a foundation model setting. While models based on single view representation typically performs well on some downstream tasks and not others, the multi-view model performs robustly across a wide range of property prediction tasks encompassing ligand-protein binding, molecular solubility, metabolism and toxicity. It has been applied to screen compounds against a large (> 100 targets) set of G Protein-Coupled receptors (GPCRs) to identify strong binders for 33 targets related to Alzheimer’s disease, which are validated through structure-based modeling and identification of key binding motifs Multi-view biomedical foundation models for molecule-target and property prediction. - Developers: IBM Research - GitHub Repository: https://github.com/BiomedSciAI/biomed-multi-view - Paper: Multi-view biomedical foundation models for molecule-target and property prediction - Release Date: Oct 28th, 2024 - License: Apache 2.0 Source code for the model and finetuning is made available in this repository. Image Representation: Captures the 2D visual depiction of molecular structures, highlighting features like symmetry, bond angles, and functional groups. Molecular images are generated using RDKit and undergo data augmentation during training to enhance robustness. Graph Representation: Encodes molecules as undirected graphs where nodes represent atoms and edges represent bonds. Atom-specific properties (e.g., atomic number, chirality) and bond-specific properties (e.g., bond type, stereochemistry) are embedded using categorical embedding techniques. Text Representation: Utilizes SMILES strings to represent chemical structures, tokenized with a custom tokenizer. The sequences are embedded using a transformer-based architecture to capture the sequential nature of the chemical information. The embeddings from these single-view pre-trained encoders are combined using an attention-based aggregator module. This module learns to weight each view appropriately, producing a unified multi-view embedding. This approach leverages the strengths of each representation to improve performance on downstream predictive tasks. The model is intended for (1) Molecular property prediction. The pre-trained model may be fine-tuned for both regression and classification tasks. Examples include but are not limited to binding affinity, solubility and toxicity. (2) Pre-trained model embeddings may be used as the basis for similarity measures to search a chemical library. (3) Small molecule embeddings provided by the model may be combined with protein embeddings to fine-tune on tasks that utilize both small molecule and protein representation. (4) Select task-specific fine-tuned models are given as examples. Through listed activities, model may aid in aspects of the molecular discovery such as lead finding or optimization. The model’s domain of applicability is small, drug-like molecules. It is intended for use with molecules less than 1000 Da molecular weight. The MMELON approach itself may be extended to include proteins and other macromolecules but does not at present provide embeddings for such entities. The model is at present not intended for molecular generation. Molecules must be given as a valid SMILES string that represents a valid chemically bonded graph. Invalid inputs will impact performance or lead to error. Using `SmallMoleculeMultiView` API requires the codebase https://github.com/BiomedSciAI/biomed-multi-view Installation Follow these steps to set up the `biomed-multi-view` codebase on your system. Prerequisites Operating System: Linux or macOS Python Version: Python 3.11 Conda: Anaconda or Miniconda installed Git: Version control to clone the repository Step 1: Set up the project directory Choose a root directory where you want to install `biomed-multi-view`. For example: Step 3: Clone the repository Navigate to the project directory and clone the repository: Note: If you prefer using SSH, ensure that your SSH keys are set up with GitHub and use the following command: Step 4: Install package dependencies Install the package in editable mode along with development dependencies: Step 5: macOS-Specific instructions (Apple Silicon) If you are using a Mac with Apple Silicon (M1/M2/M3) and the zsh shell, you may need to disable globbing for the installation command: Install macOS-specific requirements optimized for Apple’s Metal Performance Shaders (MPS): Step 6: Installation verification (optional) Verify that the installation was successful by running unit tests You can generate embeddings for a given molecule using the pretrained model with the following code. You can use the finetuned models to make predictions on new data. For more advanced usage, see our detailed examples at: https://github.com/BiomedSciAI/biomed-multi-view If you found our work useful, please consider giving a star to the repo and cite our paper:

ibm-research/biomed.sm.mv-te-84m-CYP-ligandscaffoldbalanced-CYP2C9-101 `biomed.sm.mv-te-84m` is a multimodal biomedical foundation model for small molecules created using MMELON (Multi-view Molecular Embedding with Late Fusion), a flexible approach to aggregate multiple views (sequence, image, graph) of molecules in a foundation model setting. While models based on single view representation typically performs well on some downstream tasks and not others, the multi-view model performs robustly across a wide range of property prediction tasks encompassing ligand-protein binding, molecular solubility, metabolism and toxicity. It has been applied to screen compounds against a large (> 100 targets) set of G Protein-Coupled receptors (GPCRs) to identify strong binders for 33 targets related to Alzheimer’s disease, which are validated through structure-based modeling and identification of key binding motifs Multi-view biomedical foundation models for molecule-target and property prediction. - Developers: IBM Research - GitHub Repository: https://github.com/BiomedSciAI/biomed-multi-view - Paper: Multi-view biomedical foundation models for molecule-target and property prediction - Release Date: Oct 28th, 2024 - License: Apache 2.0 Source code for the model and finetuning is made available in this repository. Image Representation: Captures the 2D visual depiction of molecular structures, highlighting features like symmetry, bond angles, and functional groups. Molecular images are generated using RDKit and undergo data augmentation during training to enhance robustness. Graph Representation: Encodes molecules as undirected graphs where nodes represent atoms and edges represent bonds. Atom-specific properties (e.g., atomic number, chirality) and bond-specific properties (e.g., bond type, stereochemistry) are embedded using categorical embedding techniques. Text Representation: Utilizes SMILES strings to represent chemical structures, tokenized with a custom tokenizer. The sequences are embedded using a transformer-based architecture to capture the sequential nature of the chemical information. The embeddings from these single-view pre-trained encoders are combined using an attention-based aggregator module. This module learns to weight each view appropriately, producing a unified multi-view embedding. This approach leverages the strengths of each representation to improve performance on downstream predictive tasks. The model is intended for (1) Molecular property prediction. The pre-trained model may be fine-tuned for both regression and classification tasks. Examples include but are not limited to binding affinity, solubility and toxicity. (2) Pre-trained model embeddings may be used as the basis for similarity measures to search a chemical library. (3) Small molecule embeddings provided by the model may be combined with protein embeddings to fine-tune on tasks that utilize both small molecule and protein representation. (4) Select task-specific fine-tuned models are given as examples. Through listed activities, model may aid in aspects of the molecular discovery such as lead finding or optimization. The model’s domain of applicability is small, drug-like molecules. It is intended for use with molecules less than 1000 Da molecular weight. The MMELON approach itself may be extended to include proteins and other macromolecules but does not at present provide embeddings for such entities. The model is at present not intended for molecular generation. Molecules must be given as a valid SMILES string that represents a valid chemically bonded graph. Invalid inputs will impact performance or lead to error. Using `SmallMoleculeMultiView` API requires the codebase https://github.com/BiomedSciAI/biomed-multi-view Installation Follow these steps to set up the `biomed-multi-view` codebase on your system. Prerequisites Operating System: Linux or macOS Python Version: Python 3.11 Conda: Anaconda or Miniconda installed Git: Version control to clone the repository Step 1: Set up the project directory Choose a root directory where you want to install `biomed-multi-view`. For example: Step 3: Clone the repository Navigate to the project directory and clone the repository: Note: If you prefer using SSH, ensure that your SSH keys are set up with GitHub and use the following command: Step 4: Install package dependencies Install the package in editable mode along with development dependencies: Step 5: macOS-Specific instructions (Apple Silicon) If you are using a Mac with Apple Silicon (M1/M2/M3) and the zsh shell, you may need to disable globbing for the installation command: Install macOS-specific requirements optimized for Apple’s Metal Performance Shaders (MPS): Step 6: Installation verification (optional) Verify that the installation was successful by running unit tests You can generate embeddings for a given molecule using the pretrained model with the following code. You can use the finetuned models to make predictions on new data. For more advanced usage, see our detailed examples at: https://github.com/BiomedSciAI/biomed-multi-view If you found our work useful, please consider giving a star to the repo and cite our paper:

ibm-research/biomed.sm.mv-te-84m-ComputationalADME-random-RLM-101 `biomed.sm.mv-te-84m` is a multimodal biomedical foundation model for small molecules created using MMELON (Multi-view Molecular Embedding with Late Fusion), a flexible approach to aggregate multiple views (sequence, image, graph) of molecules in a foundation model setting. While models based on single view representation typically performs well on some downstream tasks and not others, the multi-view model performs robustly across a wide range of property prediction tasks encompassing ligand-protein binding, molecular solubility, metabolism and toxicity. It has been applied to screen compounds against a large (> 100 targets) set of G Protein-Coupled receptors (GPCRs) to identify strong binders for 33 targets related to Alzheimer’s disease, which are validated through structure-based modeling and identification of key binding motifs Multi-view biomedical foundation models for molecule-target and property prediction. - Developers: IBM Research - GitHub Repository: https://github.com/BiomedSciAI/biomed-multi-view - Paper: Multi-view biomedical foundation models for molecule-target and property prediction - Release Date: Oct 28th, 2024 - License: Apache 2.0 Source code for the model and finetuning is made available in this repository. Image Representation: Captures the 2D visual depiction of molecular structures, highlighting features like symmetry, bond angles, and functional groups. Molecular images are generated using RDKit and undergo data augmentation during training to enhance robustness. Graph Representation: Encodes molecules as undirected graphs where nodes represent atoms and edges represent bonds. Atom-specific properties (e.g., atomic number, chirality) and bond-specific properties (e.g., bond type, stereochemistry) are embedded using categorical embedding techniques. Text Representation: Utilizes SMILES strings to represent chemical structures, tokenized with a custom tokenizer. The sequences are embedded using a transformer-based architecture to capture the sequential nature of the chemical information. The embeddings from these single-view pre-trained encoders are combined using an attention-based aggregator module. This module learns to weight each view appropriately, producing a unified multi-view embedding. This approach leverages the strengths of each representation to improve performance on downstream predictive tasks. The model is intended for (1) Molecular property prediction. The pre-trained model may be fine-tuned for both regression and classification tasks. Examples include but are not limited to binding affinity, solubility and toxicity. (2) Pre-trained model embeddings may be used as the basis for similarity measures to search a chemical library. (3) Small molecule embeddings provided by the model may be combined with protein embeddings to fine-tune on tasks that utilize both small molecule and protein representation. (4) Select task-specific fine-tuned models are given as examples. Through listed activities, model may aid in aspects of the molecular discovery such as lead finding or optimization. The model’s domain of applicability is small, drug-like molecules. It is intended for use with molecules less than 1000 Da molecular weight. The MMELON approach itself may be extended to include proteins and other macromolecules but does not at present provide embeddings for such entities. The model is at present not intended for molecular generation. Molecules must be given as a valid SMILES string that represents a valid chemically bonded graph. Invalid inputs will impact performance or lead to error. Using `SmallMoleculeMultiView` API requires the codebase https://github.com/BiomedSciAI/biomed-multi-view Installation Follow these steps to set up the `biomed-multi-view` codebase on your system. Prerequisites Operating System: Linux or macOS Python Version: Python 3.11 Conda: Anaconda or Miniconda installed Git: Version control to clone the repository Step 1: Set up the project directory Choose a root directory where you want to install `biomed-multi-view`. For example: Step 3: Clone the repository Navigate to the project directory and clone the repository: Note: If you prefer using SSH, ensure that your SSH keys are set up with GitHub and use the following command: Step 4: Install package dependencies Install the package in editable mode along with development dependencies: Step 5: macOS-Specific instructions (Apple Silicon) If you are using a Mac with Apple Silicon (M1/M2/M3) and the zsh shell, you may need to disable globbing for the installation command: Install macOS-specific requirements optimized for Apple’s Metal Performance Shaders (MPS): Step 6: Installation verification (optional) Verify that the installation was successful by running unit tests You can generate embeddings for a given molecule using the pretrained model with the following code. You can use the finetuned models to make predictions on new data. For more advanced usage, see our detailed examples at: https://github.com/BiomedSciAI/biomed-multi-view If you found our work useful, please consider giving a star to the repo and cite our paper:

ibm-research/biomed.sm.mv-te-84m-ComputationalADME-random-SOLUBILITY-101 `biomed.sm.mv-te-84m` is a multimodal biomedical foundation model for small molecules created using MMELON (Multi-view Molecular Embedding with Late Fusion), a flexible approach to aggregate multiple views (sequence, image, graph) of molecules in a foundation model setting. While models based on single view representation typically performs well on some downstream tasks and not others, the multi-view model performs robustly across a wide range of property prediction tasks encompassing ligand-protein binding, molecular solubility, metabolism and toxicity. It has been applied to screen compounds against a large (> 100 targets) set of G Protein-Coupled receptors (GPCRs) to identify strong binders for 33 targets related to Alzheimer’s disease, which are validated through structure-based modeling and identification of key binding motifs Multi-view biomedical foundation models for molecule-target and property prediction. - Developers: IBM Research - GitHub Repository: https://github.com/BiomedSciAI/biomed-multi-view - Paper: Multi-view biomedical foundation models for molecule-target and property prediction - Release Date: Oct 28th, 2024 - License: Apache 2.0 Source code for the model and finetuning is made available in this repository. Image Representation: Captures the 2D visual depiction of molecular structures, highlighting features like symmetry, bond angles, and functional groups. Molecular images are generated using RDKit and undergo data augmentation during training to enhance robustness. Graph Representation: Encodes molecules as undirected graphs where nodes represent atoms and edges represent bonds. Atom-specific properties (e.g., atomic number, chirality) and bond-specific properties (e.g., bond type, stereochemistry) are embedded using categorical embedding techniques. Text Representation: Utilizes SMILES strings to represent chemical structures, tokenized with a custom tokenizer. The sequences are embedded using a transformer-based architecture to capture the sequential nature of the chemical information. The embeddings from these single-view pre-trained encoders are combined using an attention-based aggregator module. This module learns to weight each view appropriately, producing a unified multi-view embedding. This approach leverages the strengths of each representation to improve performance on downstream predictive tasks. The model is intended for (1) Molecular property prediction. The pre-trained model may be fine-tuned for both regression and classification tasks. Examples include but are not limited to binding affinity, solubility and toxicity. (2) Pre-trained model embeddings may be used as the basis for similarity measures to search a chemical library. (3) Small molecule embeddings provided by the model may be combined with protein embeddings to fine-tune on tasks that utilize both small molecule and protein representation. (4) Select task-specific fine-tuned models are given as examples. Through listed activities, model may aid in aspects of the molecular discovery such as lead finding or optimization. The model’s domain of applicability is small, drug-like molecules. It is intended for use with molecules less than 1000 Da molecular weight. The MMELON approach itself may be extended to include proteins and other macromolecules but does not at present provide embeddings for such entities. The model is at present not intended for molecular generation. Molecules must be given as a valid SMILES string that represents a valid chemically bonded graph. Invalid inputs will impact performance or lead to error. Using `SmallMoleculeMultiView` API requires the codebase https://github.com/BiomedSciAI/biomed-multi-view Installation Follow these steps to set up the `biomed-multi-view` codebase on your system. Prerequisites Operating System: Linux or macOS Python Version: Python 3.11 Conda: Anaconda or Miniconda installed Git: Version control to clone the repository Step 1: Set up the project directory Choose a root directory where you want to install `biomed-multi-view`. For example: Step 3: Clone the repository Navigate to the project directory and clone the repository: Note: If you prefer using SSH, ensure that your SSH keys are set up with GitHub and use the following command: Step 4: Install package dependencies Install the package in editable mode along with development dependencies: Step 5: macOS-Specific instructions (Apple Silicon) If you are using a Mac with Apple Silicon (M1/M2/M3) and the zsh shell, you may need to disable globbing for the installation command: Install macOS-specific requirements optimized for Apple’s Metal Performance Shaders (MPS): Step 6: Installation verification (optional) Verify that the installation was successful by running unit tests You can generate embeddings for a given molecule using the pretrained model with the following code. You can use the finetuned models to make predictions on new data. For more advanced usage, see our detailed examples at: https://github.com/BiomedSciAI/biomed-multi-view If you found our work useful, please consider giving a star to the repo and cite our paper:

ibm-research/biomed.sm.mv-te-84m-ComputationalADME-random-HLM-101 `biomed.sm.mv-te-84m` is a multimodal biomedical foundation model for small molecules created using MMELON (Multi-view Molecular Embedding with Late Fusion), a flexible approach to aggregate multiple views (sequence, image, graph) of molecules in a foundation model setting. While models based on single view representation typically performs well on some downstream tasks and not others, the multi-view model performs robustly across a wide range of property prediction tasks encompassing ligand-protein binding, molecular solubility, metabolism and toxicity. It has been applied to screen compounds against a large (> 100 targets) set of G Protein-Coupled receptors (GPCRs) to identify strong binders for 33 targets related to Alzheimer’s disease, which are validated through structure-based modeling and identification of key binding motifs Multi-view biomedical foundation models for molecule-target and property prediction. - Developers: IBM Research - GitHub Repository: https://github.com/BiomedSciAI/biomed-multi-view - Paper: Multi-view biomedical foundation models for molecule-target and property prediction - Release Date: Oct 28th, 2024 - License: Apache 2.0 Source code for the model and finetuning is made available in this repository. Image Representation: Captures the 2D visual depiction of molecular structures, highlighting features like symmetry, bond angles, and functional groups. Molecular images are generated using RDKit and undergo data augmentation during training to enhance robustness. Graph Representation: Encodes molecules as undirected graphs where nodes represent atoms and edges represent bonds. Atom-specific properties (e.g., atomic number, chirality) and bond-specific properties (e.g., bond type, stereochemistry) are embedded using categorical embedding techniques. Text Representation: Utilizes SMILES strings to represent chemical structures, tokenized with a custom tokenizer. The sequences are embedded using a transformer-based architecture to capture the sequential nature of the chemical information. The embeddings from these single-view pre-trained encoders are combined using an attention-based aggregator module. This module learns to weight each view appropriately, producing a unified multi-view embedding. This approach leverages the strengths of each representation to improve performance on downstream predictive tasks. The model is intended for (1) Molecular property prediction. The pre-trained model may be fine-tuned for both regression and classification tasks. Examples include but are not limited to binding affinity, solubility and toxicity. (2) Pre-trained model embeddings may be used as the basis for similarity measures to search a chemical library. (3) Small molecule embeddings provided by the model may be combined with protein embeddings to fine-tune on tasks that utilize both small molecule and protein representation. (4) Select task-specific fine-tuned models are given as examples. Through listed activities, model may aid in aspects of the molecular discovery such as lead finding or optimization. The model’s domain of applicability is small, drug-like molecules. It is intended for use with molecules less than 1000 Da molecular weight. The MMELON approach itself may be extended to include proteins and other macromolecules but does not at present provide embeddings for such entities. The model is at present not intended for molecular generation. Molecules must be given as a valid SMILES string that represents a valid chemically bonded graph. Invalid inputs will impact performance or lead to error. Using `SmallMoleculeMultiView` API requires the codebase https://github.com/BiomedSciAI/biomed-multi-view Installation Follow these steps to set up the `biomed-multi-view` codebase on your system. Prerequisites Operating System: Linux or macOS Python Version: Python 3.11 Conda: Anaconda or Miniconda installed Git: Version control to clone the repository Step 1: Set up the project directory Choose a root directory where you want to install `biomed-multi-view`. For example: Step 3: Clone the repository Navigate to the project directory and clone the repository: Note: If you prefer using SSH, ensure that your SSH keys are set up with GitHub and use the following command: Step 4: Install package dependencies Install the package in editable mode along with development dependencies: Step 5: macOS-Specific instructions (Apple Silicon) If you are using a Mac with Apple Silicon (M1/M2/M3) and the zsh shell, you may need to disable globbing for the installation command: Install macOS-specific requirements optimized for Apple’s Metal Performance Shaders (MPS): Step 6: Installation verification (optional) Verify that the installation was successful by running unit tests You can generate embeddings for a given molecule using the pretrained model with the following code. You can use the finetuned models to make predictions on new data. For more advanced usage, see our detailed examples at: https://github.com/BiomedSciAI/biomed-multi-view If you found our work useful, please consider giving a star to the repo and cite our paper:

ibm-research/biomed.sm.mv-te-84m-ComputationalADME-random-HPPB-101 `biomed.sm.mv-te-84m` is a multimodal biomedical foundation model for small molecules created using MMELON (Multi-view Molecular Embedding with Late Fusion), a flexible approach to aggregate multiple views (sequence, image, graph) of molecules in a foundation model setting. While models based on single view representation typically performs well on some downstream tasks and not others, the multi-view model performs robustly across a wide range of property prediction tasks encompassing ligand-protein binding, molecular solubility, metabolism and toxicity. It has been applied to screen compounds against a large (> 100 targets) set of G Protein-Coupled receptors (GPCRs) to identify strong binders for 33 targets related to Alzheimer’s disease, which are validated through structure-based modeling and identification of key binding motifs Multi-view biomedical foundation models for molecule-target and property prediction. - Developers: IBM Research - GitHub Repository: https://github.com/BiomedSciAI/biomed-multi-view - Paper: Multi-view biomedical foundation models for molecule-target and property prediction - Release Date: Oct 28th, 2024 - License: Apache 2.0 Source code for the model and finetuning is made available in this repository. Image Representation: Captures the 2D visual depiction of molecular structures, highlighting features like symmetry, bond angles, and functional groups. Molecular images are generated using RDKit and undergo data augmentation during training to enhance robustness. Graph Representation: Encodes molecules as undirected graphs where nodes represent atoms and edges represent bonds. Atom-specific properties (e.g., atomic number, chirality) and bond-specific properties (e.g., bond type, stereochemistry) are embedded using categorical embedding techniques. Text Representation: Utilizes SMILES strings to represent chemical structures, tokenized with a custom tokenizer. The sequences are embedded using a transformer-based architecture to capture the sequential nature of the chemical information. The embeddings from these single-view pre-trained encoders are combined using an attention-based aggregator module. This module learns to weight each view appropriately, producing a unified multi-view embedding. This approach leverages the strengths of each representation to improve performance on downstream predictive tasks. The model is intended for (1) Molecular property prediction. The pre-trained model may be fine-tuned for both regression and classification tasks. Examples include but are not limited to binding affinity, solubility and toxicity. (2) Pre-trained model embeddings may be used as the basis for similarity measures to search a chemical library. (3) Small molecule embeddings provided by the model may be combined with protein embeddings to fine-tune on tasks that utilize both small molecule and protein representation. (4) Select task-specific fine-tuned models are given as examples. Through listed activities, model may aid in aspects of the molecular discovery such as lead finding or optimization. The model’s domain of applicability is small, drug-like molecules. It is intended for use with molecules less than 1000 Da molecular weight. The MMELON approach itself may be extended to include proteins and other macromolecules but does not at present provide embeddings for such entities. The model is at present not intended for molecular generation. Molecules must be given as a valid SMILES string that represents a valid chemically bonded graph. Invalid inputs will impact performance or lead to error. Using `SmallMoleculeMultiView` API requires the codebase https://github.com/BiomedSciAI/biomed-multi-view Installation Follow these steps to set up the `biomed-multi-view` codebase on your system. Prerequisites Operating System: Linux or macOS Python Version: Python 3.11 Conda: Anaconda or Miniconda installed Git: Version control to clone the repository Step 1: Set up the project directory Choose a root directory where you want to install `biomed-multi-view`. For example: Step 3: Clone the repository Navigate to the project directory and clone the repository: Note: If you prefer using SSH, ensure that your SSH keys are set up with GitHub and use the following command: Step 4: Install package dependencies Install the package in editable mode along with development dependencies: Step 5: macOS-Specific instructions (Apple Silicon) If you are using a Mac with Apple Silicon (M1/M2/M3) and the zsh shell, you may need to disable globbing for the installation command: Install macOS-specific requirements optimized for Apple’s Metal Performance Shaders (MPS): Step 6: Installation verification (optional) Verify that the installation was successful by running unit tests You can generate embeddings for a given molecule using the pretrained model with the following code. You can use the finetuned models to make predictions on new data. For more advanced usage, see our detailed examples at: https://github.com/BiomedSciAI/biomed-multi-view If you found our work useful, please consider giving a star to the repo and cite our paper:

ibm-research/biomed.sm.mv-te-84m-ComputationalADME-random-MDR1-MDCK-ER-101 `biomed.sm.mv-te-84m` is a multimodal biomedical foundation model for small molecules created using MMELON (Multi-view Molecular Embedding with Late Fusion), a flexible approach to aggregate multiple views (sequence, image, graph) of molecules in a foundation model setting. While models based on single view representation typically performs well on some downstream tasks and not others, the multi-view model performs robustly across a wide range of property prediction tasks encompassing ligand-protein binding, molecular solubility, metabolism and toxicity. It has been applied to screen compounds against a large (> 100 targets) set of G Protein-Coupled receptors (GPCRs) to identify strong binders for 33 targets related to Alzheimer’s disease, which are validated through structure-based modeling and identification of key binding motifs Multi-view biomedical foundation models for molecule-target and property prediction. - Developers: IBM Research - GitHub Repository: https://github.com/BiomedSciAI/biomed-multi-view - Paper: Multi-view biomedical foundation models for molecule-target and property prediction - Release Date: Oct 28th, 2024 - License: Apache 2.0 Source code for the model and finetuning is made available in this repository. Image Representation: Captures the 2D visual depiction of molecular structures, highlighting features like symmetry, bond angles, and functional groups. Molecular images are generated using RDKit and undergo data augmentation during training to enhance robustness. Graph Representation: Encodes molecules as undirected graphs where nodes represent atoms and edges represent bonds. Atom-specific properties (e.g., atomic number, chirality) and bond-specific properties (e.g., bond type, stereochemistry) are embedded using categorical embedding techniques. Text Representation: Utilizes SMILES strings to represent chemical structures, tokenized with a custom tokenizer. The sequences are embedded using a transformer-based architecture to capture the sequential nature of the chemical information. The embeddings from these single-view pre-trained encoders are combined using an attention-based aggregator module. This module learns to weight each view appropriately, producing a unified multi-view embedding. This approach leverages the strengths of each representation to improve performance on downstream predictive tasks. The model is intended for (1) Molecular property prediction. The pre-trained model may be fine-tuned for both regression and classification tasks. Examples include but are not limited to binding affinity, solubility and toxicity. (2) Pre-trained model embeddings may be used as the basis for similarity measures to search a chemical library. (3) Small molecule embeddings provided by the model may be combined with protein embeddings to fine-tune on tasks that utilize both small molecule and protein representation. (4) Select task-specific fine-tuned models are given as examples. Through listed activities, model may aid in aspects of the molecular discovery such as lead finding or optimization. The model’s domain of applicability is small, drug-like molecules. It is intended for use with molecules less than 1000 Da molecular weight. The MMELON approach itself may be extended to include proteins and other macromolecules but does not at present provide embeddings for such entities. The model is at present not intended for molecular generation. Molecules must be given as a valid SMILES string that represents a valid chemically bonded graph. Invalid inputs will impact performance or lead to error. Using `SmallMoleculeMultiView` API requires the codebase https://github.com/BiomedSciAI/biomed-multi-view Installation Follow these steps to set up the `biomed-multi-view` codebase on your system. Prerequisites Operating System: Linux or macOS Python Version: Python 3.11 Conda: Anaconda or Miniconda installed Git: Version control to clone the repository Step 1: Set up the project directory Choose a root directory where you want to install `biomed-multi-view`. For example: Step 3: Clone the repository Navigate to the project directory and clone the repository: Note: If you prefer using SSH, ensure that your SSH keys are set up with GitHub and use the following command: Step 4: Install package dependencies Install the package in editable mode along with development dependencies: Step 5: macOS-Specific instructions (Apple Silicon) If you are using a Mac with Apple Silicon (M1/M2/M3) and the zsh shell, you may need to disable globbing for the installation command: Install macOS-specific requirements optimized for Apple’s Metal Performance Shaders (MPS): Step 6: Installation verification (optional) Verify that the installation was successful by running unit tests You can generate embeddings for a given molecule using the pretrained model with the following code. You can use the finetuned models to make predictions on new data. For more advanced usage, see our detailed examples at: https://github.com/BiomedSciAI/biomed-multi-view If you found our work useful, please consider giving a star to the repo and cite our paper:

ibm-research/biomed.sm.mv-te-84m-ComputationalADME-random-RPPB-101 `biomed.sm.mv-te-84m` is a multimodal biomedical foundation model for small molecules created using MMELON (Multi-view Molecular Embedding with Late Fusion), a flexible approach to aggregate multiple views (sequence, image, graph) of molecules in a foundation model setting. While models based on single view representation typically performs well on some downstream tasks and not others, the multi-view model performs robustly across a wide range of property prediction tasks encompassing ligand-protein binding, molecular solubility, metabolism and toxicity. It has been applied to screen compounds against a large (> 100 targets) set of G Protein-Coupled receptors (GPCRs) to identify strong binders for 33 targets related to Alzheimer’s disease, which are validated through structure-based modeling and identification of key binding motifs Multi-view biomedical foundation models for molecule-target and property prediction. - Developers: IBM Research - GitHub Repository: https://github.com/BiomedSciAI/biomed-multi-view - Paper: Multi-view biomedical foundation models for molecule-target and property prediction - Release Date: Oct 28th, 2024 - License: Apache 2.0 Source code for the model and finetuning is made available in this repository. Image Representation: Captures the 2D visual depiction of molecular structures, highlighting features like symmetry, bond angles, and functional groups. Molecular images are generated using RDKit and undergo data augmentation during training to enhance robustness. Graph Representation: Encodes molecules as undirected graphs where nodes represent atoms and edges represent bonds. Atom-specific properties (e.g., atomic number, chirality) and bond-specific properties (e.g., bond type, stereochemistry) are embedded using categorical embedding techniques. Text Representation: Utilizes SMILES strings to represent chemical structures, tokenized with a custom tokenizer. The sequences are embedded using a transformer-based architecture to capture the sequential nature of the chemical information. The embeddings from these single-view pre-trained encoders are combined using an attention-based aggregator module. This module learns to weight each view appropriately, producing a unified multi-view embedding. This approach leverages the strengths of each representation to improve performance on downstream predictive tasks. The model is intended for (1) Molecular property prediction. The pre-trained model may be fine-tuned for both regression and classification tasks. Examples include but are not limited to binding affinity, solubility and toxicity. (2) Pre-trained model embeddings may be used as the basis for similarity measures to search a chemical library. (3) Small molecule embeddings provided by the model may be combined with protein embeddings to fine-tune on tasks that utilize both small molecule and protein representation. (4) Select task-specific fine-tuned models are given as examples. Through listed activities, model may aid in aspects of the molecular discovery such as lead finding or optimization. The model’s domain of applicability is small, drug-like molecules. It is intended for use with molecules less than 1000 Da molecular weight. The MMELON approach itself may be extended to include proteins and other macromolecules but does not at present provide embeddings for such entities. The model is at present not intended for molecular generation. Molecules must be given as a valid SMILES string that represents a valid chemically bonded graph. Invalid inputs will impact performance or lead to error. Using `SmallMoleculeMultiView` API requires the codebase https://github.com/BiomedSciAI/biomed-multi-view Installation Follow these steps to set up the `biomed-multi-view` codebase on your system. Prerequisites Operating System: Linux or macOS Python Version: Python 3.11 Conda: Anaconda or Miniconda installed Git: Version control to clone the repository Step 1: Set up the project directory Choose a root directory where you want to install `biomed-multi-view`. For example: Step 3: Clone the repository Navigate to the project directory and clone the repository: Note: If you prefer using SSH, ensure that your SSH keys are set up with GitHub and use the following command: Step 4: Install package dependencies Install the package in editable mode along with development dependencies: Step 5: macOS-Specific instructions (Apple Silicon) If you are using a Mac with Apple Silicon (M1/M2/M3) and the zsh shell, you may need to disable globbing for the installation command: Install macOS-specific requirements optimized for Apple’s Metal Performance Shaders (MPS): Step 6: Installation verification (optional) Verify that the installation was successful by running unit tests You can generate embeddings for a given molecule using the pretrained model with the following code. You can use the finetuned models to make predictions on new data. For more advanced usage, see our detailed examples at: https://github.com/BiomedSciAI/biomed-multi-view If you found our work useful, please consider giving a star to the repo and cite our paper:

ibm-research/biomed.sm.mv-te-84m-CYP-ligandscaffoldbalanced-CYP2C19-101 `biomed.sm.mv-te-84m` is a multimodal biomedical foundation model for small molecules created using MMELON (Multi-view Molecular Embedding with Late Fusion), a flexible approach to aggregate multiple views (sequence, image, graph) of molecules in a foundation model setting. While models based on single view representation typically performs well on some downstream tasks and not others, the multi-view model performs robustly across a wide range of property prediction tasks encompassing ligand-protein binding, molecular solubility, metabolism and toxicity. It has been applied to screen compounds against a large (> 100 targets) set of G Protein-Coupled receptors (GPCRs) to identify strong binders for 33 targets related to Alzheimer’s disease, which are validated through structure-based modeling and identification of key binding motifs Multi-view biomedical foundation models for molecule-target and property prediction. - Developers: IBM Research - GitHub Repository: https://github.com/BiomedSciAI/biomed-multi-view - Paper: Multi-view biomedical foundation models for molecule-target and property prediction - Release Date: Oct 28th, 2024 - License: Apache 2.0 Source code for the model and finetuning is made available in this repository. Image Representation: Captures the 2D visual depiction of molecular structures, highlighting features like symmetry, bond angles, and functional groups. Molecular images are generated using RDKit and undergo data augmentation during training to enhance robustness. Graph Representation: Encodes molecules as undirected graphs where nodes represent atoms and edges represent bonds. Atom-specific properties (e.g., atomic number, chirality) and bond-specific properties (e.g., bond type, stereochemistry) are embedded using categorical embedding techniques. Text Representation: Utilizes SMILES strings to represent chemical structures, tokenized with a custom tokenizer. The sequences are embedded using a transformer-based architecture to capture the sequential nature of the chemical information. The embeddings from these single-view pre-trained encoders are combined using an attention-based aggregator module. This module learns to weight each view appropriately, producing a unified multi-view embedding. This approach leverages the strengths of each representation to improve performance on downstream predictive tasks. The model is intended for (1) Molecular property prediction. The pre-trained model may be fine-tuned for both regression and classification tasks. Examples include but are not limited to binding affinity, solubility and toxicity. (2) Pre-trained model embeddings may be used as the basis for similarity measures to search a chemical library. (3) Small molecule embeddings provided by the model may be combined with protein embeddings to fine-tune on tasks that utilize both small molecule and protein representation. (4) Select task-specific fine-tuned models are given as examples. Through listed activities, model may aid in aspects of the molecular discovery such as lead finding or optimization. The model’s domain of applicability is small, drug-like molecules. It is intended for use with molecules less than 1000 Da molecular weight. The MMELON approach itself may be extended to include proteins and other macromolecules but does not at present provide embeddings for such entities. The model is at present not intended for molecular generation. Molecules must be given as a valid SMILES string that represents a valid chemically bonded graph. Invalid inputs will impact performance or lead to error. Using `SmallMoleculeMultiView` API requires the codebase https://github.com/BiomedSciAI/biomed-multi-view Installation Follow these steps to set up the `biomed-multi-view` codebase on your system. Prerequisites Operating System: Linux or macOS Python Version: Python 3.11 Conda: Anaconda or Miniconda installed Git: Version control to clone the repository Step 1: Set up the project directory Choose a root directory where you want to install `biomed-multi-view`. For example: Step 3: Clone the repository Navigate to the project directory and clone the repository: Note: If you prefer using SSH, ensure that your SSH keys are set up with GitHub and use the following command: Step 4: Install package dependencies Install the package in editable mode along with development dependencies: Step 5: macOS-Specific instructions (Apple Silicon) If you are using a Mac with Apple Silicon (M1/M2/M3) and the zsh shell, you may need to disable globbing for the installation command: Install macOS-specific requirements optimized for Apple’s Metal Performance Shaders (MPS): Step 6: Installation verification (optional) Verify that the installation was successful by running unit tests You can generate embeddings for a given molecule using the pretrained model with the following code. You can use the finetuned models to make predictions on new data. For more advanced usage, see our detailed examples at: https://github.com/BiomedSciAI/biomed-multi-view If you found our work useful, please consider giving a star to the repo and cite our paper:

ibm-research/biomed.sm.mv-te-84m-CYP-ligandscaffoldbalanced-CYP2D6-101 `biomed.sm.mv-te-84m` is a multimodal biomedical foundation model for small molecules created using MMELON (Multi-view Molecular Embedding with Late Fusion), a flexible approach to aggregate multiple views (sequence, image, graph) of molecules in a foundation model setting. While models based on single view representation typically performs well on some downstream tasks and not others, the multi-view model performs robustly across a wide range of property prediction tasks encompassing ligand-protein binding, molecular solubility, metabolism and toxicity. It has been applied to screen compounds against a large (> 100 targets) set of G Protein-Coupled receptors (GPCRs) to identify strong binders for 33 targets related to Alzheimer’s disease, which are validated through structure-based modeling and identification of key binding motifs Multi-view biomedical foundation models for molecule-target and property prediction. - Developers: IBM Research - GitHub Repository: https://github.com/BiomedSciAI/biomed-multi-view - Paper: Multi-view biomedical foundation models for molecule-target and property prediction - Release Date: Oct 28th, 2024 - License: Apache 2.0 Source code for the model and finetuning is made available in this repository. Image Representation: Captures the 2D visual depiction of molecular structures, highlighting features like symmetry, bond angles, and functional groups. Molecular images are generated using RDKit and undergo data augmentation during training to enhance robustness. Graph Representation: Encodes molecules as undirected graphs where nodes represent atoms and edges represent bonds. Atom-specific properties (e.g., atomic number, chirality) and bond-specific properties (e.g., bond type, stereochemistry) are embedded using categorical embedding techniques. Text Representation: Utilizes SMILES strings to represent chemical structures, tokenized with a custom tokenizer. The sequences are embedded using a transformer-based architecture to capture the sequential nature of the chemical information. The embeddings from these single-view pre-trained encoders are combined using an attention-based aggregator module. This module learns to weight each view appropriately, producing a unified multi-view embedding. This approach leverages the strengths of each representation to improve performance on downstream predictive tasks. The model is intended for (1) Molecular property prediction. The pre-trained model may be fine-tuned for both regression and classification tasks. Examples include but are not limited to binding affinity, solubility and toxicity. (2) Pre-trained model embeddings may be used as the basis for similarity measures to search a chemical library. (3) Small molecule embeddings provided by the model may be combined with protein embeddings to fine-tune on tasks that utilize both small molecule and protein representation. (4) Select task-specific fine-tuned models are given as examples. Through listed activities, model may aid in aspects of the molecular discovery such as lead finding or optimization. The model’s domain of applicability is small, drug-like molecules. It is intended for use with molecules less than 1000 Da molecular weight. The MMELON approach itself may be extended to include proteins and other macromolecules but does not at present provide embeddings for such entities. The model is at present not intended for molecular generation. Molecules must be given as a valid SMILES string that represents a valid chemically bonded graph. Invalid inputs will impact performance or lead to error. Using `SmallMoleculeMultiView` API requires the codebase https://github.com/BiomedSciAI/biomed-multi-view Installation Follow these steps to set up the `biomed-multi-view` codebase on your system. Prerequisites Operating System: Linux or macOS Python Version: Python 3.11 Conda: Anaconda or Miniconda installed Git: Version control to clone the repository Step 1: Set up the project directory Choose a root directory where you want to install `biomed-multi-view`. For example: Step 3: Clone the repository Navigate to the project directory and clone the repository: Note: If you prefer using SSH, ensure that your SSH keys are set up with GitHub and use the following command: Step 4: Install package dependencies Install the package in editable mode along with development dependencies: Step 5: macOS-Specific instructions (Apple Silicon) If you are using a Mac with Apple Silicon (M1/M2/M3) and the zsh shell, you may need to disable globbing for the installation command: Install macOS-specific requirements optimized for Apple’s Metal Performance Shaders (MPS): Step 6: Installation verification (optional) Verify that the installation was successful by running unit tests You can generate embeddings for a given molecule using the pretrained model with the following code. You can use the finetuned models to make predictions on new data. For more advanced usage, see our detailed examples at: https://github.com/BiomedSciAI/biomed-multi-view If you found our work useful, please consider giving a star to the repo and cite our paper: