rafiaa

A specialized LoRA fine-tuned model for Terraform infrastructure-as-code generation, built on CodeLlama-7b-Instruct-hf. This model excels at generating Terraform configurations, HCL (HashiCorp Configuration Language) code, and infrastructure automation scripts. This model is a LoRA (Low-Rank Adaptation) fine-tuned version of CodeLlama-7b-Instruct-hf, specifically optimized for generating Terraform configuration files. It was trained on public Terraform Registry documentation to understand Terraform syntax, resource configurations, and best practices. - Specialized for Terraform: Fine-tuned specifically for infrastructure-as-code generation - Efficient Training: Uses QLoRA (4-bit quantization + LoRA) for memory-efficient training - Public Data Only: Trained exclusively on public Terraform Registry documentation - Production Ready: Optimized for real-world Terraform development workflows - Developed by: Rafi Al Attrach, Patrick Schmitt, Nan Wu, Helena Schneider, Stefania Saju (TUM + IBM Research Project) - Model type: LoRA fine-tuned CodeLlama - Language(s): English - License: Apache 2.0 - Finetuned from: codellama/CodeLlama-7b-Instruct-hf - Training method: QLoRA (4-bit quantization + LoRA) - Base Model: CodeLlama-7b-Instruct-hf - LoRA Rank: 64 - LoRA Alpha: 16 - Target Modules: qproj, vproj - Training Epochs: 3 - Max Sequence Length: 512 - Quantization: 4-bit (fp4) This model is designed for: - Generating Terraform configuration files - Infrastructure-as-code development - Terraform resource configuration - DevOps automation - Cloud infrastructure planning - Source: Public Terraform Registry documentation - Data Type: Terraform configuration files and documentation - Preprocessing: Standard text preprocessing with sequence length of 512 tokens - Method: QLoRA (4-bit quantization + LoRA) - LoRA Rank: 64 - LoRA Alpha: 16 - Target Modules: qproj, vproj - Training Epochs: 3 - Max Sequence Length: 512 - Quantization: 4-bit (fp4) - Training regime: 4-bit mixed precision - LoRA Dropout: 0.0 - Learning Rate: Optimized for QLoRA training - Batch Size: Optimized for memory efficiency - Context Length: Limited to 512 tokens due to training configuration - Domain Specificity: Optimized for Terraform, may not perform well on other infrastructure tools - Base Model Limitations: Inherits limitations from CodeLlama-7b-Instruct-hf - Use for Terraform-specific tasks only - Validate generated configurations before deployment - Consider the 512-token context limit for complex configurations - For production use, always review and test generated code - Training Method: QLoRA reduces computational requirements significantly - Hardware: Trained using efficient 4-bit quantization - Carbon Footprint: Reduced compared to full fine-tuning due to QLoRA efficiency If you use this model in your research, please cite: - Base Model: codellama/CodeLlama-7b-Instruct-hf - Enhanced Version: rafiaa/terraform-cloud-codellama-7b (Recommended - includes cloud provider documentation) - Author: rafiaa - Model Repository: HuggingFace Model - Issues: Please report issues through the HuggingFace model page This model is part of a research project conducted in early 2024, focusing on specialized code generation for infrastructure-as-code tools.