# Tutorial 10: LoRA Adapters - Multi-Domain Inference ## Table of Contents 1. [Introduction](#introduction) 2. [Why Use LoRA Adapters?](#why-use-lora-adapters) 3. [Training Your First Adapter](#training-your-first-adapter) 4. [Training Multiple Domain Adapters](#training-multiple-domain-adapters) 5. [Loading and Swapping Adapters](#loading-and-swapping-adapters) 6. [Real-World Use Cases](#real-world-use-cases) 7. [Best Practices](#best-practices) 8. [Troubleshooting](#troubleshooting) ## Introduction LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique that allows you to train specialized adapters for different domains without modifying the base model. This enables: - **Fast domain switching**: Swap between domains in milliseconds - **Minimal storage**: Adapters are ~2-10 MB vs ~100-500 MB for full models - **Domain specialization**: Train separate adapters for legal, medical, financial, etc. - **Easy deployment**: Keep one base model + multiple lightweight adapters ## Why Use LoRA Adapters? ### Memory Efficiency ``` Full Model Fine-tuning: - Legal model: 450 MB - Medical model: 450 MB - Financial model: 450 MB Total: 1.35 GB LoRA Adapters: - Base model: 450 MB - Legal adapter: 5 MB - Medical adapter: 5 MB - Financial adapter: 5 MB Total: 465 MB (65% less!) ``` ### Fast Training LoRA adapters train **2-3x faster** than full fine-tuning because: - Only ~1-5% of parameters are trainable - Smaller gradient computations - Less GPU memory required ### Easy Multi-Domain Inference ```python # One base model, multiple domains model = GLiNER2.from_pretrained("fastino/gliner2-base-v1") # Legal domain model.load_adapter("./legal_adapter") legal_results = model.extract_entities(legal_text, ["company", "law"]) # Medical domain (swap in <1 second) model.load_adapter("./medical_adapter") medical_results = model.extract_entities(medical_text, ["disease", "drug"]) ``` ## Training Your First Adapter ### Step 1: Prepare Domain-Specific Data ```python from gliner2.training.data import InputExample # Legal domain examples legal_examples = [ InputExample( text="Apple Inc. filed a lawsuit against Samsung Electronics.", entities={"company": ["Apple Inc.", "Samsung Electronics"]} ), InputExample( text="The plaintiff Google LLC accused Microsoft Corporation of patent infringement.", entities={"company": ["Google LLC", "Microsoft Corporation"]} ), InputExample( text="Tesla Motors settled the case with the Securities and Exchange Commission.", entities={ "company": ["Tesla Motors"], "organization": ["Securities and Exchange Commission"] } ), # Add 100-1000+ examples for best results ] ``` ### Step 2: Configure LoRA Training ```python from gliner2 import GLiNER2 from gliner2.training.trainer import GLiNER2Trainer, TrainingConfig # LoRA configuration config = TrainingConfig( output_dir="./legal_adapter", experiment_name="legal_domain", # Training parameters num_epochs=10, batch_size=8, gradient_accumulation_steps=2, encoder_lr=1e-5, task_lr=5e-4, # LoRA settings use_lora=True, # Enable LoRA lora_r=8, # Rank (4, 8, 16, 32) lora_alpha=16.0, # Scaling factor (usually 2*r) lora_dropout=0.0, # Dropout for LoRA layers lora_target_modules=["encoder"], # Apply to all encoder layers (query, key, value, dense) save_adapter_only=True, # Save only adapter (not full model) # Optimization eval_strategy="epoch", # Evaluates and saves at end of each epoch eval_steps=500, # Used when eval_strategy="steps" logging_steps=50, fp16=True, # Use mixed precision if GPU available ) ``` ### Step 3: Train the Adapter ```python # Load base model base_model = GLiNER2.from_pretrained("fastino/gliner2-base-v1") # Create trainer trainer = GLiNER2Trainer(model=base_model, config=config) # Train adapter trainer.train(train_data=legal_examples) # Adapter automatically saved to ./legal_adapter/final/ ``` **Training output:** ``` šŸ”§ LoRA Configuration ====================================================================== Enabled : True Rank (r) : 8 Alpha : 16.0 Scaling (Ī±/r) : 2.0000 Dropout : 0.0 Target modules : query, key, value, dense LoRA layers : 144 ---------------------------------------------------------------------- Trainable params : 1,327,104 / 124,442,368 (1.07%) Memory savings : ~98.9% fewer gradients ====================================================================== ***** Running Training ***** Num examples = 1000 Num epochs = 10 Batch size = 8 Effective batch size = 16 Total optimization steps = 625 LoRA enabled: 1,327,104 trainable / 124,442,368 total (1.07%) ``` ## Training Multiple Domain Adapters Let's train adapters for three different domains: **Legal**, **Medical**, and **Customer Support**. ### Complete Multi-Domain Training Script ```python from gliner2 import GLiNER2 from gliner2.training.trainer import GLiNER2Trainer, TrainingConfig from gliner2.training.data import InputExample # ============================================================================ # Define Domain Data # ============================================================================ # Legal domain legal_examples = [ InputExample( text="Apple Inc. filed a lawsuit against Samsung Electronics.", entities={"company": ["Apple Inc.", "Samsung Electronics"]} ), InputExample( text="The plaintiff Google LLC accused Microsoft Corporation of patent infringement.", entities={"company": ["Google LLC", "Microsoft Corporation"]} ), # Add more examples... ] # Medical domain medical_examples = [ InputExample( text="Patient diagnosed with Type 2 Diabetes and Hypertension.", entities={"disease": ["Type 2 Diabetes", "Hypertension"]} ), InputExample( text="Prescribed Metformin 500mg twice daily and Lisinopril 10mg once daily.", entities={ "drug": ["Metformin", "Lisinopril"], "dosage": ["500mg", "10mg"] } ), # Add more examples... ] # Customer support domain support_examples = [ InputExample( text="Customer John Smith reported issue with Order #12345.", entities={ "customer": ["John Smith"], "order_id": ["Order #12345"] } ), InputExample( text="Refund of $99.99 processed for Order #98765 on 2024-01-15.", entities={ "order_id": ["Order #98765"], "amount": ["$99.99"], "date": ["2024-01-15"] } ), # Add more examples... ] # ============================================================================ # Training Function # ============================================================================ def train_domain_adapter( base_model_name: str, examples: list, domain_name: str, output_dir: str = "./adapters" ): """Train a LoRA adapter for a specific domain.""" adapter_path = f"{output_dir}/{domain_name}_adapter" config = TrainingConfig( output_dir=adapter_path, experiment_name=f"{domain_name}_domain", # Training num_epochs=10, batch_size=8, gradient_accumulation_steps=2, encoder_lr=1e-5, task_lr=5e-4, # LoRA use_lora=True, lora_r=8, lora_alpha=16.0, lora_dropout=0.0, lora_target_modules=["encoder"], # All encoder layers save_adapter_only=True, # Logging & Checkpointing eval_strategy="no", # Set to "epoch" or "steps" if you have validation set eval_steps=500, # Used when eval_strategy="steps" logging_steps=50, fp16=True, ) # Load base model print(f"\n{'='*60}") print(f"Training {domain_name.upper()} adapter") print(f"{'='*60}") model = GLiNER2.from_pretrained(base_model_name) trainer = GLiNER2Trainer(model=model, config=config) # Train results = trainer.train(train_data=examples) print(f"\nāœ… {domain_name.capitalize()} adapter trained!") print(f"šŸ“ Saved to: {adapter_path}/final/") print(f"ā±ļø Training time: {results['total_time_seconds']:.2f}s") return f"{adapter_path}/final" # ============================================================================ # Train All Adapters # ============================================================================ if __name__ == "__main__": BASE_MODEL = "fastino/gliner2-base-v1" # Train adapters for each domain legal_adapter_path = train_domain_adapter( BASE_MODEL, legal_examples, "legal" ) medical_adapter_path = train_domain_adapter( BASE_MODEL, medical_examples, "medical" ) support_adapter_path = train_domain_adapter( BASE_MODEL, support_examples, "support" ) print("\n" + "="*60) print("šŸŽ‰ All adapters trained successfully!") print("="*60) print(f"Legal adapter: {legal_adapter_path}") print(f"Medical adapter: {medical_adapter_path}") print(f"Support adapter: {support_adapter_path}") ``` ## Loading and Swapping Adapters ### Basic Usage ```python from gliner2 import GLiNER2 # Load base model once model = GLiNER2.from_pretrained("fastino/gliner2-base-v1") # Load legal adapter model.load_adapter("./adapters/legal_adapter/final") # Use the model result = model.extract_entities( "Apple Inc. sued Samsung over patent rights.", ["company", "legal_action"] ) print(result) ``` ### Swapping Between Adapters ```python # Load base model model = GLiNER2.from_pretrained("fastino/gliner2-base-v1") # Legal domain print("šŸ“‹ Legal Analysis:") model.load_adapter("./adapters/legal_adapter/final") legal_text = "Google LLC filed a complaint against Oracle Corporation." legal_result = model.extract_entities(legal_text, ["company", "legal_action"]) print(f" {legal_result}") # Swap to medical domain print("\nšŸ„ Medical Analysis:") model.load_adapter("./adapters/medical_adapter/final") medical_text = "Patient presents with Pneumonia and was prescribed Amoxicillin." medical_result = model.extract_entities(medical_text, ["disease", "drug"]) print(f" {medical_result}") # Swap to support domain print("\nšŸ’¬ Support Analysis:") model.load_adapter("./adapters/support_adapter/final") support_text = "Customer reported Order #12345 not delivered on time." support_result = model.extract_entities(support_text, ["order_id", "issue"]) print(f" {support_result}") # Use base model without adapter print("\nšŸ”§ Base Model (no adapter):") model.unload_adapter() base_result = model.extract_entities("Some generic text", ["entity"]) print(f" {base_result}") ``` **Output:** ``` šŸ“‹ Legal Analysis: {'entities': [{'text': 'Google LLC', 'label': 'company', ...}, {'text': 'Oracle Corporation', 'label': 'company', ...}]} šŸ„ Medical Analysis: {'entities': [{'text': 'Pneumonia', 'label': 'disease', ...}, {'text': 'Amoxicillin', 'label': 'drug', ...}]} šŸ’¬ Support Analysis: {'entities': [{'text': 'Order #12345', 'label': 'order_id', ...}]} šŸ”§ Base Model (no adapter): {'entities': [{'text': 'text', 'label': 'entity', ...}]} ``` ### Batch Processing with Adapter Swapping ```python def process_documents_by_domain(model, documents_by_domain, adapters): """ Process multiple documents across different domains efficiently. Args: model: Base GLiNER2 model documents_by_domain: Dict[domain_name, List[document_text]] adapters: Dict[domain_name, adapter_path] Returns: Dict[domain_name, List[results]] """ results = {} for domain, documents in documents_by_domain.items(): print(f"Processing {domain} domain ({len(documents)} documents)...") # Load domain-specific adapter model.load_adapter(adapters[domain]) # Process all documents for this domain domain_results = [] for doc in documents: result = model.extract_entities(doc, get_entity_types(domain)) domain_results.append(result) results[domain] = domain_results return results def get_entity_types(domain): """Get entity types for each domain.""" types = { "legal": ["company", "person", "law", "legal_action"], "medical": ["disease", "drug", "symptom", "procedure"], "support": ["customer", "order_id", "product", "issue"] } return types.get(domain, ["entity"]) # Example usage model = GLiNER2.from_pretrained("fastino/gliner2-base-v1") documents_by_domain = { "legal": [ "Apple Inc. filed suit against Samsung.", "Microsoft acquired LinkedIn for $26B.", ], "medical": [ "Patient has Type 2 Diabetes.", "Prescribed Metformin 500mg daily.", ], "support": [ "Issue with Order #12345 reported.", "Refund processed for Order #98765.", ] } adapters = { "legal": "./adapters/legal_adapter/final", "medical": "./adapters/medical_adapter/final", "support": "./adapters/support_adapter/final", } results = process_documents_by_domain(model, documents_by_domain, adapters) # Results organized by domain for domain, domain_results in results.items(): print(f"\n{domain.upper()} Results:") for i, result in enumerate(domain_results, 1): print(f" Document {i}: {len(result['entities'])} entities found") ``` ## Real-World Use Cases ### Use Case 1: Multi-Tenant SaaS Platform ```python class MultiTenantEntityExtractor: """Entity extraction service for multi-tenant platform.""" def __init__(self, base_model_name: str, tenant_adapters: dict): """ Args: base_model_name: Path to base model tenant_adapters: Dict mapping tenant_id to adapter_path """ self.model = GLiNER2.from_pretrained(base_model_name) self.tenant_adapters = tenant_adapters self.current_tenant = None def extract_for_tenant(self, tenant_id: str, text: str, entity_types: list): """Extract entities for specific tenant.""" # Load tenant-specific adapter if needed if self.current_tenant != tenant_id: adapter_path = self.tenant_adapters.get(tenant_id) if adapter_path: self.model.load_adapter(adapter_path) else: self.model.unload_adapter() # Use base model self.current_tenant = tenant_id return self.model.extract_entities(text, entity_types) # Setup extractor = MultiTenantEntityExtractor( base_model_name="fastino/gliner2-base-v1", tenant_adapters={ "legal_firm_123": "./adapters/legal_adapter/final", "hospital_456": "./adapters/medical_adapter/final", "ecommerce_789": "./adapters/support_adapter/final", } ) # Usage legal_result = extractor.extract_for_tenant( "legal_firm_123", "Apple sued Samsung", ["company"] ) medical_result = extractor.extract_for_tenant( "hospital_456", "Patient has diabetes", ["disease"] ) ``` ### Use Case 2: Document Classification Pipeline ```python def classify_and_extract(document: str, model: GLiNER2, adapters: dict): """ Classify document type and extract relevant entities. 1. Classify document type using base model 2. Load appropriate domain adapter 3. Extract domain-specific entities """ # Step 1: Classify document type doc_type_result = model.extract_entities( document, ["legal_document", "medical_record", "support_ticket", "financial_report"] ) # Determine document type if doc_type_result['entities']: doc_type = doc_type_result['entities'][0]['label'] doc_type = doc_type.replace("_document", "").replace("_record", "").replace("_ticket", "").replace("_report", "") else: doc_type = "general" # Step 2: Load appropriate adapter adapter_mapping = { "legal": adapters.get("legal"), "medical": adapters.get("medical"), "support": adapters.get("support"), "financial": adapters.get("financial"), } if doc_type in adapter_mapping and adapter_mapping[doc_type]: model.load_adapter(adapter_mapping[doc_type]) # Step 3: Extract domain-specific entities entity_types = { "legal": ["company", "person", "law", "legal_action"], "medical": ["disease", "drug", "symptom", "procedure", "dosage"], "support": ["customer", "order_id", "product", "issue", "status"], "financial": ["company", "amount", "date", "stock_symbol"], } entities = model.extract_entities( document, entity_types.get(doc_type, ["entity"]) ) return { "document_type": doc_type, "entities": entities['entities'] } # Usage model = GLiNER2.from_pretrained("fastino/gliner2-base-v1") adapters = { "legal": "./adapters/legal_adapter/final", "medical": "./adapters/medical_adapter/final", "support": "./adapters/support_adapter/final", } document = "Patient John Smith diagnosed with Type 2 Diabetes on 2024-01-15." result = classify_and_extract(document, model, adapters) print(f"Document Type: {result['document_type']}") print(f"Entities: {result['entities']}") ``` ### Use Case 3: A/B Testing Adapters ```python import random class AdapterABTester: """A/B test different adapter versions.""" def __init__(self, base_model_name: str, adapter_variants: dict): """ Args: adapter_variants: {"v1": path1, "v2": path2, ...} """ self.model = GLiNER2.from_pretrained(base_model_name) self.adapter_variants = adapter_variants self.results = {variant: [] for variant in adapter_variants} def test_sample(self, text: str, entity_types: list, true_entities: list): """Test a sample with all adapter variants.""" sample_results = {} for variant, adapter_path in self.adapter_variants.items(): # Load variant self.model.load_adapter(adapter_path) # Get predictions pred = self.model.extract_entities(text, entity_types) # Compute metrics f1 = self.compute_f1(pred['entities'], true_entities) sample_results[variant] = { "predictions": pred['entities'], "f1_score": f1 } self.results[variant].append(f1) return sample_results def compute_f1(self, predicted, ground_truth): """Simple F1 computation (simplified for demo).""" pred_set = {(e['text'], e['label']) for e in predicted} true_set = {(e['text'], e['label']) for e in ground_truth} if not pred_set and not true_set: return 1.0 if not pred_set or not true_set: return 0.0 tp = len(pred_set & true_set) precision = tp / len(pred_set) if pred_set else 0 recall = tp / len(true_set) if true_set else 0 if precision + recall == 0: return 0.0 return 2 * precision * recall / (precision + recall) def get_summary(self): """Get A/B test summary.""" summary = {} for variant, scores in self.results.items(): if scores: summary[variant] = { "avg_f1": sum(scores) / len(scores), "samples": len(scores) } return summary # Usage tester = AdapterABTester( base_model_name="fastino/gliner2-base-v1", adapter_variants={ "v1_r4": "./adapters/legal_v1_r4/final", "v2_r8": "./adapters/legal_v2_r8/final", "v3_r16": "./adapters/legal_v3_r16/final", } ) # Test samples test_samples = [ { "text": "Apple Inc. sued Samsung Electronics.", "entity_types": ["company"], "true_entities": [ {"text": "Apple Inc.", "label": "company"}, {"text": "Samsung Electronics", "label": "company"} ] }, # More samples... ] for sample in test_samples: results = tester.test_sample( sample["text"], sample["entity_types"], sample["true_entities"] ) # Get summary summary = tester.get_summary() for variant, metrics in summary.items(): print(f"{variant}: Avg F1 = {metrics['avg_f1']:.3f} ({metrics['samples']} samples)") ``` ## Best Practices ### 1. Choosing LoRA Hyperparameters ```python # Small datasets (< 1K examples) config = TrainingConfig( lora_r=4, # Lower rank = fewer parameters lora_alpha=8.0, # alpha = 2 * r num_epochs=10, ) # Medium datasets (1K-10K examples) config = TrainingConfig( lora_r=8, # Standard rank lora_alpha=16.0, num_epochs=5, ) # Large datasets (> 10K examples) config = TrainingConfig( lora_r=16, # Higher rank = more capacity lora_alpha=32.0, num_epochs=3, ) ``` ### 2. Target Module Selection **Understanding Module Groups:** GLiNER2 supports fine-grained control over which layers receive LoRA adaptation: ```python # Option 1: Encoder only - all layers (query, key, value, dense) # Use case: General domain adaptation, good starting point # Memory: Moderate (~1-2% of model parameters) lora_target_modules=["encoder"] # Option 2: Encoder - attention layers only # Use case: Very memory-constrained scenarios # Memory: Low (~0.5-1% of model parameters) lora_target_modules=["encoder.query", "encoder.key", "encoder.value"] # Option 3: Encoder - FFN layers only # Use case: Alternative to attention-only, sometimes better for certain tasks # Memory: Low (~0.5-1% of model parameters) lora_target_modules=["encoder.dense"] # Option 4: Encoder + task heads # Use case: When you want to adapt both representation and task-specific layers # Memory: Moderate-High (~2-4% of model parameters) lora_target_modules=["encoder", "span_rep", "classifier"] # Option 5: All modules (DEFAULT) # Use case: Maximum adaptation capacity, best performance # Memory: High (~3-5% of model parameters) lora_target_modules=["encoder", "span_rep", "classifier", "count_embed", "count_pred"] ``` **Recommendations:** - **Start with encoder only** (`["encoder"]`) for most tasks - **Add task heads** if performance is insufficient - **Use attention-only** for extreme memory constraints - **Use all modules** (default) when you need maximum performance ### 3. Adapter Organization ``` project/ ā”œā”€ā”€ base_model/ ā”‚ ā””ā”€ā”€ gliner2-base-v1/ ā”œā”€ā”€ adapters/ ā”‚ ā”œā”€ā”€ legal/ ā”‚ ā”‚ ā”œā”€ā”€ v1_r8/ ā”‚ ā”‚ ā”‚ ā””ā”€ā”€ final/ ā”‚ ā”‚ ā””ā”€ā”€ v2_r16/ ā”‚ ā”‚ ā””ā”€ā”€ final/ ā”‚ ā”œā”€ā”€ medical/ ā”‚ ā”‚ ā””ā”€ā”€ final/ ā”‚ ā””ā”€ā”€ support/ ā”‚ ā””ā”€ā”€ final/ ā””ā”€ā”€ scripts/ ā”œā”€ā”€ train_adapters.py ā””ā”€ā”€ evaluate_adapters.py ``` ### 4. Version Control for Adapters ```python # adapter_metadata.json { "legal_v1": { "path": "./adapters/legal/v1_r8/final", "base_model": "fastino/gliner2-base-v1", "lora_r": 8, "lora_alpha": 16.0, "trained_on": "2024-01-15", "training_samples": 5000, "eval_f1": 0.87, "notes": "Initial legal domain adapter" }, "legal_v2": { "path": "./adapters/legal/v2_r16/final", "base_model": "fastino/gliner2-base-v1", "lora_r": 16, "lora_alpha": 32.0, "trained_on": "2024-02-01", "training_samples": 10000, "eval_f1": 0.92, "notes": "Improved with more data and higher rank" } } ``` ### 5. Monitoring Adapter Performance ```python def evaluate_adapter(model, adapter_path, test_data): """Evaluate adapter performance on test data.""" model.load_adapter(adapter_path) results = { "total": 0, "correct": 0, "precision_sum": 0, "recall_sum": 0, } for sample in test_data: pred = model.extract_entities(sample["text"], sample["entity_types"]) # Compute metrics metrics = compute_metrics(pred['entities'], sample["true_entities"]) results["total"] += 1 results["precision_sum"] += metrics["precision"] results["recall_sum"] += metrics["recall"] avg_precision = results["precision_sum"] / results["total"] avg_recall = results["recall_sum"] / results["total"] f1 = 2 * avg_precision * avg_recall / (avg_precision + avg_recall) return { "precision": avg_precision, "recall": avg_recall, "f1": f1, "samples": results["total"] } ``` ## Troubleshooting ### Issue 1: Adapter Not Affecting Predictions **Symptom**: Predictions are the same with and without adapter. **Solution**: ```python # Check if adapter is actually loaded print(f"Has adapter: {model.has_adapter}") # Check LoRA layers from gliner2.training.lora import LoRALayer lora_count = sum(1 for m in model.modules() if isinstance(m, LoRALayer)) print(f"LoRA layers: {lora_count}") # Should be > 0 if adapter is loaded assert lora_count > 0, "No LoRA layers found!" ``` ### Issue 2: Out of Memory During Training **Solution**: ```python config = TrainingConfig( # Reduce batch size batch_size=4, # Instead of 8 gradient_accumulation_steps=4, # Maintain effective batch size # Use smaller LoRA rank lora_r=4, # Instead of 8 # Enable mixed precision fp16=True, # Target only attention layers (fewer parameters) lora_target_modules=["encoder.query", "encoder.key", "encoder.value"], ) ``` ### Issue 3: Adapter File Not Found **Solution**: ```python import os from gliner2.training.lora import LoRAAdapterConfig adapter_path = "./adapters/legal_adapter/final" # Check if path exists if not os.path.exists(adapter_path): print(f"Path does not exist: {adapter_path}") # List available checkpoints checkpoint_dir = "./adapters/legal_adapter" if os.path.exists(checkpoint_dir): checkpoints = os.listdir(checkpoint_dir) print(f"Available checkpoints: {checkpoints}") # Check if it's a valid adapter if LoRAAdapterConfig.is_adapter_path(adapter_path): print("Valid adapter path!") config = LoRAAdapterConfig.load(adapter_path) print(f"Adapter config: {config}") else: print("Not a valid adapter path!") ``` ### Issue 4: Slow Adapter Switching **Problem**: Switching between adapters takes too long. **Solution**: ```python # Pre-load adapters in memory (if you have enough RAM) adapters = {} for domain, path in adapter_paths.items(): # Load adapter weights into memory adapters[domain] = load_adapter_to_memory(path) # Fast switching from memory (not implemented in base API, # but possible with custom caching layer) ``` ## Summary ### Key Takeaways āœ… **LoRA adapters** enable efficient multi-domain inference āœ… **Training** is 2-3x faster than full fine-tuning āœ… **Storage** savings of 65-95% compared to multiple full models āœ… **Swapping** adapters takes < 1 second āœ… **Domain specialization** improves accuracy on specific tasks ### Quick Reference ```python # Training config = TrainingConfig( use_lora=True, lora_r=8, lora_alpha=16.0, save_adapter_only=True, ) trainer.train(train_data=examples) # Loading model = GLiNER2.from_pretrained("base-model") model.load_adapter("./adapter/final") # Swapping model.load_adapter("./other_adapter/final") # Unloading model.unload_adapter() # Checking print(model.has_adapter) print(model.adapter_config) ``` ### Next Steps 1. **Train your first adapter** with domain-specific data 2. **Evaluate performance** on test set 3. **Experiment with hyperparameters** (rank, alpha, target modules) 4. **Deploy multiple adapters** for different use cases 5. **Monitor and iterate** based on real-world performance For more information: - LoRA Paper: https://arxiv.org/abs/2106.09685 - Implementation: `gliner2/training/lora.py` - Tests: `tests/test_lora_adapters.py` - Verification Guide: `LORA_VERIFICATION_TESTS.md`