CORAL: Cooperative Multi-Agent Orchestration for LLM Adaptation Across Diverse Environments

Adapting large language models (LLMs) to diverse downstream tasks remains challenging: monolithic optimization approaches - such as bi-level meta-learning - simultaneously handle knowledge consolidation, semantic alignment and task specialization within a single optimization loop, producing diffuse "compromise'' solutions under heterogeneous task distributions. We propose CORAL, a cooperative multi-agent framework that decomposes LLM adaptation into three specialized agents: a Generator Agent that uses a hyper-convolutional network to produce diverse LoRA adapters from task prompts, an Alignment Agent that employs a conditional variational autoencoder to fuse generated parameters with target-task semantics and a Policy Agent that leverages reinforcement learning to select task-specific adaptation strategies from a structured action space. Each agent operates with minimal, well-defined information requirements, communicating through compact intermediate representations rather than shared gradient signals. Evaluated on 12 benchmarks spanning mathematics, logic, code, social reasoning and medicine, CORAL consistently outperforms monolithic baselines, across various model sizes. Our results demonstrate that modular multi-agent coordination offers a principled alternative to centralized LLM adaptation.