Transformer^2: Self-Adaptive LLMs(sakana.ai)

    - Fine-tuning LLMs for every new task is slow, expensive, and often doesn't generalize well.
    - Models trained on one task may perform poorly on others, especially unseen ones.
    - Current methods (like LoRA) can add new capabilities but aren't efficient enough.

    - Transformer² uses a new fine-tuning method called Singular Value Fine-tuning (SVF). This focuses on adjusting only certain parts of the model’s "weight matrices" rather than changing everything.
    - By tweaking specific components (called "singular values"), it trains smaller, efficient "expert" modules that specialize in particular types of tasks.

    - Training phase: Train these smaller expert modules offline using reinforcement learning (RL) to specialize in tasks like coding, math, or reasoning.
    - Inference phase: When a new input is given, the system analyzes the task (e.g., “Is this a math or coding problem?”) in the first pass. Based on this, it combines the right expert modules and adapts the model’s behavior in the second pass.

    - Prompt-based: Use a cleverly designed text prompt to figure out the task type and pick the right expert module.
    - Classifier-based: Train a separate model to classify tasks and match them to experts.
    - Few-shot adaptation: Look at a small number of examples (few-shot learning) to dynamically combine expert modules for the best results.

    - The system uses fewer parameters than traditional fine-tuning methods like LoRA.
    - Adaptation works even on small datasets without overfitting or forgetting older tasks.