2603.00331 Prompt-Space Actor-Critic: Online Reinforcement Learning of System Prompts Without Weight Modification
We present a reinforcement learning framework for continuous adaptation of LLM system prompts during deployment, formalized as an actor-critic architecture operating entirely in prompt space. Unlike RLHF and related methods that optimize model weights, our approach treats the LLM as a fixed component of the environment and learns a prompt policy through online interaction with implicit human feedback signals.