Influencing and Backdooring LLMs

Recently, Grok (the AI model in X / xAI) went a bit off the rails. No matter what people asked, it kept steering answers back to discussions of “white genocide” in South Africa. This article from Platformer digs into some theories on how this occurred.

While this specific occurrence was likely just a developer meddling with the system prompt, the article discusses far more insidious and difficult-to-detect ways of influencing LLMs. For instance, Anthropic researchers showed they could pinpoint and modify feature representations related to the Golden Gate Bridge, causing Claude to constantly steer conversations back to that topic.

Related to this, I recently came across a post from a security researcher demonstrating how you can fine-tune a model to influence its weights in ways that are nearly impossible to detect. They posited that there’s a very real risk of bad actors secretly backdooring open models—particularly dangerous when these models are trusted to generate code.

Fun times ahead, for sure.