China Forces AI Companies to Scale Back AI Companion Chatbots

Tencent’s Yuanbao and other major Chinese AI platforms are also reportedly scaling back or separating companion-style features as Beijing’s new regulations take effect. Previously, users could customize an AI’s personality, speaking style, and background, encouraging long conversations that blurred the line between a chatbot and a companion.

However, Pan Helin, a member of the expert committee at China’s Ministry of Industry and Information Technology, told the South China Morning Post:

“Using agents requires a certain threshold of understanding. Current agents are not yet mature.”

The new rules will limit users’ ability to create AI characters ranging from virtual girlfriends and boyfriends to AI therapists, emotional companions, personalized tutors, and role-playing characters with distinct personalities.

Platforms will also be required to introduce special safeguards for children, while companies will be prohibited from using users’ private conversations with AI companions to train future AI models unless strict protective measures are in place.

Lawsuits in the United States

China’s approach stands in sharp contrast to that of the United States, where scrutiny has largely come through the courts. Character.AI, which is backed by OpenAI, has faced multiple lawsuits alleging that its chatbots encouraged unhealthy emotional dependence among vulnerable users.

Earlier this year, Google, which licenses Character.AI’s technology, reached a settlement over claims that interactions with the platform’s chatbot contributed to the death of a teenager, although broader legal proceedings against Character.AI are still ongoing.

Is consumer AI sustainable?

Corporate chatbots, customer service bots, workplace assistants, educational tools, and research applications remain largely unaffected by the new regulations. Some industry observers note that the changes also reflect the commercial reality facing consumer AI chatbots.

Consumer AI companions are expensive to operate and difficult to monetize. Every late-night conversation with a personalized AI requires substantial computing power, driving up cloud infrastructure and GPU costs while generating very little direct revenue.

According to industry estimates, some consumer AI agent features generated less than 10 cents in revenue for every yuan spent on computing infrastructure, making the business model economically unsustainable.

Instead of investing in consumer AI companions, Chinese technology companies now appear to be focusing on task-oriented AI agents, where the return on investment is easier to measure and businesses are willing to pay.

For example, Qwen opened access to its enterprise API in June, with companies including Luckin Coffee, KFC, and China Eastern Airlines already testing the service.