AI Hallucinations Open a New Path for Hackers Through Fake Websites

01.07.2026 5 minutes Author: Newsman

Large language models continue to generate web addresses that don’t actually exist. Attackers have started registering these AI-invented domains before anyone else and using them to host phishing pages that capture traffic sent by AI-powered tools.

Unit 42, the threat intelligence team at Palo Alto Networks, has uncovered a new attack technique called Phantom Squatting. The method is already being used by cybercriminals and takes advantage of large language models generating domain names that do not actually exist.

The attack works because users and AI assistants are increasingly trusting links suggested by AI. If a model generates a website address that has never been registered, an attacker can simply purchase that domain and capture traffic from anyone who follows the AI’s recommendation. In this scenario, there is no need for phishing emails or malicious ads. The AI effectively directs victims to the fake website on its own.

To measure the scale of the problem, Unit 42 researchers asked two large language models 685,339 questions related to 913 well-known brands across technology, finance, healthcare, government, gambling, and other industries.

The models generated more than 2.1 million URLs. Analysis revealed that 13,229 of them were already known to be malicious. Researchers also identified around 250,000 AI-generated domains that were still available for registration, making them attractive targets for attackers.

Phantom Squatting is effective for a simple reason. A newly registered domain has no reputation, meaning security systems have no reason to flag it as malicious. Blocklists, threat intelligence feeds, and reputation services only react after a website has already demonstrated malicious behavior. Until then, users or AI agents may freely visit a site that the AI itself presented as legitimate.

Researchers highlighted two additional factors that make the technique even more dangerous. First, these fake domains were not present in the models’ training data. Instead of recalling existing websites, the AI created them from linguistic patterns. Second, different models often generated the same nonexistent domains for identical prompts, making it much easier for attackers to predict which domains they should register.

Increasing the model’s creativity setting only produced even more fabricated domains. As Unit 42 researchers noted, this attack vector “leverages a structural property of LLM architectures that is inherently difficult to eliminate.”

During the investigation, researchers documented two real-world Phantom Squatting attacks.

The first occurred in March 2026. On March 8, Unit 42 predicted that AI models would consistently generate a domain resembling the online portal of a national postal service. Both language models produced the same fake domain regardless of temperature settings, indicating they treated it as a legitimate website.

Just 23 days later, on March 31, attackers registered that exact domain. They deployed a phishing kit known as Montana Empire, which mirrored the legitimate postal service website in real time. The operation was designed to steal payment card details, bank transfer information, and national identity documents.

Researchers also discovered that the operators used a Telegram bot to manually approve victims’ one-time authentication codes. Project files and session logs further indicated that the phishing kit itself had been created with the help of an AI coding assistant.

The second case followed a similar pattern. Unit 42 identified another AI-generated postal service domain 51 days before attackers registered it. After acquiring the domain, they built an almost identical copy of the official website, added a fake 4.8-star rating, claimed to have more than two million users, and used the site to distribute a malicious Android application.

Researchers also identified additional domains impersonating a major bank in the United Arab Emirates, a European bank, and sports betting websites targeting users in Bangladesh.

According to the researchers, Phantom Squatting is a natural evolution of Slopsquatting, a technique in which attackers register nonexistent software packages invented by AI coding assistants.

Previous research has shown that code generation models frequently recommend packages that do not exist. Attackers behind the PhantomRaven campaign exploited this behavior by distributing malware through 126 malicious npm packages, which were downloaded more than 86,000 times.

Researchers believe this reflects a broader shift in cybersecurity. AI-generated output is increasingly becoming the starting point for real-world actions taken by developers, users, and autonomous agents before anyone verifies the information. At the same time, AI is reducing the amount of time defenders have to respond to emerging threats.

The situation is further complicated by the growing availability of phishing-as-a-service platforms. Researchers pointed to the Lucid and Lighthouse phishing kits, which together relied on approximately 17,500 fake domains impersonating 316 brands across 74 countries.

To reduce the risk, organizations are encouraged to predict which fake domains AI models are likely to generate and monitor their registration before attackers can exploit them.

Researchers also recommend that users:

  • Never trust a URL simply because it was suggested by AI, and always verify that the domain belongs to the official service.

  • Prevent AI agents from automatically opening or downloading resources without validation.

  • Treat all AI-generated information as an unverified draft rather than an authoritative source.

Unit 42 concludes that the opportunity for Phantom Squatting attacks is already here. The remaining question is who will reach these AI-generated domains first: defenders or cybercriminals.

Subscribe
Notify of
0 Коментарі
Oldest
Newest Most Voted
Found an error?
If you find an error, take a screenshot and send it to the bot.