Google Reports Model Extraction Attacks on Gemini AI

Background

Google released a quarterly self‑assessment that highlights emerging threats to its AI products. Central to the report is the claim that “commercially motivated” actors have attempted to duplicate the knowledge embedded in Google’s Gemini large language model (LLM) by simply prompting the chatbot. The assessment frames Google as both a victim of illicit activity and a defender of its intellectual property.

Recent Attack on Gemini

According to the report, one adversarial session prompted Gemini more than 100,000 times across a variety of non‑English languages. The purpose of the massive query volume was to collect the model’s responses, which the attackers could then use to train a cheaper, copycat version of Gemini. Google labels this behavior “model extraction” and characterizes it as intellectual‑property theft, even though the Gemini model itself was trained on publicly available internet material that was scraped without explicit permission.

Industry Context and Prior Controversy

The practice of building a new model from the outputs of an existing one is widely known in the AI field as “distillation.” Distillation allows organizations to develop functional LLMs without the billions of dollars and years of research that companies like Google invest in original model training. The report notes that the attackers appear to be private companies and researchers seeking a competitive edge, and that the activity has been observed worldwide. Google declined to identify any specific suspects.

Google’s own history with model‑cloning tactics is referenced. In 2023, The Information reported that Google’s Bard team was accused of using ChatGPT outputs from ShareGPT—a public site where users post chatbot conversations—to help train its own chatbot. Senior Google AI researcher Jacob Devlin, creator of the influential BERT language model, warned leadership that this approach violated OpenAI’s terms of service. Devlin subsequently left Google to join OpenAI. While Google denied the allegation, internal sources indicated that the company stopped using the external data.

Google’s Policy and Response

Google’s terms of service explicitly forbid extracting data from its AI models in the manner described in the Gemini incident. The company’s self‑assessment serves both as a warning to potential attackers and as a public statement of its commitment to protecting its AI assets. By labeling the activity as theft, Google underscores the seriousness with which it views model extraction, even as it acknowledges that the original training data for Gemini was gathered without explicit permission.

Implications for the AI Landscape

The disclosed attack highlights a growing tension between open‑source AI research and commercial protection of proprietary models. As more organizations seek to leverage powerful LLMs without bearing the full cost of development, practices like distillation may become more common, raising legal and ethical questions about ownership of model‑generated knowledge. Google’s report suggests that the industry will continue to grapple with how to balance innovation, competition, and intellectual‑property rights in the rapidly evolving AI ecosystem.