The UN recently released a research paper on the enviromental impact of AI (https://news.un.org/en/story/2026/06/1167658) which gives a lot of good insight into the energy cost of AI. I also wanted to post some of my own opinions on this to give the bigger perspective on this. Not to show that its wrong, but many are not seeing the whole picture.
Before I get to my point, I want to highlight a few other research papers, from Google, Mistral, and independent researchers:
Mistral –> https://mistral.ai/news/our-contribution-to-a-global-environmental-standard-for-ai/
Google –>https://arxiv.org/pdf/2508.15734
UC Riverside / UT Arlington –> https://arxiv.org/pdf/2304.03271
Epoch AI –> https://epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use
Epoch AI states that a ChatGPT query’s energy use is small compared to everyday electricity use, using GPT-4o with a typical query of ~400 tokens as its example.It is important to note that GPT-4o is a “small” model compared to frontier models like GPT-5.x and Claude Opus/Fable. Per-token energy scales with active model size, so frontier models plausibly use several times more energy per token.
Mistral states that their ai assistant Le Chat for a 400-token response, excluding the user’s device.
- 1.14 gCO₂e,
- 45 mL of water,
- and 0.16 mg of Sb eq.
Google stated that the median Gemini Prompt (500 output tokens) uses 0.24 Wh. Where 58% of power usage comes from the AI accelerators (Google uses TPUs compared to GPUs that Mistral use, source: https://mistral.ai/news/mistral-compute/).
We identify that the median Gemini Apps text prompt uses less energy than watching nine seconds of television (0.24 Wh) and consumes the equivalent of five drops of water (0.26 mL).”
What does median mean? How many input tokens? Are we counting the system prompt? The system prompt? How output tokens? Which model Gemini flash?? Does this include systems like Web Search or Reasoning? Is this Web-only, API-only, both? so it is really difficult to understand what median means without them giving better context.
It should also note that Google and other have done a lot in terms of reducing the energy usage because of improvements in hardware and the platform. Stating that over 12 months (May 2024→May 2025) energy per median prompt fell 33× and total emissions fell 44× and secondly their number is not just counting the GPU usage, but also host CPU/DRAM, idle capacity, and data-center overhead. Also they exclude training, external networking, and end-user device energy from scope.
For context: I used Claude Cowork earlier today to create a PowerPoint presentation, which consumed roughly 50k tokens across various Skills and MCPs. Does that mean my energy use was ((500 output tokens = 0.24 Wh) × 100) = 24 Wh? Probably not I don’t think it scales linearly like that (and since the calculation is only on output tokens…)
Coming back to Googles article, this was released back in August 2025. Since then they have done several things to improve the overall the efficiency and reduce the power usage of GenAI. Including TurboQuant and TPUv8 which provides compression of the KVCache and better hardware for inferencing.
- TurboQuant (https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/)
- Tpu-8t and Tpu-8i(https://cloud.google.com/blog/products/compute/tpu-8t-and-tpu-8i-technical-deep-dive)
A speculative estimate
To be perfectly clear I do not think we have good enough data to understand the actual enviromental impact that AI has, sure it uses a lot of compute resources but we cannot directly describe how much. For instance if we use Claude Code we are using close to 20000 tokens before we even start!
But if we look more closely at this example and if power usage really does scale linearly (and it doesn’t make sense for the output-vs-input energy ratio to be constant across input lengths generating output tokens gets more expensive as context grows) Generating output tokens does get more expensive as the context grows, and I estimate that my Claude Code or (Insert other LLM Agent) here, uses about 4000 of the typical queries (like Median Google Gemini output) which is close to 2 million tokens.
Of that ~2 million tokens, the vast majority is input (context being re-read each turn) if we assume roughly 5–10% of it is actually generated output, that lands somewhere around 100,000 output tokens and since generation is the expensive part, that’s the figure I’ll use to estimate energy.
When plotting this into AI I got a measure when using a frontier model like Opus, and getting 100,000 output tokens would be the equvilant of 225Wh.
While this figure is highly speculative, and does not take into account that Anthropic is using different inferencing hardware both TPU from Google, AWS Trainium and NVIDIA GPUs. It atleast showcases a bit of the puzzle.
A long Opus response can use anywhere from a microwave-minute to a household’s-worth of electricity but only if we assume energy scales linearly with output length, that Opus draws several times more power per token than GPT-4o, and that the chosen chip (GPU, TPU, or Trainium) performs at its published efficiency.
It is also important to note that this estimate does not factor in LLM training, external networking (MCP and API calls), or end-user device energy. So we can safely assume the true energy use is even higher.
So even if the energy efficiency is improving because of hardware improvements, so is the token usage from the end-users because of all the workflows, graphical workflows (MCP Apps) and integrations that are being used by developers. This also alignes with Jevons paradox, also called the rebound effect “It says that when technology becomes more efficient and cheaper to use, adoption and usage often grow enough that total resource or energy consumption increases rather than decreases“
So how big is a fish?
In Norway we have a saying, “How big is a fish?”, which we use when something is difficult to answer in terms of size or measurement because it depends entirely on which fish. The power usage of GenAI falls into this category, since we don’t have all the data needed to calculate it.
While Google for one stated that 500 output tokens uses about 0.24 Wh (for an older model) this does not take into effect other big factors like training so we can treat this number as a floor and not the true cost, and for those heavy users of AI 500 tokens output is a REALLY small number.
In addition
* Agentic workflows are becoming the norm. With integrations, API calls, and multi-step tool use, even simple tasks now burn far more tokens a typical interaction is heading closer to 10,000 tokens than to a few hundred.
* Hardware and infrastructure keep getting more efficient (e.g. next-gen TPUs, KV-cache compression), but not every vendor runs the most efficient stack — many still rely on NVIDIA GPUs, so the gains aren’t uniform across the industry.
* Models keep getting larger, and per-token energy scales with active model size so each token from a frontier model tends to cost more than the same token from a smaller, older one.
To wrap things up… Per-token efficiency is improving, but total consumption is being pushed up by more tokens and bigger models which is exactly why “how big is a fish?” is the honest answer to this question.