Table of contents
Ayming Institute : the think tank of the Ayming Group.
The Ayming Institute (AI) aims to help leaders in the private and public sector gain a deeper understanding of the evolving global economy by focusing on three areas.
The first area is sustainability. We believe that the environment and social responsibility are critical issues for businesses today. For this reason, our content aims to help companies integrate these issues into the way they make decisions.
The second area is business development. Through our content, we wish to help companies to develop a stronger business culture and a sustainable approach to growth.
The third area is the people side of the business. With our content, we want to support individuals as they navigate their careers, learn new skills, and find ways to contribute in a world that is constantly changing.
Our strongest commitment is to help organizations better understand how markets are changing, and how they can build better businesses as a result. We aim to do this by providing analysis of the global economy’s transformation; sharing our insights through thought-provoking publications, and engaging business leaders in conversations about the economic changes that are affecting all of us.
Sustainable computing
Over the past 60 years there has been a 1 trillion-fold increase in the processing performance of computers – now including smart phones and watches, video game consoles as well as supercomputers.1
The seemingly inexorable growth in computing power is a significant driver of economic growth and innovation. For example, in fields such as medical research and weather forecasting, between a half and over 90% of improvements achieved can be explained by that expansion in the power to compute.2
If the next decade belongs to artificial intelligence (AI) and generative AI (GenAI), again today’s computing power not only takes the credit – for the development and training of large language models, along with processing the vast datasets involved in machine learning (ML) and deep learning (DL). Capacity will also need to expand at a rapid rate to sustain the technology’s deployment across entire economies.
However, continued exponential expansion risks violating the laws of physics and nature. Researchers have been warning for years that limits on transistor density have been slowing the rise in processing power (and the fall in component costs). Also, the deployment of AI risks overwhelming the world’s capacity to compute – or to absorb the mushrooming carbon emissions from running, cooling (and manufacturing) servers and data centres.
Day of reckoning
Moore’s Law foretold the rapid rise of semiconductor capacity, doubling every two years since the 1970s. Computing’s capabilities, efficiency and cost-effectiveness have spiralled over that time. However, the prescient co-founder of Intel also predicted the now approaching ‘day of reckoning’ when miniaturisation of microchip circuitry reaches its limits.
Meanwhile, power-hungry AI has been on an even steeper growth curve. Between 2012 and 2018, data published by OpenAI showed the ‘compute’ required to reach key AI landmarks increased exponentially – by 300,000 times, doubling every 3-4 months.3
Enter generative AI, and a new rapid cycle of logarithm training and development and adoption across multiple industries – triggering concerns that the technology will exacerbate the climate crisis.4
The world’s data centres already account for about 1-1.5% of global electricity usage.5 In the US, that rises to around 2%. Its Department of Energy estimated in 2018 that data centres worldwide consumed some 200 terawatt-hours of power a year – more than some countries. That will grow significantly over the coming decade, with predictions that by 2030, computing and communications technology will consume between 8% and 20% of the world’s electricity, with data centres accounting for a third of that.6
Current adoption trends could see giant tech manufacturer NVIDIA shipping 1.5 million AI server units per year by 2027. Running at full capacity, they would consume at least 85 terawatt-hours (TWh) of electricity.7
A double dilemma
Creating NLP (natural language processing) models can be a massive undertaking. It is estimated that training OpenAI’s GPT-3 large language model consumed more than 1,200 megawatt-hours (MWh) of electricity and generated some 550 tonnes of carbon dioxide (excluding energy use for cooling). That’s the equivalent of running 123 gas-guzzling cars for a year.8 Or driving to the moon and back; and, including cooling, evaporating 700,000 litres of freshwater.9 This water impact goes unreported, yet global AI demand could be responsible for the withdrawal of up to 6 billion litres of water a year by 2027 – half that of the UK.
The footprint from consumers and businesses using these systems is far heavier. Models are trained once and run millions of times. GenAI programs use around 10 times the energy of previous search engines. If Google search was AI-enhanced like ChatGPT and its 9 billion daily searches became chatbot interactions, its annual energy use would leap 30% to 29TWh, overtaking that of Ireland.10 Meanwhile, a rapidly growing mass of business users across most industries will be interrogating AI models in search of their own operating efficiencies and competitive advantage.
Computing faces a double sustainability dilemma – how to extend its technical progress and capacity to support AI without trashing the planet.
How to decarbonize computing?
The obvious solution in the medium or longer term is grid decarbonisation so servers run on green electricity. Great strides are being made in developed countries to switch to renewables for power generation. But despite the somewhat equivocal commitment to ‘transition away from fossil fuels’ at COP28 in Dubai, coal plants continue to be developed, especially in Asia, to meet surging electricity needs, while still-rising oil and gas production risks locking in demand.11
At corporate level, cloud computing and AI-as-a-service allow companies to outsource much of their computing footprint. Organisations can optimize how they use in-house hardware, reducing their direct energy consumption. And there are some global efficiency gains running multiple virtual machines on providers’ servers helps economize on power overall.
Also, encouraged by ESG (environment, social and governance) reporting, and energy regulations in some countries, cloud computing providers – like many of their corporate clients – have signed clean energy supply contracts. Google claims this means its data centres have net zero carbon emissions. Microsoft aims to become ‘carbon negative’ by 2030. It is investigating the use of small modular reactors – like those that power nuclear submarines and aircraft carriers – and even using AI to streamline the planning approval process.12
Cloud providers do not disclose the overall energy demands of machine-learning systems. Some companies may only account for their direct and energy emissions (Scope 1 and 2), excluding those downstream in the supply chain (Scope 3), which can be significant. Offsetting by buying renewable-energy credits may not be sufficient to cover growing energy demands. Most electricity powering data centres comes from fossil fuels, while cooling also contributes to water distress.13
Towards green AI
However, many options for reducing environmental impact are being pursued and interest in ‘green’ AI and computing has grown in recent years.
Energy savings of over 50% are commonly reported. But researchers caution that a concerted effort is essential to target all the different lifecycle stages of highly complex AI systems (including data collection, training, monitoring) and their components (data model, pipeline, architecture, hardware).14
Cloud and edge computing systems allow data processing and storage to be spread across devices, systems, and locations. AI processing distributed among edge devices and local servers means data centres can be smaller, less data has to be transferred, and energy consumption is lower.15
Locating data centres where green energy is abundant can reduce emissions by a factor of 30-40, compared with a grid dominated by fossil fuels.16 One study suggested the choice of data centre, deep neural network model, and processor could reduce the carbon footprint a thousandfold.17
Some providers are making data crunching greener. Crusoe Energy puts its prefabricated units on sites with waste energy, such as methane otherwise flared into the environment. Deep Green uses immersion cooling technology to extract heat from on-site edge micro data centres for local swimming pools, housing, and industry. Its serverless cloud is claimed to be faster and cheaper using 60% less energy than conventional data centres – and carbon neutral (including Scope 3).
The power of hardware
Hardware manufacturers are also trying to do their bit for energy efficiency. This R&D effort includes processors, memory modules, and hard drives designed to use less power without compromising performance, while also generating less heat. Running the hardware cooler should also increase reliability and service lifetime.
Microchip makers are vying to power the accelerated AI goldrush and more efficiently. NVIDIA claims its Hopper H200 chips make supercomputers more eco-friendly as well as faster.18 Amazon claimed its AWS Trainium machine learning chips reduce energy consumption during AI model training by 29%.19
Significant energy savings can also be made in how models are operated. In the training phase, evaluating the rate at which models are learning allows those that underperform to be stopped early; as many as seven in 10 models built for commercial use never make it to the marketplace.20 Studies showed dramatic savings up to 80% in energy use.21
More efficient operation
When deployed, AI models rely on redundant hardware, running continuously, ready to respond to users’ queries. Optimizing the architecture for the model can improve efficiency by 10-20% for the same quality of service. Cloud customers leasing systems from data centres tend to over-estimate their needs, but can use a tool to specify the most carbon efficient mix (of high-power GPUs and low-power CPUs).22
By capping the amount of power a GPU is allowed to draw, manufacturers can reduce energy consumption by about 12-15%, depending on the model. Other power management techniques include sleep modes, dynamic frequency scaling, and screen dimming when devices are idle or underutilized.
One study from 2023 suggests that researchers may have over-estimated AI’s voracious appetite for energy. Algorithmic improvements, hardware specialisation and growing efficiencies compensate at least partially for the AI explosion.23 However, as more and more devices use AI, energy consumption escalates. As with other technology, when it becomes more efficient, demand tends to rise, so energy consumption overall still increases.
So other, more radical solutions may be required.
However, the hard reality for hardware is that manufacturers cannot cram ever-more transistors on chips (the defining trend underpinning Moore’s Law). Small, specialized chiplets can be linked instead. New industry standards will make it easier to package different manufacturers’ chiplets together for a new generation of machines ideal for fast- moving sectors like AI.24
Nanotechnology also may have a large part to play in cutting carbon. Emerging solutions could enable new energy-efficient transistor and memory nanosystems for machine learning, cloud computing and other applications.25
AI, heal thyself?
The climate won’t be a victim of AI’s success if it provides at least part of its own cure. As far back as 2016, Google’s Deep Mind demonstrated the potential of AI-powered energy management to reduce data centre energy consumption – by 40%.26
And there is no denying the massive contribution AI algorithms can make to sustainability in multiple industries – from optimizing electric grids and buildings’ energy efficiency27 to smarter detection of leaks in water management systems.28 The technology may even enable more effective climate modelling, as well as monitoring.29 Research by PwC (commissioned by Microsoft) found that application of AI levers could reduce worldwide greenhouse gas (GHG) emissions by 4% in 2030, an amount equivalent to the annual emissions of Australia, Canada and Japan combined.30
The AI industry, ethical guardrails and regulations need to take account of the climate impacts. Transparency is key. Tools to measure the carbon footprint of AI models are available.31 AI developers, hardware manufacturers, cloud providers and date centre operators must be held to account for their energy and water use to foster environmental responsibility and spur continuous improvement.
Author
Mark Herrity
Senior Manager, Ayming UK
Mark is a Senior Manager at Ayming, UK and leads the IT consulting team who help clients optimize their IT processes, systems, and investments. He has a Master’s degree in Computer Science from UCL, and a BA in Information Technology from National University of Ireland, Galway. Mark has over 9 years of experience in IT R&D consulting, working across numerous sectors and is passionate about helping clients achieve their IT goals and objectives, and creating value for their businesses.
1 https://www.visualcapitalist.com/visualizing-trillion-fold-increase-computing-power/
2 https://news.mit.edu/2022/neil-thompson-computing-power-innovation-0624
3 https://openai.com/research/ai-and-compute#modern
4 https://www.scientificamerican.com/article/a-computer-scientist-breaks-down-generative-ais-hefty-carbon-footprint/
5 https://www.iea.org/energy-system/buildings/data-centres-and-data-transmission-networks#overview
6 https://www.wired.com/story/ai-great-things-burn-planet/
7 https://www.cell.com/joule/abstract/S2542-4351(23)00365-3
8 https://www.scientificamerican.com/article/a-computer-scientist-breaks-down-generative-ais-hefty-carbon-footprint/
9 https://arxiv.org/abs/2304.03271
10 https://www.cbc.ca/radio/quirks/ai-energy-consumption-1.6995014
11 https://www.iea.org/reports/world-energy-investment-2023/overview-and-key-findings?
12 https://www.itpro.com/technology/artificial-intelligence/microsoft-is-using-ai-to-get-its-nuclear-projects-approved-in-the-us
13 https://www.wired.com/story/ai-great-things-burn-planet/
14 https://wires.onlinelibrary.wiley.com/doi/10.1002/widm.1507
15 https://www.squash.io/6-sustainable-technology-innovations-to-look-out-for-in-2024
16 https://www.linkedin.com/pulse/here-comes-sun-why-large-language-models-dont-have-cost-paul-walsh/
17 https://arxiv.org/abs/2104.10350
18 https://www.nvidia.com/en-gb/data-center/h200
19 https://www.aboutamazon.com/news/sustainability/how-amazon-uses-ai-sustainability-goals
20 https://sifted.eu/articles/ai-has-a-trust-problem-meet-the-startups-trying-to-fix-it
21 https://news.mit.edu/2023/new-tools-available-reduce-energy-that-ai-models-devour-1005
22 https://arxiv.org/pdf/2304.09781.pdf
23 https://www.sciencedirect.com/science/article/pii/S2210537923000124
23 https://www.technologyreview.com/2024/01/08/1085094/10-breakthrough-technologies-2024/ 25 https://ee.stanford.edu/research/energy-efficient-hardware-systems
26 https://www.deepmind.com/blog/deepmind-ai-reduces-google-data-centre-cooling-bill-by-40
27 https://www.linkedin.com/advice/0/how-can-ai-algorithms-improve-energy-efficiency
28 https://wint.ai/
29 https://www.caltech.edu/about/news/artificial-intelligence-is-key-to-better-climate-models-say-researchers
30 https://www.pwc.co.uk/services/sustainability-climate-change/insights/how-ai-future-can-enable-sustainable-future.html
31 https://mlco2.github.io/impact/#home
