Watt Matters in AI: Hardware-based views on energy efficiency

Researchers are tackling AI-related energy efficiency challenges technologically on two fronts: through hardware innovations (creating low-power, brain-inspired computing devices and accelerators) and software innovations (developing algorithms and techniques to reduce the computational cost of AI models). In addition, considerable attention is being paid to behavioral change (e.g., ethical frameworks, corporate responsibility, user involvement, policy development, and public awareness). Many of these advances have been published in leading journals and conferences and are available as open-access articles.

This article focuses on opportunities on the hardware side. We provide an overview of recent open-access research (2024-2025) that focuses on strategies to improve the energy efficiency of AI hardware.

Neuromorphic and brain-inspired computing

One promising direction takes inspiration from the energy efficiency of the human brain, leading to neuromorphic computing. Neuromorphic systems use spiking neurons or analog memory elements to mimic brain processes, potentially achieving orders-of-magnitude efficiency gains over conventional digital chips. A recent open-access Nature Communications paper (2024) demonstrated a neuromorphic hardware platform using 2D material tunnel FETs (field-effect transistors) to implement leaky-integrate-and-fire neurons​. By leveraging a novel 2D semiconductor device, this neuromorphic circuit achieved two orders of magnitude higher energy efficiency compared to a standard 7nm CMOS baseline. This result paves the way toward “brain-like energy-efficient computing” using non-traditional device technologies.

Another breakthrough in neuromorphic hardware employed memristors (resistive memory devices) as artificial synapses. Weilenmann et al. (2024) built a system with a single memristor per synapse that can emulate multiple synaptic functions (long-term and short-term plasticity, meta-learning) within one device​. Using these multi-functional memristors in a deep neural network, they demonstrated learning to play Atari Pong with far lower energy. The memristive network’s energy consumption was about 100× lower than an equivalent GPU-based implementation​. This shows that neuromorphic hardware with brain-like synaptic dynamics can dramatically cut energy usage for AI tasks, “broadening the applicability of neuromorphic computing” and reducing AI’s energy cost​.

Watt Matters in AI: in search of energy-efficient AI

In anticipation of the "Watt Matters in AI" conference, IO+ describes the current situation, the societal needs, and the scientific progress

Read More

Researchers are also improving neuromorphic efficiency through circuit design innovations. For example, an npj Unconventional Computing 2024 paper introduced adiabatic spiking neuron circuits that recover and reuse energy. By operating CMOS neurons in an adiabatic regime (slowly charging/discharging to minimize dissipation), the design achieved >90% energy recovery efficiency and a 9× reduction in energy per spiking operation compared to non-adiabatic circuits​. This adiabatic neuromorphic approach demonstrates how combining brain-inspired models with low-power circuit techniques yields ultra-efficient computing.

Analog and photonic computing (physics-based AI)

Moving beyond digital electronics, researchers are exploring analog, optical, and other physics-based computers to run AI models with much lower energy. Photonic computing, which uses light rather than electrical signals, can perform massively parallel operations with minimal resistive loss, promising tremendous speed and energy advantages. A 2024 Nature paper by Dong et al. showed that using partially coherent light (from inexpensive LEDs instead of lasers) in an optical neural network can boost computing throughput without sacrificing accuracy​. This challenges the conventional wisdom that lasers (highly coherent but power-hungry) are required. By embracing “less rigorous” light sources, they enabled high-throughput photonic tensor cores with simpler cooling and control, pointing to more practical and energy-efficient optical AI hardware. The system they demonstrated performed inference tasks (like classifying handwritten digits and medical gait data) with >90% accuracy, using on-chip photonic memory and modulators​. This is a key step toward optical AI processors that could outperform electronics in speed and efficiency​.

This chip slashes AI energy consumption by 1000 times

A team of American researchers developed a chip that can dramatically cut down on AI's energy consumption.

Read More

Other physics-based computing approaches include Ising machines and analog circuits that solve optimization problems or compute neural networks in specialized hardware. For instance, researchers have built oscillator-based Ising solvers using VO₂ devices that operate at microwatt power levels​​. These exploit physical dynamics to find solutions with minimal energy. More generally, analog compute-in-memory (CIM) architectures perform matrix operations directly within memory arrays (often using resistive memory or capacitors), eliminating energy-intensive data movement. A notable advance in late 2024 was IBM’s demonstration of a 64-core analog-digital hybrid CIM chip that achieved nearly 40× higher energy efficiency on neural network inference tasks​. By combining analog matrix-multiply units (efficient but noise-prone) with digital logic for precision, the hybrid “AnDi” architecture ran a full YOLO object detection network in hardware – the first time an analog CIM system tackled such a complex task​. The result was a 39.2× energy reduction compared to conventional digital FP32 computation, with only minimal accuracy loss, thanks to the clever integration of analog computing for parallel operations and digital computing for control​. This work (published in Nature Electronics 2025) highlights the potential of mixed-signal AI accelerators to cut power usage while handling real-world AI workloads dramatically.

Techxplore.com

Figure: A hybrid analog–digital compute-in-memory system (“AnDi”) for neural network inference. The analog memristor arrays perform energy-efficient matrix multiplications, while digital processors provide high-precision control. This dual-domain architecture achieved 39× better energy efficiency on YOLO object detection compared to a purely digital approach​.

Low-power AI accelerators and in-memory computing

In parallel with neuromorphic and analog paradigms, more conventional digital accelerators are being optimized for energy efficiency. Techniques like processing-in-memory (to reduce costly data transfers), custom datapath designs, and low-voltage operation can yield significant power savings. Many of these accelerators draw inspiration from brain-like sparsity or event-driven processing, even if implemented digitally. For example, one recent design used digital approximate memory combined with CMOS circuits to implement a sensory neuron–inspired system for spike-based signal processing on-body, enabling energy-efficient monitoring (e.g., a spike-based wearable sensor)​.

Artificial Intelligence unveils some 160,000 new virus species, revolutionizing viral discovery

The discovery of over 160,000 new RNA virus species is attributed to a collaborative effort led by Alibaba Cloud, Sun Yat-sen University, and The University of Sydney.

Read More

Another trend is compute near-memory: integrating AI cores with memory arrays (e.g., SRAM, ReRAM) to minimize data movement. By 2025, prototypes of in-memory NN accelerators have shown promising gains. Wang et al. (Tsinghua Univ.) built a memristor-based CIM module that, when evaluated on deep networks, delivered high precision (2.7× better than prior analog systems) and huge efficiency gains. Such accelerators often exploit approximate computing principles (discussed more below) at the circuit level – for instance, using lower precision arithmetic or analog accumulation – to save energy.

Overall, the past year has seen multiple real-chip demonstrations of low-power AI accelerators, from specialized neuromorphic chips (event-driven spiking processors) to hybrid analog-digital NPUs (Neural Processing Units). These hardware advancements are bringing us closer to AI platforms that can deliver major performance-per-watt improvements, enabling AI deployment in power-constrained settings like edge devices and battery-powered systems.

Disclaimer: In finding and analyzing relevant studies for this article, artificial intelligence was used.