Researchers from Technion have developed a revolutionary software package that enables computers to perform processing operations directly in memory, bypassing the CPU. This advancement is a significant step towards creating computers that perform calculations in memory, thereby eliminating time-consuming and energy-intensive data transfers between hardware components. The new field of in-memory computing introduces a paradigm shift in computer operations.
Traditionally, CPUs run calculations based on information stored in the computer’s memory. However, the in-memory approach allows some operations to be performed directly within the memory. This reduces the need for data transfers between memory and the CPU, leading to substantial time and energy savings.
In recent decades, the performance of both processors and memory units has improved dramatically. However, the “memory wall problem” persists, where data transfer bottlenecks limit a computer’s overall speed. Professor Shahar Kvatinsky from the Andrew and Erna Viterbi Faculty of Electrical and Computer Engineering has focused on solving this issue.
He has published papers presenting hardware technologies that enable some operations to run in memory, alleviating the congestion between the processor and memory in conventional computers. This approach has promising applications in many fields, including artificial intelligence, bioinformatics, finance, and information systems. Despite significant progress in memory architecture and innovative memory unit production, software development for in-memory computing has been relatively unexplored.
Advancements in in-memory computing
Traditional computer programs are written for standard computer architecture, which has remained mostly unchanged since the 1940s. With some computations now handled by the memory, new software based on instructions that support in-memory computations is required.
Professor Kvatinsky explains, “This new computation method is so different from the conventional one that it renders some of the existing building blocks of computer science unusable. Therefore, we need to write new code, which requires a lot of time and effort from software developers.”
A new platform developed by Professor Kvatinsky’s research group, led by Ph.D. student Orian Leitersdorf in collaboration with researcher Ronny Ronen, addresses this challenge. The platform, known as PyPIM, uses a set of commands that bridge the gap between in-memory computing solutions and popular programming languages like Python.
The researchers developed a theory for the programming interfaces of in-memory computing architecture and created software development libraries that convert Python commands into machine commands executed directly in the computer’s memory. PyPIM enables software developers to write programs for processing-in-memory (PIM) computers with ease. The platform includes a simulation tool for developing hardware and measuring performance, allowing developers to estimate improvement in code runtime compared to conventional computers.
The researchers demonstrate significant performance improvements in various mathematical and algorithmic computations using PyPIM. The new research was presented at a recent conference in Austin, Texas, and the details are available on a preprint server. Orian Leitersdorf, only 21 years old, is soon to become the Technion’s youngest-ever Ph.D. graduate.
Ronny Ronen is a senior researcher in the faculty and heads the Architectures and Circuits Research Center (ACRC).
Cameron is a highly regarded contributor in the rapidly evolving fields of artificial intelligence (AI) and machine learning. His articles delve into the theoretical underpinnings of AI, the practical applications of machine learning across industries, ethical considerations of autonomous systems, and the societal impacts of these disruptive technologies.
