More energy efficient and cheaper platform for AI inferencing

Dejan Kostic describes his planned work as”ChatGPT for every child". As a Wallenberg Scholar, he wants to create a scalable and adaptable platform for running energy-efficient, domain-specialised large-scale language models like ChatGPT, known as LLMs, to solve the problem of high cost and unsustainable energy consumption.

By creating pruned, energy-efficient LLMs, the research aims to democratise AI by supporting multilingual user instructions and making it accessible to a wider community.

The research has the potential to revolutionise the way LLMs are used, which would enable more sustainable and affordable AI adoption. The implementation involves eliminating power waste and delays caused by CPUs, i.e. processors, as well as enabling direct network communication with specialised accelerators such as GPUs,  and building an adaptive framework to incorporate new LLMs. 

Ten times more efficient

By using off-the-shelf solutions and standard hardware, Kostic is aiming for a 10-fold efficiency gain compared to state-of-the-art technology. The platform will not only make LLMs more sustainable but also help answer fundamental scientific questions about model pruning and domain specialisation. 

Kostic believes that the project's impact could be significant as it could enable wider adoption of AI technologies and support multilingual interfaces, ensuring that the benefits of AI reach a wider audience and promoting a more equitable AI-driven future.