Artificial neural networks are becoming increasingly large and complex – and therefore require more and more energy for training with data. A research team led by Felix Dietrich, Professor of Physics-Enhanced Machine Learning at the Technical University of Munich (TUM), is working on replacing training steps with mathematical methods. This reduces power consumption and helps to better understand how artificial intelligence (AI) works. “We are replacing iterative training steps with probabilistic calculations and are specifically searching for values in the data sets that change particularly strongly and quickly when parameters are changed,” Dietrich explains the process in an interview with the Leibniz Supercomputing Centre (LRZ). Although the calculations cause the artificial neural networks to grow, they eliminate the need for tens of thousands of training runs. Currently, the strategy works for simple feedforward and recurrent networks used in machine learning for time series and tabular data, as well as for graph models for image and pattern recognition.
Dietrich’s team is working on mathematical solutions for convolutional and attention layers, which are important in the development of generative AI. The mathematician advises researchers who want to use AI to evaluate data not to constantly reinvent the wheel but to rely on existing models. Such existing models can also be adapted and balanced for your own new data sets.
The entire interview can be found on the LRZ website.
Artificial neural networks are becoming increasingly large and complex – and therefore require more and more energy for training with data. A research team led by Felix Dietrich, Professor of Physics-Enhanced Machine Learning at the Technical University of Munich (TUM), is working on replacing training steps with mathematical methods. This reduces power consumption and helps to better understand how artificial intelligence (AI) works. “We are replacing iterative training steps with probabilistic calculations and are specifically searching for values in the data sets that change particularly strongly and quickly when parameters are changed,” Dietrich explains the process in an interview with the Leibniz Supercomputing Centre (LRZ). Although the calculations cause the artificial neural networks to grow, they eliminate the need for tens of thousands of training runs. Currently, the strategy works for simple feedforward and recurrent networks used in machine learning for time series and tabular data, as well as for graph models for image and pattern recognition.
Dietrich’s team is working on mathematical solutions for convolutional and attention layers, which are important in the development of generative AI. The mathematician advises researchers who want to use AI to evaluate data not to constantly reinvent the wheel but to rely on existing models. Such existing models can also be adapted and balanced for your own new data sets.
The entire interview can be found on the LRZ website.