“The noise makes it difficult to quantitatively and qualitatively analyze spectral data obtained using optical methods,” Yury Kistenev, head of the Laboratory of Laser Molecular Imaging and Machine Learning at TSU, explained. “Various filters have been created to reduce noise, but the majority of them distort the useful signal while filtering. For example, the commonly used Gaussian filter leads to signal blurring. It can be compared to a picture with blurred boundaries.”
TSU scientists have found a technical solution to this issue. They trained a deep neural network on a large sample of data, including noisy data that were synthesized according to the problem specification. Then the neural network was tested on other spectral data that were not used in training the neural network. According to the developers, model experiments showed high filtering efficiency. Moreover, the new approach enabled not only significantly reducing the noise level, but also recovering the original signal from the noisy one.
The approach was created as part of a project implemented by Tomsk State University and University of the Littoral Opal Coast (France) under the support of Ministry of Science and Higher Education of the Russian Federation. A new way of analyzing molecular components in the atmosphere using terahertz spectroscopy and artificial intelligence methods has also been created as part of the joint research. It can be used for the analysis of chemical compounds in the atmosphere for the environmental monitoring and control of industrial pollution. In addition, the development will help to counter human-made, biogenic, and terrorist threats.
“In fact, the technology developed for noise filtering is universal and it can be adapted to different optical methods, whether it is terahertz spectroscopy, Raman spectroscopy, or other types of spectral analysis,” said Yury Kistenev. “For example, we use a deep neural network that can clean the spectral signal from noise while developing new methods for diagnosis based on exhaled air. The research is being carried out at the laboratory under the support of a megagrant from the Russian government.”