The real monster was priority inversion in kernel's scheduler that caused audio processing thread to starve during model inference!"