I had the same problem, playing with the speed helped for me. Similar to meditation, I also learned to focus more on the listening. I completely agree that text is better for jumping and audio might also not be perfect for everybody. For fiction audio it also depends a lot on the narration for me. It needs to be top notch or I "check out".