Interesting topic, Nik!
You make multiple assumptions that are untrue to back up this argument. For example: everyone who listens to an audiobook is multitasking. Another one: assuming people don't rewind. There are literally buttons that jump backwards and forwards 30 seconds.
And the idea that listening to audiobooks seriously impedes your learning is pretty funny coming from the Four Minute Books guy. You yourself could probably make the argument that most books are worthless beyond their overall message.
On the other hand you did make a few decent points though.
In conclusion, I'd say if you consume audiobooks while multitasking, you could potentially learn less than reading a physical book. But it's not a given.
Thanks for making me think, sir! :)