Yes, profile. Anecdotally, I once removed a number of virtual function calls as they were imposing significant overhead in a tight loop (image processing). Would that be the case today? No idea; I'd have to measure it again.
Image processing is a different beast. A lot of subtle things are in play here. I remember getting big improvements by iterating through the images buffer either line vs row or even arranging memory in 64 byte squares instead of linear. I was often surprised when I looked at the optimized assembly. Optimizer are really clever and do a lot of surprising things....
Yes: row traversal (no hopping around in memory and causing cache flushes) and block size reads/writes are a big deal. Luckily they're easy enough to do right every time once you know. I used to take some flak from the "premature optimization blah blah" side of the house on occasion.
