I just think it brings up the wider topic of what would you do if he changed anything around in your car?
I personally never let anyone touch my car. I used to always find everything moved around in places where it was too proper for me. When i sit in anyone else's car i try my best to not touch a single setting, even the radio, just to make sure they feel that i respected their space and property.
If i were to change someone's setting i feel that they have all the right to never let me touch the driver's seat again and the same would be applied vice versa.