Exposure work is a good start to help progress through this. What's in front of you is most important. Use relaxation techniques before driving to keep you calm and collected.
I know what you mean. I am bothered by the same thing. Not always, but sometimes. I'm pretty good about just not looking in my mirrors constantly unless I need to. I get that strange jumpy feeling in my chest if I see a car pull up to a stop as i'm driving through an intersection or such. That is what bothers me the most about driving. I almost always feel the calmest when I am driving. I think it's cause my mind is on the road and not what is going to kill me on that day. Just try to get use to not looking in your mirrors so much. Before I was a really anxious person, I use to get so mad that people would ride my bumper, now I have to ignore them or I get very anxious. I too find myself driving a little slower than I use to. I guess that's a good thing. I am a pretty good driver but somedays, I don't feel as good as I use to be. I sometimes wonder if i'm spacing out when I'm driving! I haven't had any accidents since I was 16. I had a couple tiny ones on my permit. LOL!! Knock on Wood!!