I too have panic attacks while driving. I have frozen in the middle of an intersection a few times. This makes me even more scared to drive. However, I remember the days when I did drive without any anxiety at all. This motivates me to get better because I KNOW I CAN DO IT.
What I do right now is drive on days when I'm not filled with anxiety, but almost always with someone in the car (just in case). I've had to pull over a few times to gather my thoughts, but I did it. When things get really bad, I'll switch drivers. The key is that I continue trying.
Now, I'm able to drive to the convenient store about 1/4 of a mile away from my house by myself. I've even driven myself to work on a snowy day. What I do is focus on something else. It may be work, food, or just getting out of the house. When you really want to get out of the house and are too embarrassed to ask someone to bring you food, you'll find the courage to do it.
I've been shakey and nervous, that's when I know I have to slow down or pull over. I try not to think about the other drivers. Driving is my responsiblility not theirs, they can go another route if they want.
"Take ownership of this", is what I say. I want to be in control and not allow my fears to be in control. So, I will do what it takes to overcome this. Does this make sense? It does to me and this is what has helped me so far. I'm certainly not out of the woods, but I'm butchering those trees one by one! Be as strong as you can be.