Essie,
Although I don't have them directly from the thought of going through the bathroom door. I have had panic attacks so bad, that I had to go right then and there, very loose, and one time had a panic attack so bad, didnt even make it to the bathroom, it was scary.
Perhaps your associating the panic with the bathroom itself, since your literally shaking as you approach it. Im the same with cars, etc. or places.
Yet your right, it is not a place you can avoid.
Have you tried during the day, when you dont feel the need to go, to just walk and stand in the bathroom, and see if that is anxiety provoking? Or is it more that you do fine in the bathroom for cleaning etc, and it only happens when you need to have a bowel movement?