I had a similar situation, but it was too toxic, with too many people actively being bullies. The management just let it go, since they didn't want their jobs threatened, nor did they wish to rock the boat.
I think I've made a good choice by not returning, since there were too few "friends". I don't wish to relive the nightmare of being victimized. I was too depleted to exercise other options. Also, the supports I had outside of work encouraged me to leave the bad situation.
All I can do is move on, since mental and physical health are important.
It is lonely being outside the large and connected company I was in, and there's no status in not working, or leaving abruptly, but some people literally don't live if they continue in negative situations because of the excessive stress, possibility of heart attack and other possible illnesses.
I know at some level that the "embarrassment" and shame I feel is a result of some depression, due the loss of an important role in my life.
Sometimes I find I had to cut my losses.