As they say, you can pick your friends, but you are stuck with family.
I have an array of family members that I do not care to see. I see them as rarely as possible. But there are times that it is necessary.
At the risk of sounding repetative, if you had some counselling, and were feeling better about yourself, the whispers and general "not-niceness" would not bother you so much.
That is what worked for me.