I sort of avoid uncomfortable situations. I am not telling you to do the same thing but while I would not interfer with choices anyone makes I do get to decide not to hang around with my daughters in laws. I am not one who is good at direct confrontation. I get tongue tied and say the wrong things and have started realizing that as I dont make my kids defend themselves or their other families or what they do, i dont have to defend myself either.
I think I would cheerfully plan to be out of town for the holidays. I am very uncomfortable with people I dont know well, even if they are nice, and would avoid the group holiday thing. You can find ways to avoid it too. Sounds like its more trouble than its worth. Not telling you to actually do this, just reminding you that its an option.
This family is causing you much angst that you don't need. They had holidays without you before you moved. They can do it now too. I do all I can for my kids and can do ALOT for the two who live right near me. But I would take care of me too. in my opinion your son has no business telling you who to spend your time with, and has no right to yell at you if you decide not to enmesh yourself wirh HIS inlaws. There are ways around it without causing blow ups.
Have a peaceful, calm day.
All is well.