Warn your boss, then get a restraining order placed against him if he shows up anywhere near you. Keep threatening voicemails or texts to show that you need one. You don't need to be a part of his drama. You can legally keep him away from you. I know, I know. Not quite THAT easy, but you can stifle him. Make sure you change your locks so he can't get in your house. He sounds dangerous. I'd keep contact to a minimum. No reason to agitate him further or throw oil on the fire, so to speak. Why answer his calls? You know he's not calling to ask about your well being.
Your little child needs a strong mother, a healthy mother and a peaceful life. And you need your serenity intact.