Feel so lonely and alone

I will try to make it short but the story is quite complicated. I wasn't happy with in my marriage (9yrs together 2yrs married) and wanted a baby too. I met someone who obviously at the time seemed to want exactly what I wanted from life. I decided to leave my husband. I'm not going to lie... I left him for the other man.
Started seeing this man after I separated from my husband and it was great. We spoke about everything and again wanted the same things. 
After 3 months he kind of left me saying that he was struggling with himself and didn't want to make me unhappy. He explained in more details what was bothering him. So we stopped seeing each other. After a week he wanted to get back. We did. After another 3.months we moved in together but he started being weird again, distant and a bit depressed. So I had enough and asked him to leave. On the same day later I found out that I was pregnant. He always was clear about wanting a baby and wanting to be in Baby's life. So when he found out he said that exactly again. I agreed as I didn't want to use an unborn baby as a blackmail. He however, after couple of days returned for a chat and said that he wanted to try again to have a family. I agreed as that's what I have wanted my whole life, to have a loving family and someone I can feel safe with. 
He moved back in. That's was 2 months ago. Since then we have been bickering. We are both opinionated and stubborn. We are both defensive. It started feeling as we were just mates. I mentioned that to him yesterday and he agreed. He again said that he is not making me happy. Every time it comes to that it doesn't seem that he wants it to work or even fight for it. It seems that he is just wanting me to end it and he doesn't fight against it. 
So it seems that we are not together but he is behaving like my mate explaining that we need some kind of relationship as we will be bringing the baby up together/separately.
I have to go spend Christmas with his family as agreed before. I tried to explain that the baby is not born yet so there is no need for us to be close. I just can't deal with it having him so close. I don't feel as he cares much about me but obviously I want a family so it is hurting very badly. 
I don't know what to do or what to say and what is right. All my strength that I used to have is somehow gone. I can't get him out of my life, which would make it easier to deal with my emotions and feelings. Has anyone been in similar situation? How do I look after myself? At the moment I regret everything that has happened so far. It even comes to that point where I don't even want this child (I will not hurt myself or the baby, ever) but I can't seem to be able to bring myself back to my strong self. I keep crying constantly and wanting to just run away. I feel lonely, hopeless and worthless.