A relationship thats falling apart since having a baby. Please helpful advice!

Im very new to the forum circle! So am a little nervous! But would very much appreciate some advise if anyone has the time to listen to my post.

I'm 31 and have always lived at home with my parents and sister. I was saving to move out - then I fell pregnant, Im self employed in childcare so I needed the savings in order to take maternity leave.

I jumped feet first into a relationship with a good friend of 6 years, we loved each other and decided to give it a go. He is 36 and lives with his mum 50 mins away from me. However He lost his driving liscence and was banned for 2 years.. then a year into the relationship I fell pregnant!  Having always been around babies, I was so happy and grateful to have fallen pregnant. Unplanned and a big surprise - but still happy! He was not. He wanted me to get an abortion. Was cross because i didn't want one, told me I was selfish and thought it was a terrible idea as he wouldn't get his licence back till baby was 3 months old. ( luckily he appealed and got it back a week before due date) it took him till my first scan to come round to the idea.. LuckilyI had my family around me, and I've had their love and support from day 1.

For the majority of my pregnancy I felt I couldn't talk about too much, or how I was feeling with him. Things just spiralled downhill - for example. I had done all the driving since the start of the relationship &  staying at his at the weekends.
I reached 7.5 months pregnant and I said nicely i just couldn't do the journey anymore or stay at his, his bathroom was downstairs, I was up down in the night, he'd sleep till lunchtime and he'd be quite happy to eat once a day. I know i wasnt the only one pregnant in the world! And luckily I felt relatively well throughout, but - I was still pregnant and its hard on the body at times! At 7.5 months I wanted to be near home, rest eat! Etc. He got the hump with me, and said well I'll have to get the train down then won't I. 

On New years eve, at 2 am I said please can we go soon, again he got the hump. And I didn't end up getting to bed till 5 am. Crying from being so tired carrying a baby!

 If i said things like -  is it okay if we don't go on the motorway too soon with a newborn he would reply. "Well let know when i can see your baby"

these sort of situations went on, always making me feel bad. But me always apologising making things ok. I would try and talk about how things would work when baby comes, etc that we should maybe think about moving out- but it would be very hard to get anything out of him, yet if i suggested things he'd again get the hump with me! Saying well I have no choice do I! So I said I would live at my home with baby. 

Our baby arrived, a little girl. And my whole world changed as it does! Your heart fills with so much love you don't know what to do with! Baby and i stayed in hospital  for 7 days for a course of antibiotics for us to finish. My boyfriend was very happy when she arrived, and fell in love with her. - I thought things might pick up! He went back to work straight away, which i didnt stop him doing as I was at home with my fanily.. however reality hit and early on we had problems. I couldn't be more grateful to have been with my family. 

He would come most nights at first. I breastfed, so obviously its hard for the dads. But he would get to mine at 7pm sometimes later - When new borns cluster feed, I would pass her to him yet she needed feeding within 5 mins and was stuck on the boob for most of the night,  then it was - attempt bed, sleep etc. He never ever got up in the night, would just roll over and say you ok. To then snore all night. 

When baby was 5 weeks old he wanted to go to a bbq and stay at his- I found my strength after giving birth and I said no,  I wasn't ready to go anywhere yet, I know some ladies are ready to be up and out and I admire them! I just wasn't ready. He then sat in bed in silence for 2 hours while our baby was unsettled for 3 hours didnt once ask if we were ok if we needed anything etc,  even my dad came to me.and said ill walk with her all night if you need me too! 

Things got worse and it fell into every other night he was coming over. At 9 weeks old,  he messaged me and said he was coming to get her and taking her to his with or without me. Whether I'm coming or not. I obviously felt instantly sick! I said you can't demand you just take her! Shes never been without me! I dont want to be away from her! Plus He's never settled her when she cries, never spent more than 45 mins awake time With her. He said ill just give her a bottle she'll be fine. I said it doesn't work like That! She's never had a bottle. 

I will say I thought id have a really laid back baby! Oh i was wrong! She isn't a baby that feeds and is then happy for 2 hours. Shes fed on demand, alot! hates being in the car, only just sleeps in pram. Only cat naps 30 mins here and there unless shes on me. So since she was born I was more than happy to go with it! Instead of fight for a routine etc. 

So I felt I had no choice but go. It took me 2 hours to get There. Baby was upset, I was upset. Had to stop numerous times. She was out of sorts wouldn't sleep, Then we piled in the car again to go to his brothers And back agajn.And all I kept thinking was I have do the journey to get us home. I said I wasn't doing that again till shes a bit more settled. 

We bickered for days, he then stormed out of my house one night after shouting at me saying im not aknowledging any of my behaviour. He then didnt see baby for a week. 

She's now 17 weeks,  I've been with her day and night, bath with her most nights, we have managed to get into a nice night time routine led by baby, have been by her side every day since birth and wouldn't have it any way! It's now fallen into a pattern that he was coming every 2 - 3 days. But doesnt get here till 7.15 ish. When she feeds and is ready to sleep by 8. So he sees her for max 20 mins. I feed her, he kisses her and she goes down to bed and he leaves. Then on a Sunday I walk her down to the garden centre we sit for a bit, he sees her,we might walk her, she then falls asleep and he tends to leave. 
One week he brought his mum, he Didnt tell me. I just turned up and they were both there, she made it so awkward and horrible. She Held baby for a little bit. Then passed her to the b and then went and looked round the shop for 20 mins. 

The last couple of weeks i decided we shouldn't be together right now, im constantly anxious, i get worried and feel sick because of the situation I worry about saying anything to his face incase he gets cross and i worry about the future. Scared he's just going to take her.

 I'm now back to work however its only 7 mins from home, im up at 5.45 for a school/ nursery runs, but am home by 9 am. I then go back at 4pm till 7.30/8pm and my mum has her for me. 

He has turned around and said well I will put her to bed when you're at work. After everything that's been said and done im not comfortable with this. He's never been around to see her routine, has never put her to bed, now barely speaks to my family so that is now awkward! And i dont really want him there without me, as baby doesn't really see him at night - to suddenly be put to bed by him. 

I've spoken to health v and  child organisations for advise and they've told me to do what is best for the child.. And if im not comfortable with things I'm in my rights to say so. 

I've never ever stopped him from see her,  ever. I've never said no don't come. Ive never said Only  come on Sunday's, or  No you're not to see her now. Never anything of the sort. What he does with his time as a 36 yr old is up to him. And of course he has every right to see her, But I'm also trying to work around baby and when is best, i dont feel he can just rock up when he wants?

I don't feel its best for baby, to try and keep her awake for him to see her for 10 mins at night for her to be grizzley,  feed and then fall asleep. They then don't see each other.  Or for him to put her to bed when I'm not here. It would take a bit of time to know her ways etc and for her to get used to someone else putting her to bed. 

I'm trying to not be difficult and i suggested maybe he could come in the day time in the week so at least he could see her awake! - he has his own business so can take time off. I suggested we take her swimming on Sunday together. But it's so hard to not feel an overwhelming protection over our baby, because of things that have been said and him not really being around to get to know her. 
 
He has said that this situation is my fault because i chose to have her. That I'm stopping him from seeing her. That he's cried because he doesnt see her. He Doesnt understand why I couldn't move into his when  I had baby. I'm constantly being told its me, my behaviour. And I'm starting to believe that it is my fault?! I understand how hard it must be for him, I Do, but surely you'd want to do something about this situation? Make a plan to move out together etc? But that's never happened!

I would love any kind of advise or any similar stories that have worked out ok? Am i being unreasonable in the things im not overly comfortable with after this long? I'm I in my rights to say no? Am I over reacting?! We are all mummys so please dont shoot me down too much if you disagree with my post! Thank you all for your time!