In desperate need of advice please

A very long story for who ever reads it I admire you.... I wasn’t sure if it’s approriate to post this on here but I felt like I need advice and someone to talk to.

A bit of background to the problem

My son is currently 8 and half months old.
His dad who I’m married to has never ever taken care of us since he was born (we lived together untill last weekend), he neglcted him as a child and me as a wife. He would go out all day doing everything for other people or sleep, demand to be served upon like he was a king, basically he never even picked up a piece of rubbish after himself. To do with our son he maybe changed two nappies since he was born, I have encouraged him times and times again to even just simple play with him or sit with him throughout a meal but he always was busy doing everything else (going out, playing xbox etc). His excuse was always I won’t let him do anything with his son, what he wanted eg. Drag the baby around with him all day while he does what he wants where I would gently tell him it’s not a good idea as the baby needs to eat and nap. He also claimed to hate living with my family (it was him who convinced me it was a good idea to move in with them, and also my family did everything for him, been more supportive than his own family). So basically I physically done the job of a single parent since my son was born, also he hasn’t really supported us financially, he would buy a toy or a pack of nappies here or there but I dealt with all the bills etc. He happened to be verbally aggressive during argument and call me names eg ‘fat fucking pig’ or ‘stupid mong’ etc. He belittled me over everything as well as my family who we lived with.

On Saturday last weekend I had enough and wanted to tell him I want to split up (the neglect have gotten too far)...

I tried first talking calmly while getting the baby ready in the bedroom as soon as he raised his voice I left the bedroom and asked my mom to look after my little boy while I go and finish that conversation between me and his dad. He started shouting at me etc saying I control him (cuz I ask him not to go out at 12pm and come back 3am in the night and actually help me with the baby). He got aggressive including punching his head and he punching a hole in our bathroom door, upon that my family ran to my rescue thinking he has hit me ( my family has notice he was aggressive forward to me) a non physical fight broke out where he started being aggressive to my family after I asked him to leave several times and he started to swear at my parents on a way out untill he finally left...

Next day I agreed that he can see his son, but only if I’m present (sorry but in my mind it’s necessary as he doesn’t even know how to look after him as he never bothered to).

So far he has seen him a couple of times, I’m not making it difficult to see him or anything I only insist that I been present but I told him the reason I split up with him was that I fallen out of love as I’m terrified if I say anything else he will get aggressive again ( basically every argument we had in the last two years if I was right or tried to get my point across he would either get aggressive or start offending my family etc). He is starting to insist to see his son Alone and I know law wise I cannot deny him that but I can’t let it happen as I know he won’t take care of him properly.

I’m going to see a health visitor tomorrow and I was wondering if I tell her what’s been happening would they be able to offer me support and help without taking drastic steps and anger him? I know I shouldn’t have but I kept promising him I will never make it difficult to see his son.

Has anybody been in this situation and have any advice?

Thank you so much for reading and your reply’s guys

i feel like such a horrible person wanting to officially say what he was like as he had a bad upbringing and didn’t know his dad untill he was about 16 years old (supposly his mom stopped his dad from seeing him) due to that I promised him I would never take his son away from him or stop seeing him but I can’t do it mentally everytime he comes to visit or I got he is coming to visit the next day Im getting so stressed our I’m crying and having panic attacks.