Ex is drug user and dealer, yet judge wants to give parental responsibility and contact?!

My ex partner is a drug dealer, and a heavy drug user. During my pregnancy he became abusive, mentally and twice physically during the pregnancy. 

Evidence I have is his toxicology report - shows heavy drug use to lighter most recent (he delayed the drug test by 2 - 3 months as well!) and he told my midwife he was a drug dealer (she is head of midwives) which is in her statement. Also evidence: he used to call the hospital, any maternal department and say I was crazy and should be locked up. He called me a vessel for carrying his child and that he will take her from me. I have this on text messages. And more.

Yet I got an email from my solicitor today that stated the judge wanted to give parental responsibility and allow contact? What the actual f? I've had social services involved, an independent domestic violence advisor...what is going on. I've left the UK to be safe from him and his suppliers AND I have a non molestation against him. I cannot even get my head around this.