My Partners son is horrible to my daughter

I've been with my partner for 2 years and I've tried really hard to get along with his 7 year old son I get on well with his 13 year old daughter. His son is horrible to my 3 year old daughter and I dread when he comes to stay. My daughter dotes on his boy which makes it worse. He's always telling his dad he doesn't want my daughter near him and always nasty to her. My partner tries to tell him to be nice but falls on death ears I'm at the point I will end up ending the relationship or taking my daughter out the house while he is there stay n