Childs father in or out

So my sons dad has never been much of a dad. We split just before he was one. I done all the night feeds and paid for everything when we was together. He was a good dad but could have done wat more. We split just before he was one (he's nearly three) and he has been nothingn but trouble since. he didnt want to know him in the beginning but then he come round to the idea. He only sees him because I take him there he will not go out of his way. He does not help financially. He has said some very nasty things about myself and my son. Whenever he doent like something he threatens things like I won't see him, I won't do this I won't do that. Whenever I have stopped him seeing him he's never really been bothered he hust leaves it a couple of weekes before asking for contact again. He teaches him bad things, he tells him he's not his dad (which he 110% is) he's not a good role model as he's picked up bad habits since we have split and turned into a different person. I do not want him around my son any more. He doesn't care about him only when it suits him. Whenever I stop contact I'm made to feel guilty or people think it's because I want my new partner to be his dad which is not the case (although he is a better role model and does idolise my son) opinions on whether u agree it's best to stop contact or wait to see if my son decides he wants to stop seeing him when he's older? Or see if he changes (he hasn't in 2 years). Opinions please