Will our child inevitably have a disability?

My significant other and I are trying to conceive.

I am 25 years old and my partner is 52. He has 3 kids from a previous relationship, we have been together for 3 years now. We are keen on having a baby together, we have been told by our close friends that our baby would “have problems” because of my partners age and health. 

My 52 year old partner was obese, he is still big but he has lost a few stone in the past year, plus he has been stressed and anxious just lately which makes the weight drop off even more. I would say his average daily intake of food would consist of 50% healthy and 50% sort-of-healthy-but-on-the-brink-of-bad-junk-food. He doesn’t smoke, he has never touched a cigarette nor has he taken drugs. As ridiculous as it sounds, I’m not aware if he’s on tablets for his heart... he hides many stuff including his feelings and upsets so even if I asked him I would most likely get blanked (he’s stubborn). But would heart tablets affect the quality of the sperm? I need to know. He goes down the pub every weekend with his mates, he gets drunk every now and then. 

I’m in really good health. I have asthma and reoccurring wheezing but it’s mild and it’s not enough to limit my daily activities. I go for plenty of walks and my partner comes with me, he does park “runs” (he just fast walks the whole way)... we are pretty active but more so me, as he is older and not really in the best of health but he is WAY better than he was last year. 

Another question, does sperm quality get better if a man cuts back on drinking? Does the quality get better if he loses weight? 

He is as eager as me to try for a baby and he has came really far... I know every pregnancy has a risk but we are scared due to what others have put in our minds.. we were always told “you can’t have his kids!” and “do you want children with problems?”

We don’t know what to believe anymore! Can I have some input please? 

Thank you!!