Baby Number 2, and PCOS

I am 27 and have Polycystic Ovary, I have a son who's 2 and we now want to try for baby number 2. When trying for my son I was lucky as I caught within 7 months and everything fine but at the time I was taking metformin tablets. when I caught pregnant with my son I was advised to stop taking them, once I had my son and seen doctor for check up she advised I no longer needed them and recently asked question again and got same response I don't need them. I was on the yasmin contraceptive pill but now come of it to try for baby number two. Ive not had scan to check on my pcos since being diagnosed over 5 years ago.

Advice needed please. Do I try without the metformin see if I can catch or do I discuss metformin again with my doctor?