Low motile sperm count, chances of conceiving naturally?

Hi,

i have a daughter 2 and half years old, I conceived naturally on my 6th cycle of trying with her. We've been trying for 19 months for our second, just had tpops and first consultation at the fertility clinic, have been told that hubby has 3.5 million motile count, our consultant said anything under 5 million will be referred to IVF, his sample is being repeated and we will be referred if its not significantly better. Just wondering if anyone knew the chances of us conceiving naturally with this count? Not sure if we got damn lucky with our first or if something has happened to drastically reduce his count.

thanks