If the harmonic mean of two positive numbers $a$ and $b$ is $4$ and their geometric mean is $\sqrt{20}$, what is their arithmetic mean?