If the space diagonal of a rectangular prism is $d = \sqrt{50}$ and the base is a square of side $3$, what is the height of the prism?