jmc

We compute that $4!=1\times 2\times 3\times 4=2^3\times 3=24$. So we want exactly the numbers in the set $\{1,\dots ,24\}$ which are divisible by neither $2$ nor $3$, since an integer $a$ is invertible modulo $n$ for some positive integer $n$ if and only if $\gcd (a,n)=1$. These turn out to be $\{1,5,7,11,13,17,19,23\}$. Then $$\begin{array}{rl}m& \equiv 1\cdot 5\cdot 7\cdot 11\cdot 13\cdot 17\cdot 19\cdot 23\\ & \equiv 1\cdot 5\cdot 7\cdot 11\cdot (-11)\cdot (-7)\cdot (-5)\cdot (-1)\\ & \equiv (5\cdot 7\cdot 11{)}^2\\ & \equiv (35\cdot 11{)}^2\\ & \equiv (11\cdot 11{)}^2\\ & \equiv (121{)}^2\\ & \equiv 1^2\\ & \equiv \boxed{1}(\mathrm{mod}24)\end{array}$$