jmc

Since no two of the four integers are congruent modulo 6, they must represent four of the possible residues $0,1,2,3,4,5$.

None of the integers can be $0(\mathrm{mod}6)$, since this would make their product a multiple of 6.

The remaining possible residues are $1,2,3,4,5$. Our integers must cover all but one of these, so either 2 or 4 must be among the residues of our four integers. Therefore, at least one of the integers is even, which rules out also having a multiple of 3 (since this would again make $N$ a multiple of 6). Any integer that leaves a remainder of 3 (mod 6) is a multiple of 3, so such integers are not allowed.

Therefore, the four integers must be congruent to $1,2,4,$ and $5(\mathrm{mod}6)$. Their product is congruent modulo 6 to $1\cdot 2\cdot 4\cdot 5=40$, which leaves a remainder of $\boxed{4}$.