APMO

We prove that there is no such function. For arbitrary elements $a$ and $b$ of $S$, choose an integer $c$ that is greater than both of them. Since $bc>a$ and $c>b$, we have $$f\left(a^4{b}^4{c}^4\right)=f\left(a^2\right)f\left(b^2{c}^2\right)=f\left(a^2\right)f(b)f(c)$$ Furthermore, since $ac>b$ and $c>a$, we have $$f\left(a^4{b}^4{c}^4\right)=f\left(b^2\right)f\left(a^2{c}^2\right)=f\left(b^2\right)f(a)f(c)$$ Comparing these two equations, we find that for all elements $a$ and $b$ of $S$, $$f\left(a^2\right)f(b)=f\left(b^2\right)f(a)⟹\frac{f\left(a^2\right)}{f(a)}=\frac{f\left(b^2\right)}{f(b)}$$ It follows that there exists a positive rational number $k$ such that $$\begin{array}{c}f\left(a^2\right)=kf(a),\text{ for all }a\in S.\end{array}\text{\hspace{1em}(1)}$$ Substituting this into the functional equation yields $$\begin{array}{c}f(ab)=\frac{f(a)f(b)}{k},\text{ for all }a,b\in S\text{ with }a\ne b.\end{array}\text{\hspace{1em}(2)}$$ Now combine the functional equation with equations (1) and (2) to obtain $$f(a)f\left(a^2\right)=f\left(a^6\right)=\frac{f(a)f\left(a^5\right)}{k}=\frac{f(a)f(a)f\left(a^4\right)}{k^2}=\frac{f(a)f(a)f\left(a^2\right)}{k},\text{ for all }a\in S.$$ It follows that $f(a)=k$ for all $a\in S$. Substituting $a=2$ and $b=3$ into the functional equation yields $k=1$, however $1\notin S$ and hence we have no solutions.