XIX OBM

First of all, since $g(f(x))=x^3$ and $x^3$ is injective, $f$ is injective. Indeed, $f(x)=f(y)⟹g(f(x))=g(f(y))⟺x^3=y^3⟺x=y$. Plugging $f(x)$ in $f(g(x))=x^2$, we obtain $f(g(f(x)))=f(x{)}^2⟺f(x^3)=f(x{)}^2$. If $f,g:R\to R$, consider $x=-1,0,1$ (i.e., the roots of $x^3=x$): $f(-1)=f(-1{)}^2$, $f(0)=f(0{)}^2$ and $f(1)=f(1{)}^2$, so $\{f(-1),f(0),f(1)\}\subset \{0,1\}$, which contradicts the fact that $f$ is injective. So in this case there are no such functions $f,g$.

If $f,g:S\to S$, consider $f(x^3)=(f(x){)}^2$ and $g(f(g(x)))=(g(x){)}^3⟺$ $g(x^2)=(g(x){)}^3$. Applying these two identities $n$ times we obtain $f(x^{3^n})=$ $(f(x){)}^{2^n}$ and $g(x^{2^n})=(g(x){)}^{3^n}$. In particular, $f(2^{3^n})=(f(2){)}^{2^n}$ and $g(2^{2^n})=(g(2){)}^{3^n}$. Extend these identities for all $n$ real. We have $x=$ $2^{3^n}⟺n={\log }_{3}2{\log }_{2}x$ and $x=2^{2^n}⟺n={\log }_2{\log }_{2}x$. This means that $f(x)=a^{2^{{\log }_{3}2{\log }_{2}x}}$ and $g(x)=b^{2^{{\log }_{2}2{\log }_{2}x}}$. For the sake of simplicity, write all equations in base $2$: $f(x)=2^{2^{{\log }_{3}2{\log }_2{\log }_{2}x+k}}$ and $g(x)=2^{2^{{\log }_{2}3{\log }_2{\log }_{2}x+\ell }}$. Now substitute in the original equations: $$\begin{array}{rl}f(g(x))& =f(2^{2^{{\log }_{2}3{\log }_2{\log }_{2}x+\ell }})=2^{2^{{\log }_{3}2{\log }_2{\log }_2{2}^{2^{{\log }_{2}3{\log }_2{\log }_{2}x+\ell }}}+k}=2^{2^{{\log }_{3}2\cdot ({\log }_{2}3{\log }_2{\log }_{2}x+\ell )+k}}\\ & =2^{2^{{\log }_2{\log }_{2}x+\ell }{\log }_{3}2+k}=2^{2^{{\log }_2{\log }_{2}x}\cdot 2^{\ell }{\log }_{3}2+k}=(2^{2^{{\log }_2{\log }_{2}x}}{)}^{2^{\ell }{\log }_{3}2+k}=x^{2^{\ell }{\log }_{3}2+k}\end{array}$$ $$\text{so }{2}^{\ell {\log }_{3}2+k}=2⟺\ell {\log }_{3}2+k=1.$$ $$\begin{array}{rl}g(f(x))& =g(2^{2^{{\log }_{3}2{\log }_2{\log }_{2}x+k}})=2^{2^{{\log }_{2}3{\log }_2{\log }_2{2}^{2^{{\log }_{3}2{\log }_2{\log }_{2}x+k}}}+\ell }=2^{2^{{\log }_{2}3\cdot ({\log }_{3}2{\log }_2{\log }_{2}x+k)+\ell }}\\ & =2^{2^{{\log }_2{\log }_{2}x+k}{\log }_{2}3+\ell }=2^{2^{{\log }_2{\log }_{2}x}\cdot 2^k{\log }_{2}3+\ell }=(2^{2^{{\log }_2{\log }_{2}x}}{)}^{2^k{\log }_{2}3+\ell }=x^{2^k{\log }_{2}3+\ell }\end{array}$$ $$\text{so }{2}^{k{\log }_{2}3+\ell }=3⟺k{\log }_{2}3+\ell ={\log }_{2}3.$$ It is clear that both $f$ and $g$ are well defined in $S$ and that it's possible to choose $k$ and $\ell$ (for example, $k=1$ and $\ell =0$), so such functions do exist.