Austrian Mathematical Olympiad

First, we set $y=(x^2-f(x))/2$ which gives $\alpha f(x)(x^2-f(x))/2=0$. Now, we distinguish the cases $\alpha \ne 0$ and $\alpha =0$.

a. In the case $\alpha \ne 0$, this equation implies $f(x)=0$ or $f(x)=x^2$ for each $x$ separately. In particular, $f(0)=0$.

◦ It is easily verified that $f(x)=0$, $x\in \mathbb{R}$, is a solution.

◦ Next, we investigate the function $f(x)=x^2$, $x\in \mathbb{R}$. The functional equation becomes $$(x^2+y{)}^2=(x^2-y{)}^2+\alpha x^{2}y⟺2x^{2}y=-2x^{2}y+\alpha x^{2}y⟺(\alpha -4)x^{2}y=0$$ which holds for all $x$ and $y$ exactly when $\alpha =4$. Therefore, for $\alpha =4$, there is an additional solution $f(x)=x^2$, $x\in \mathbb{R}$.

◦ It remains to investigate the case, where there are numbers $x,y\in \mathbb{R}\setminus \{0\}$ with $f(y)=y^2$ and $f(x)=0$. Suppose that $x$ and $y$ were two such numbers. Then the original functional equation becomes $f(y)=f(x^2-y)$. Because of $f(y)=y^2\ne 0$, we have $f(x^2-y)\ne 0$ and therefore $f(x^2-y)=(x^2-y{)}^2$. This implies $$y^2=(x^2-y{)}^2=x^4-2x^{2}y+y^2,$$ i.e., $y=x^2/2$, so that $y$ is the only number with $f(y)=y^2$ and $f(z)=0$ for all $z\in \mathbb{R}\setminus \{y\}$. Repeating this argument, we obtain $y=z^2/2$ for all $z\in \mathbb{R}\setminus \{y\}$, a contradiction.

b. For $\alpha =0$, the functional equation becomes $$f(f(x)+y)=f(x^2-y).(2)$$ It is easy to check that constant functions and the function $f(x)=-x^2$ are solutions.

Now, assume that there is a real number $a$ with $f(a)=b\ne -a^2$. We define $d=b+a^2\ne 0$. Putting $x=a$ in the functional equation (2) gives $f(b+y)=f(a^2-y)$ for all $y\in \mathbb{R}$. With $y=z-b$, we obtain $f(z)=f(d-z)$ for all $z\in \mathbb{R}$. Using $x=z$ and $x=d-z$ in the functional equation (2), we get $$f(z^2-y)=f(f(z)+y)=f(f(d-y)+y)=f((d-z{)}^2-y).$$ Therefore, $$f(z^2-y)=f((d-z{)}^2-y)$$ for all real numbers $y$ and $z$. With $y=z^2$, we obtain $f(0)=f(d^2-2dz)$ for all $z\in \mathbb{R}$. Because of $d\ne 0$ the second argument attains all real numbers, so that $f$ is constant. This proves that there are no other solutions.