VMO

a. For $a=0$, we have $$\{\begin{array}{l}x=yz,\\ y=zx,\\ z=xy.\end{array}$$ If one of three numbers is equal to $0$ then the other numbers are equal to $0$ too. We consider the case $xyz\ne 0$ and multiply the equations, side by side, we get $xyz=1$ then $$x^2=y^2=z^2=1.$$ From these identities, we conclude that the solutions of the system are $$(0,0,0),(1,1,1),(-1,-1,1)$$ and permutations. We can easily check that there are 5 different solutions for this system.

b. Clearly, $x=y=z=0$ satisfies the system. For $a>1$, first of all it is easy to see that if any number equals $0$, the other numbers also equal $0$. Now, we try to transform this system to an equation for $z$. $$\{\begin{array}{l}x=ay+yz,\\ y-az=z(ay+yz),\\ z-a(ay+yz)=y(ay+yz),\end{array}\iff \{\begin{array}{l}x=ay+yz,\\ y=\frac{az}{1-az-z^2},\\ z-a(ay+yz)=y(ay+yz).\end{array}$$ Hence, $$z-(a^2+az)\frac{az}{1-az-z^2}=(a+z){\left(\frac{az}{1-az-z^2}\right)}^2$$ or $$z^4+(a^2+2a)z^3+2(a^3-1)z^2+(a^4-a^3-a^2-2a)z+(1-a^3)=0.$$ Note that the original system has two roots $x=y=z=0$ and $x=y=z=1-a$ so the above equation must have a root $z=1-a$. Hence, we have $$z^3+(a^2+a+1)z^2+(a^3-1)z-a^2-a-1=0.$$ Consider the polynomial $f(z)=z^3+(a^2+a+1)z^2+(a^3-1)z-a^2-a-1$. By Rolle's theorem, we have $$f(-\infty )<0,f(-2a)>0,f(0)<0,f(+\infty )>0$$ which means $f$ has three distinct real roots. Note that from the simplified system, it is clear that when we get $z$; $x$, $y$ are identified uniquely. Therefore, the system has exactly 5 different solutions when $a>1$. $\square$