69th Belarusian Mathematical Olympiad

Answer: no. Let $b=1$ and let $a$ be equal to some irreducible fraction $p/q$ with the denominator $q$ such that the squares of integers are never congruent to $q-1$ modulo $q$ (i.e., $-1$ is a quadratic nonresidue modulo $q$). For example, let $a=1/3$. Transform the expression from the problem condition: $$(am+b{)}^2+(a+nb{)}^2={\left(\frac{m}{3}+1\right)}^2+{\left(\frac{1}{3}+n\right)}^2=\frac{m^2+6m+1+6n}{9}+n^2+1.$$ For this number to be integer, $m^2+1$ must be divisible by $3$, which is impossible. Hence for $a=1/3$ and $b=1$ such integers $m$ and $n$ don't exist.