jmc

Consider the points on the complex plane. The point $b+37i$ is then a rotation of $60$ degrees of $a+11i$ about the origin, so: $$(a+11i)\left(\mathrm{cis}60^{\circ }\right)=(a+11i)\left(\frac{1}{2}+\frac{\sqrt{3}i}{2}\right)=b+37i.$$ Equating the real and imaginary parts, we have: $$\begin{array}{rl}b& =\frac{a}{2}-\frac{11\sqrt{3}}{2}\\ 37& =\frac{11}{2}+\frac{a\sqrt{3}}{2}\end{array}$$ Solving this system, we find that $a=21\sqrt{3},b=5\sqrt{3}$. Thus, the answer is $\boxed{315}$. Note: There is another solution where the point $b+37i$ is a rotation of $-60$ degrees of $a+11i$; however, this triangle is just a reflection of the first triangle by the $y$-axis, and the signs of $a$ and $b$ are flipped. However, the product $ab$ is unchanged.