Estonian Math Competitions

The ratio of the inradius and the circumradius of a regular $n$-gon is $\cos \frac{180^{\circ }}{n}$. Hence the ratio of the areas of the incircle and the circumcircle of a regular $n$-gon is ${\cos }^2\frac{180^{\circ }}{n}$.

Let a regular $n$-gon and a regular $m$-gon with a common circumcircle be given. W.l.o.g., let $n\le m$ and the area of the common circumcircle be $1$. By the above, the areas of the incircles of these polygons are ${\cos }^2\frac{180^{\circ }}{n}$ and ${\cos }^2\frac{180^{\circ }}{m}$, respectively. As their sum must equal the area of the common circumcircle, we get the equation

$${\cos }^2\frac{180^{\circ }}{n}+{\cos }^2\frac{180^{\circ }}{m}=1$$

which is equivalent to

$${\cos }^2\frac{180^{\circ }}{n}={\sin }^2\frac{180^{\circ }}{m}$$

As both $n$ and $m$ are larger than $2$, both $\frac{180^{\circ }}{n}$ and $\frac{180^{\circ }}{m}$ are less than $90^{\circ }$, whence the equation reduces to

$$\cos \frac{180^{\circ }}{n}=\sin \frac{180^{\circ }}{m}$$

Thus

$$\frac{180^{\circ }}{n}+\frac{180^{\circ }}{m}=90^{\circ }$$

implying that

$$\frac{1}{n}+\frac{1}{m}=\frac{1}{2}$$

If $n=3$ then $m=6$; if $n=4$ then $m=4$; if $n>4$ then $m<4$, contradicting the assumption $n\le m$.