jmc

Let $t=\frac{x}{y}+\frac{y}{x}.$ Then we have $$t^2={\left(\frac{x}{y}+\frac{y}{x}\right)}^2=\frac{x^2}{y^2}+2+\frac{y^2}{x^2},$$so $t^2-2=\frac{x^2}{y^2}+\frac{y^2}{x^2},$ and the equation becomes $$3=k^2(t^2-2)+kt.$$Rearranging, we have the quadratic $$0=k^2{t}^2+kt-(2k^2+3).$$By the quadratic formula, $$t=\frac{-k\pm \sqrt{k^2+4k^2(2k^2+3)}}{2k^2}=\frac{-1\pm \sqrt{8k^2+13}}{2k}.$$Since $x$ and $y$ are positive, $t$ is also positive, and furthermore, $$t=\frac{x}{y}+\frac{y}{x}\ge 2\sqrt{\frac{x}{y}\cdot \frac{y}{x}}=2$$by AM-GM. Therefore, the above equation must have a root in the interval $[2,\infty ).$ It follows that $$\frac{-1+\sqrt{8k^2+13}}{2k}\ge 2.$$Multiplying both sides by $2k$ and adding $1,$ we get $\sqrt{8k^2+13}\ge 4k+1.$ Then $8k^2+13\ge (4k+1{)}^2=16k^2+8k+1,$ so $$0\ge 8k^2+8k-12.$$By the quadratic formula, the roots of $8k^2+8k-12=0$ are $$k=\frac{-8\pm \sqrt{8^2+4\cdot 8\cdot 12}}{2\cdot 8}=\frac{-1\pm \sqrt{7}}{2},$$so $\frac{-1-\sqrt{7}}{2}\le k\le \frac{-1+\sqrt{7}}{2},$ and the maximum value of $k$ is $\boxed{\frac{-1+\sqrt{7}}{2}}.$