SAMC

The inequality is equivalent to $${\left(\frac{AB}{BC}-\frac{AC}{BC}\right)}^2{\left(1+4\frac{AB}{BC}\cdot \frac{AC}{BC}\right)}^2\le 2,$$ hence $$(\cos B-\sin B{)}^2(1+4\cos B\sin B{)}^2\le 2,$$ that is $$(1-\sin 2B)\left(1+4\sin 2B+4{\sin }^{2}2B\right)\le 2.$$ After easy computations we get the equivalent inequality $$(1+\sin 2B)(2\sin 2B-1{)}^2\ge 0.$$ We have equality if and only if $\sin 2B=\frac{1}{2}$, i.e. $\hat{B}=\frac{\pi }{12}$.

---

Alternative solution.

Let $\alpha =\frac{AB}{BC},\beta =\frac{AC}{BC}$. We have ${\alpha }^2+{\beta }^2=1$, and the inequality is equivalent to: $$(\alpha -\beta {)}^2(1+4\alpha \beta {)}^2\le 2,$$ hence $$(1-2\alpha \beta )(1+4\alpha \beta {)}^2\le 2.$$ Put $x=2\alpha \beta$ and obtain $$(1-x)(1+2x{)}^2\le 2\text{ or }4x^2-3x+1\ge 0,$$ which is equivalent to $(2x-1{)}^2(x+1)\ge 0$.