imc

Label the bottom left corner of the larger rectangle (without the square cut out) as $A$ and the top right as $D$. $w$ is the width of the rectangle and $\ell$ is the length. Now we have vertices $E,F,G,H$ as vertices of the irregular octagon created by cutting out the squares. Let $I,J$ be the two closest vertices formed by the squares. The distance between the two closest vertices of the squares is thus $IJ=\left(4\sqrt{2}\right).$ Substituting, we get $$(IJ{)}^2=(w-2{)}^2+(\ell -2{)}^2={\left(4\sqrt{2}\right)}^2=32⟹w^2+{\ell }^2-4w-4\ell =24.$$ Using the fact that the diagonal of the rectangle is $8,$ we get $$w^2+{\ell }^2=64.$$ Subtracting the first equation from the second equation, we get $$4w+4\ell =40⟹w+\ell =10.$$ Squaring yields $$w^2+2w\ell +{\ell }^2=100.$$ Subtracting the second equation from this, we get $2w\ell =36,$ and thus area of the original rectangle is $w\ell =\boxed{18}.$