smc

If we fill every third chair with a person, then the condition is satisfied, giving $N=20$. Decreasing $N$ any further means there is at least one gap of $4$, so that the person can sit themselves in the middle (seat $2$ of $4$) and not be next to anyone. Hence the minimum value of $N$ is $20$. $\boxed{20}$