Here’s where the diagonal formula comes from and why it works. Each diagonal connects one point to another point in the polygon that isn’t its next-door neighbor.
In an n-sided polygon, you have n starting points for diagonals. And each diagonal can go to (n – 3) ending points because a diagonal can’t end at its own starting point or at either of the two neighboring points.
So the first step is to multiply n by (n – 3).
Then, because each diagonal’s ending point can be used as a starting point as well, the product n(n – 3) counts each diagonal twice. That’s why you divide by 2.
So number of diagonals becomes n(n-3)/2
Hope this helps.
Good luck.