Question

Let the number of triangles whose angular points are at the angular points of a given polygon of $n$ sides, but none of whose are the sides of the polygon be $\frac{1}{k}n(n-m)(n-p)$. Find k+m-p ?

Answer 1

A polygon of $n$ sides has $n$ angular points.
Number of triangles formed from these $n$ angular points $={}^n{C}_3$. These are comprised of two exclusive cases
$\left(i\right)$ at least one side of the triangle is a side of the polygon
$\left(ii\right)$ no side of the triangle is a side of the polygon.
Let $AB$ be one side of the polygon. If each angular point of the remaining $(n-2)$ points is joined with $A$ and $B$, we get a triangle with one side $AB$.
$\therefore$ No. of triangles of which $AB$ is one side $=(n-2)$ like wise number of triangles of which $BC$ is one side $=(n-2)$ and of which at least one side is the side of the polygon $=n(n-2)$.
Out of these triangles, some are counted twice. For example, the triangle when $C$ is joined with $AB$ is $△ABC$ is taken as one side. Again triangle formed when $A$ is joined with $BC$ are counted when $BC$ is taken as one side.
Number of such triangles $=n$.
Hence, the number of triangles of which one side is the side of the triangle
$=n(n-2)-n=n(n-3)$
Hence, the total no. of required triangles $={}^n{C}_3-n(n-3)=\frac{1}{6}n(n-4)(n-5)$.