The correct option is C −2x2+12xy+6y2
Consider the first expression,(x+y)(5x−y)=x(5x−y)+y(5x−y)
=5x2−xy+5xy−y2
=5x2+4xy−y2-----(1)
Consider the second expression,
(3x+y)(x+5y)=3x(x+5y)+y(x+5y)
=3x2+15xy+xy+5y2
=3x2+16xy+5y2----(2)
Subtracting equation (1) from equation (2),
=(3x2+16xy+5y2)−(5x2+4xy−y2)
Grouping the like terms together,
=(3x2−5x2)+(16xy−4xy)+(5y2+y2)
=−2x2+12xy+6y2