Thinking of the quadratic formula, the roots of the equation are real and unequal whenever B^2 - 4AC > 0, and not real whenever B^2 - 4AC < 0. Pay attention to the capitalization here, or it could get confusing. In the first equation, A = 1, B = 2c, and C = ab, so:
B^2 - 4AC =
(2c)^2 - 4(1)(ab) =
4c^2 - 4ab
So, 4c^2 - 4ab > 0.
In the second equation, A = 1, B = -2(a + b), and C = (a^2 + b^2 + 2c^2).
B^2 - 4AC =
(-2(a + b))^2 - 4(1)(a^2 + b^2 + 2c^2) =
4(a + b)^2 - 4 (a^2 + b^2 + 2c^2) =
4a^2 + 8ab + 4b^2 - 4a^2 - 4b^2 - 8c^2 =
8ab - 8c^2
Essentially, then, we want to prove that if 4c^2 - 4ab > 0, then 8ab - 8c^2 < 0.
4c^2 - 4ab > 0
Multiply the inequality by -2.
-8c^2 + 8ab < 0
8ab - 8c^2 < 0
Therefore, if the roots of x^2 + 2cx + ab = 0 are real and unequal, then the roots of x^2 - 2(a+b)x + a^2 + b^2 + 2c^2 are not real.