The given inequality may be written in the form
7c2−6(a+b)c−(a2+b2−6ab)<0.
Putting x=7c2,y=−6(a+b)c,z=−(a2+b2−6ab), we have to prove that x+y+z<0.
Observe that x,y,z are not all equal (x>0,y<0).
Using the identity x3+y3+z3−3xyz=12(x+y+z)[(x−y)2+(y−z)2+(z−x)2],
we infer that it is sufficient to prove x3+y3+z3−3xyz<0.
Substituting the values of x,y,z, we see that this is equivalent to
343c6−216(a+b)3c3−(a2+b2−6ab)3−126c3(a+b)(a2+b2−6ab)<0.
Using c3=a3+b3, this reduces to 343(a3+b3)2−216(a+b)3(a3+b3)−(a2+b2−6ab)3−126((a3+b3)(a+b)(a2+b2−6ab)<0.
This may be simplified (after some tedious calculations) to,
−a2b2(129a2−254ab+129b2)<0.
But 129a2−254ab+129b2=129(a−b)2+4ab>0. Hence the result follows.
Remark: The best constant θ in the inequality a2+b2−c2≥θ(c−a)(c−b), where a,b,c
are positive reals such that a3+b3=c3, is θ=2(1+21/3+2−1/3).