The correct option is B a>b
Let (1−x+x2)20=a0+a1x+.....+a40x40
⇒(1+x+x2)20=a0−a1x+a2x2−......+a40x40
Thus, coefficient of x20 in (1−x+x2)20
= coefficient of x20 in (1−x+x2)20
Therefore, we consider the expansion
(1−x+px2)20=b0+b1x+b2x2+........+b40x40
Dividing both the sides by x20, we get
b20= coefficient of constant term in
(1x+1+px)20
But (1x+1+px)20=1+20C1(1x+px)+20C2(1x+px)2+....+20C20(1x+px)20
∴b20=1+(20C2)(2C1)p+(20C4)(4C2)p2+......+(20C20)(20C10)p10
Thus,
a=[b20]p=1 and a=[b20]p=−1
Hence, a>b