The correct option is D a singleton set
The given set of equation can be written in matrix form as:
⎡⎢⎣1111a1ab1⎤⎥⎦⎡⎢⎣xyz⎤⎥⎦=⎡⎢⎣110⎤⎥⎦
Since this is a non-homogeneous equation, the determinant of the co-efficient matrix should be 0 for no solution to exist.
∴∣∣
∣∣1111a1ab1∣∣
∣∣=0
⇒1(a−b)−1(1−a)+1(b−a2)=0
⇒a−b−1+a+b−a2=0
⇒2a−1−a2=0
⇒−(a−1)2=0
⇒a=1
For a=1, the equation become:
x+y+z=1
x+y+z=1
x+by+z=0
From the above three equations, we can see that if b=1, the system will be inconsistent and hence will produce no solution. For b≠1, the system will produce infinite solutions.
Hence, for no solution, S has to be a singleton set