The correct option is A a=−3andb≠1/3
Given system of equations are ⎡⎢⎣3−215−8921a⎤⎥⎦⎡⎢⎣xyz⎤⎥⎦=⎡⎢⎣b3−1⎤⎥⎦
system of equations has no solution if rank of coefficient matrix and augumented matrix are diffferent.
⇒∣∣
∣∣3−215−8921a∣∣
∣∣=0
⇒3(−8a−9)+2(5a−18)+21=−14a−42=0
∴a=−3
⇒∣∣
∣∣3−2b5−8321−1∣∣
∣∣≠0
⇒−7+21b≠0
∴b≠1/3
Hence, given system has no solution if a=−3 and b≠1/3
Hence, option A.