x+y+z=4 ...(1)
2x+y+3z=6 ...(2)
x+2y+pz=q ...(3)
Solving (1) and (2),
x=2−2z and y=2+z
Putting in equation (3), we get
pz=q−6
For unique solution, p≠0,q∈R
For no solution, we must have p=0,q≠6
For infinite solution, p=0 and q=6
∴p′=0 and q′=6
⇒(p′)2+(q′)2=0+36=36