Is it only when the direction of electric current and the direction of magnetic field are perpendicular, that the current carrying conductor experiences a force?
No, the conductor will experience a force even when it is at an angle to the magnetic field. Although, the force acting will depend upon the angle as
F = B I L sinθ
where
B = Magnitude of magnetic field
I = Current
L = Length of conductor
θ = Angle between the conductor and magnetic field.
when the angle between the two is 0, sinθ = sin 0 = 0. In this case there will be no force on the conductor.