The correct option is C (−∞,1]
f(x)=cosx−x∫0f′(t)(2cost+cos2t)dt
Differentiating w.r.t. x, we get
⇒f′(x)=−sinx−f′(x)(2cosx+cos2x)⇒f′(x)(1+cosx)2=−sinx⇒f′(x)=−sinx(1+cosx)2
Integrating w.r.t. x. we get
⇒∫f′(x) dx=−∫sinx(1+cosx)2 dx
Assuming 1+cosx=t⇒−sinx dx=dt,
⇒f(x)=∫1t2 dx⇒f(x)=−1t+Cf(x)=−11+cosx+C
Now, we know that f(0)=1, so
C=32⇒f(x)=32−11+cosx
We know that,
cosx∈[−1,1]⇒1+cosx∈[0,2]⇒11+cosx∈[12,∞)∴f(x)∈(−∞,1]