In \[\Delta ABC\], then find the value \[\left( {\cot \dfrac{A}{2} + \cot \dfrac{B}{2}} \right)\left( {a{{\sin }^2}\dfrac{B}{2} + b{{\sin }^2}\dfrac{A}{2}} \right)\].
A. \[\cot C\]
B. \[c\cot C\]
C. \[\cot \dfrac{C}{2}\]
D. \[c\cot \dfrac{C}{2}\]

Hint First we will simplify the \[\cot \dfrac{A}{2}\] and \[\cot \dfrac{B}{2}\]. Then we will apply trigonometric functions of half angles of triangles and simplify the expression to reach the solution.

Formula use
Trigonometric functions of half angles
\[\sin \dfrac{A}{2} = \sqrt {\dfrac{{\left( {s - b} \right)\left( {s - c} \right)}}{{bc}}} \]
\[\sin \dfrac{B}{2} = \sqrt {\dfrac{{\left( {s - a} \right)\left( {s - c} \right)}}{{ac}}} \]
\[\sin \dfrac{C}{2} = \sqrt {\dfrac{{\left( {s - a} \right)\left( {s - b} \right)}}{{ab}}} \]
Sum of sin rule: \[\sin A\cos B + \cos A\sin B = \sin \left( {A + B} \right)\]
Complement of trigonometry angles: \[\sin \left( {\dfrac{\pi }{2} - \theta } \right) = \cos \theta \]

Complete step by step solution
Given expression is
\[\left( {\cot \dfrac{A}{2} + \cot \dfrac{B}{2}} \right)\left( {a{{\sin }^2}\dfrac{B}{2} + b{{\sin }^2}\dfrac{A}{2}} \right)\]
Apply the formula \[\cot \theta = \dfrac{{\cos \theta }}{{\sin \theta }}\]
\[ = \left( {\dfrac{{\cos \dfrac{A}{2}}}{{\sin \dfrac{A}{2}}} + \dfrac{{\cos \dfrac{B}{2}}}{{\sin \dfrac{B}{2}}}} \right)\left( {a{{\sin }^2}\dfrac{B}{2} + b{{\sin }^2}\dfrac{A}{2}} \right)\]
\[ = \left( {\dfrac{{\cos \dfrac{A}{2}\sin \dfrac{B}{2} + \cos \dfrac{B}{2}\sin \dfrac{A}{2}}}{{\sin \dfrac{A}{2}\sin \dfrac{B}{2}}}} \right)\left( {a{{\sin }^2}\dfrac{B}{2} + b{{\sin }^2}\dfrac{A}{2}} \right)\]
Now we apply the formula \[\sin A\cos B + \cos A\sin B = \sin \left( {A + B} \right)\]
\[ = \left( {\dfrac{{\sin \left( {\dfrac{A}{2} + \dfrac{B}{2}} \right)}}{{\sin \dfrac{A}{2}\sin \dfrac{B}{2}}}} \right)\left( {a{{\sin }^2}\dfrac{B}{2} + b{{\sin }^2}\dfrac{A}{2}} \right)\] …(i)
We know, \[A + B + C = \pi \]
Divide both sides by 2
\[ \Rightarrow \dfrac{A}{2} + \dfrac{B}{2} + \dfrac{C}{2} = \dfrac{\pi }{2}\]
\[ \Rightarrow \dfrac{A}{2} + \dfrac{B}{2} = \dfrac{\pi }{2} - \dfrac{C}{2}\]
Now substitute \[\dfrac{A}{2} + \dfrac{B}{2} = \dfrac{\pi }{2} - \dfrac{C}{2}\] in (i)
\[ = \left( {\dfrac{{\sin \left( {\dfrac{\pi }{2} - \dfrac{C}{2}} \right)}}{{\sin \dfrac{A}{2}\sin \dfrac{B}{2}}}} \right)\left( {a{{\sin }^2}\dfrac{B}{2} + b{{\sin }^2}\dfrac{A}{2}} \right)\]
Now Applying the formula \[\sin \left( {\dfrac{\pi }{2} - \theta } \right) = \cos \theta \]
\[ = \left( {\dfrac{{\cos \dfrac{C}{2}}}{{\sin \dfrac{A}{2}\sin \dfrac{B}{2}}}} \right)\left( {a{{\sin }^2}\dfrac{B}{2} + b{{\sin }^2}\dfrac{A}{2}} \right)\]
\[ = \cos \dfrac{C}{2}\left( {\dfrac{{a{{\sin }^2}\dfrac{B}{2}}}{{\sin \dfrac{A}{2}\sin \dfrac{B}{2}}} + \dfrac{{b{{\sin }^2}\dfrac{A}{2}}}{{\sin \dfrac{A}{2}\sin \dfrac{B}{2}}}} \right)\]
\[ = \cos \dfrac{C}{2}\left( {\dfrac{{a\sin \dfrac{B}{2}}}{{\sin \dfrac{A}{2}}} + \dfrac{{b\sin \dfrac{A}{2}}}{{\sin \dfrac{B}{2}}}} \right)\]
Now applying the formula \[\sin \dfrac{A}{2} = \sqrt {\dfrac{{\left( {s - b} \right)\left( {s - c} \right)}}{{bc}}} \] and \[\sin \dfrac{B}{2} = \sqrt {\dfrac{{\left( {s - a} \right)\left( {s - c} \right)}}{{ac}}} \]
\[ = \cos \dfrac{C}{2}\left( {a\dfrac{{\sqrt {\dfrac{{\left( {s - a} \right)\left( {s - c} \right)}}{{ac}}} }}{{\sqrt {\dfrac{{\left( {s - b} \right)\left( {s - c} \right)}}{{bc}}} }} + b\dfrac{{\sqrt {\dfrac{{\left( {s - b} \right)\left( {s - c} \right)}}{{bc}}} }}{{\sqrt {\dfrac{{\left( {s - a} \right)\left( {s - c} \right)}}{{ac}}} }}} \right)\]
\[ = \cos \dfrac{C}{2}\left( {a\sqrt {\dfrac{{\left( {s - a} \right)\left( {s - c} \right)}}{{ac}} \times \dfrac{{bc}}{{\left( {s - b} \right)\left( {s - c} \right)}}} + b\sqrt {\dfrac{{\left( {s - b} \right)\left( {s - c} \right)}}{{bc}} \times \dfrac{{ac}}{{\left( {s - a} \right)\left( {s - c} \right)}}} } \right)\]
\[ = \cos \dfrac{C}{2}\left( {a\sqrt {\dfrac{{b\left( {s - a} \right)}}{{a\left( {s - b} \right)}}} + b\sqrt {\dfrac{{a\left( {s - b} \right)}}{{b\left( {s - a} \right)}}} } \right)\]
\[ = \cos \dfrac{C}{2}\left( {\sqrt {ab} \sqrt {\dfrac{{\left( {s - a} \right)}}{{\left( {s - b} \right)}}} + \sqrt {ab} \sqrt {\dfrac{{\left( {s - b} \right)}}{{\left( {s - a} \right)}}} } \right)\]
\[ = \sqrt {ab} \cos \dfrac{C}{2}\left( {\dfrac{{s - a + s - b}}{{\sqrt {\left( {s - a} \right)\left( {s - b} \right)} }}} \right)\]
\[ = \sqrt {ab} \cos \dfrac{C}{2}\left( {\dfrac{{2s - a - b}}{{\sqrt {\left( {s - a} \right)\left( {s - b} \right)} }}} \right)\]
We know that, \[2s = a + b + c\] which implies \[2s - a - b = c\]
\[ = \sqrt {ab} \cos \dfrac{C}{2}\left( {\dfrac{c}{{\sqrt {\left( {s - a} \right)\left( {s - b} \right)} }}} \right)\]
\[ = \cos \dfrac{C}{2}\left( {\dfrac{c}{{\sqrt {\dfrac{{\left( {s - a} \right)\left( {s - b} \right)}}{{ab}}} }}} \right)\]
Now applying \[\sin \dfrac{C}{2} = \sqrt {\dfrac{{\left( {s - a} \right)\left( {s - b} \right)}}{{ab}}} \]
\[ = \cos \dfrac{C}{2}\left( {\dfrac{c}{{\sin \dfrac{C}{2}}}} \right)\]
Hence option D is the correct option.

Note Students can remember the simple trick for the trigonometric functions half-angle formulas i.e. \[\sin \dfrac{A}{2} = \sqrt {\dfrac{{\left( {s - b} \right)\left( {s - c} \right)}}{{bc}}} \]; If with sin the angle A is taken on L.H.S then on R.H.S b and c will be used, similarly, if B is used on L.H.S then on R.H.S, a and c will be used and if C is used on L.H.S then on R.H.S a and b will be used.