Correct option is (B) 2, 6, 18

Let required three numbers in G.P. are a, ar & \(ar^2.\)

\(\therefore\) Their sum \(=a+ar+ar^2\)

\(\therefore\) \(a+ar+ar^2=26\)              (Given)

\(\Rightarrow a(1+r+r^2)=26\)     ______________(1)

Also \(a.ar+ar.ar^2+a.ar^2=156\)     (Given)

\(a^2r(1+r+r^2)=156\)    ______________(2)

Divide equation (2) by (1), we get

\(\frac{a^2r(1+r+r^2)}{a(1+r+r^2)}=\frac{156}{26}\)

\(\Rightarrow ar=6\)                     ______________(3)

Then from (1), we have

\(a+ar+ar.r=26\)

\(\Rightarrow a+6+6r=26\)         \((\because ar=6)\)

\(\Rightarrow a+6r=26-6=20\)

\(\Rightarrow ar+6r^2=20r\)

\(\Rightarrow6+6r^2=20r\)          \((\because ar=6)\)

\(\Rightarrow6r^2-20r+6=0\)

\(\Rightarrow3r^2-10r+3=0\)

\(\Rightarrow3r^2-9r-r+3=0\)

\(\Rightarrow3r(r-3)-1(r-3)=0\)

\(\Rightarrow(r-3)(3r-1)=0\)

\(\Rightarrow r-3=0\;or\;3r-1=0\)

\(\Rightarrow r=3\;or\;r=\frac13\)

From (3), we have

\(a=\frac6r=\frac63=2\) or

\(a=\frac6r=\frac6{\frac13}=18\)

If r = 3 then a = 2

or if \(r=\frac13\) then a = 18

Case I :-

r = 3 & a = 2

\(\therefore ar=2\times3=6\)

\(ar^2=2\times9=18\)

Hence, required number are 2, 6 and 18.

Case II :-

If \(r=\frac13\) & a = 18

\(\therefore ar=\frac{18}3=6\)

& \(ar^2=\frac{18}9=2\)

Hence, required numbers in G.P. are 2, 6 & 18 or 18, 6, 2.

Alternate :-

Let \(a,ar,ar^2\) be required three numbers in G.P.

\(\therefore a+ar+ar^2=26\)

\(\Rightarrow a(1+r+r^2)=26\)         _______________(1)

And \(a.ar+ar.ar^2+ar^2.a=156\)

\(\Rightarrow a^2r(1+r+r^2)=156\)    _______________(2)

Divide equation (2) by (1), we get

\(\frac{a^2r(1+r+r^2)}{a(1+r+r^2)}=\frac{156}{26}=6\)

\(\Rightarrow ar=6\)            _______________(3)

Put ar = 6 in equation (1), we get

\(a+6+6r=26\)

\(\Rightarrow a+6r=26-6=20\)

\(\Rightarrow ar+6r^2=20r\)       (Multiply both sides by r)

\(\Rightarrow6+6r^2-20r=0\)

\(\Rightarrow3r^2-10r+3=0\)

\(\Rightarrow3r^2-9r-r+3=0\)

\(\Rightarrow3r(r-3)-1(r-3)=0\)

\(\Rightarrow(r-3)(3r-1)=0\)

\(\Rightarrow r-3=0\;or\;3r-1=0\)

\(\Rightarrow r=3\;or\;r=\frac13\)

Case I :-

r = 3 then \(a=\frac6r\)

\(=\frac63=2\)

\(\therefore ar=2\times3=6\)

\(ar^2=2\times9=18\)

Case II :-

\(r=\frac13\) then \(a=\frac6r=\frac6{\frac13}\)

\(=6\times3=18\)

\(\therefore ar=18\times\frac13=6\)

and \(ar^2=18\times\frac19=2\)

Hence, required numbers are 2, 6, 18.

Correct option is B) 2, 6, 18