Right option is (c) \(\BEGIN{BMATRIX}\frac{-1}{4}&\frac{1}{4}&0\\ \frac{1}{40}&\frac{-1}{8}&\frac{1}{5}\\ \frac{3}{40}&0&\frac{-1}{10}\END{bmatrix}\)

For explanation: Consider the MATRIX A=\(\begin{bmatrix}1&2&4\\5&2&4\\3&6&2\end{bmatrix}\)

Using the elementary ROW operation, we write A=IA

\(\begin{bmatrix}1&2&4\\5&2&4\\3&6&2\end{bmatrix}\)=\(\begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix}\)A

Applying R1→R1-R2

\(R_1 \rightarrow \frac{R_1}{-4}\)

\(\begin{bmatrix}1&0&0\\5&2&4\\3&6&2\end{bmatrix}\)=\(\begin{bmatrix}\frac{-1}{4}&\frac{1}{4}&0\\0&1&0\\0&0&1\end{bmatrix}\)A

Applying R2→R2-5R1and R3→R3-3R1

\(\begin{bmatrix}1&0&0\\0&2&4\\0&6&2\end{bmatrix}\)=\(\begin{bmatrix}\frac{-1}{4}&\frac{1}{4}&0\\\frac{5}{4}&\frac{-1}{4}&0\\\frac{3}{4}&\frac{-3}{4}&1\end{bmatrix}\)A

Applying R2→R2-2R3and \(R_2 \rightarrow \frac{R_2}{-10}\)

\(\begin{bmatrix}1&0&0\\0&1&0\\0&6&2\end{bmatrix}\)=\(\begin{bmatrix}\frac{-1}{4}&\frac{1}{4}&0\\ \frac{1}{40}&\frac{-5}{40}&\frac{1}{5}\\\frac{3}{4}&\frac{-3}{4}&1\end{bmatrix}\)A

Applying R3→R3-6R2and \(R_2 \rightarrow \frac{R_2}{2}\)

\(\begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix}\)=\(\begin{bmatrix}\frac{-1}{4}&\frac{1}{4}&0\\ \frac{1}{40}&\frac{-1}{8}&\frac{1}{5}\\ \frac{3}{40}&0&\frac{-1}{10}\end{bmatrix}\)A

A^-1=\(\begin{bmatrix}\frac{-1}{4}&\frac{1}{4}&0\\ \frac{1}{40}&\frac{-1}{8}&\frac{1}{5}\\ \frac{3}{40}&0&\frac{-1}{10}\end{bmatrix}\).