Many computing applications in statistics require computation of a generalized inverse A('-) of given matrix A. Computation of A('-) using floating-point arithmetic on digital computers sometimes produces results which contain substantial amounts of error in some or all of the matrix elements. Error is introduced because floating-point arithmetic is not exact. Multiple modulus residue arithmetic and finite segment p-adic number arithmetic, however, are capable of producing exact results in many cases with only the assumption that matrix has rational entries. Methods which use multiple modulus residue arithmetic and finite segment p-adic number arithmetic are developed to compute a reflexive generalized inverse for an arbitrary rectangular matrix. The methods are then applied to find exact solution to linear programming problems. The comparisons are made for these two methods, the relative advantages of each method are pointed out. Since a large amount of memory and execution time are needed for performing these two methods, a procedure which extracts the exact solution of the linear system ax = b from its computed floating-point approximation is developed as an alternative. The error bounds of computed solution and det(A) are derived. Also, an overall error-free computational procedure is proposed for solving the linear system of equations. Algorithms are developed and implemented which compute the exact solution for matrix inversion, linear programming problems and linear systems.