Introduction and Error Analysis
Numerical methods are algorithms that approximate solutions to mathematical problems when analytical methods are impractical or impossible. They play a central role in scientific computing, engineering, finance, and computer graphics. Every digital computer is limited to finite precision, so understanding errors and their propagation is the first step in numerical analysis.
Why Numerical Methods?
Most real-world mathematical problems do not have closed-form solutions. Nonlinear equations, large systems, definite integrals of complicated functions, differential equations modelling physics, and optimization problems all require approximation. Numerical methods provide systematic, computer-implementable recipes for such problems.
Types of Errors
Two fundamental error types occur. Inherent errors arise from inaccuracies in input data: measurements, rounded constants, model simplifications. Procedural errors are introduced by the method itself. Procedural errors subdivide into truncation errors, from replacing an infinite process (like an infinite series or the limit of a derivative) by a finite approximation, and round-off errors, from representing real numbers with finite precision.
Absolute and Relative Errors
If the true value is x and the approximation is x*, the absolute error is |x − x*|, and the relative error is |x − x*| / |x|. Relative error is usually more meaningful because it captures the significance of an error with respect to the magnitude of the quantity. Many numerical algorithms produce guaranteed bounds on relative error.
Significant Digits and Accuracy
A digit is significant if it provides useful information about the value. Accuracy refers to closeness to the true value; precision refers to closeness of repeated measurements. Careful distinction between the two is essential when reporting numerical results. Conventional rules estimate the number of accurate digits based on the relative error.
Floating-Point Representation
Computers represent real numbers using floating-point formats such as IEEE 754. A number is stored as sign, exponent, and mantissa. The finite number of bits causes round-off. Special values include positive and negative zero, infinities, and NaN for invalid operations. Understanding the representation helps explain why certain calculations behave unexpectedly.
Error Propagation
In a sequence of calculations, errors accumulate. Addition and subtraction propagate absolute errors; multiplication and division propagate relative errors approximately. Catastrophic cancellation occurs when subtracting two nearly equal numbers, destroying significant digits. Rewriting formulas to avoid cancellation is an important numerical technique.
Stability and Conditioning
An algorithm is numerically stable if small input perturbations lead to small output perturbations. A problem is well-conditioned if its solution depends continuously on inputs; otherwise it is ill-conditioned. The condition number of a problem measures sensitivity. Using a stable algorithm on a well-conditioned problem is the goal; mismatches lead to unreliable results.
Convergence
Iterative methods generate a sequence of approximations that converge to the true answer. Order of convergence measures how quickly the error decreases; linear, quadratic, and cubic convergence are common. A method with quadratic convergence roughly doubles the correct digits at each iteration.
Summary
Numerical methods start from the reality that computers work with finite data. Identifying the sources of error, quantifying their magnitudes, and understanding their propagation form the foundation for trusting the numerical answers we compute. Subsequent chapters apply these principles to specific problem classes: roots of equations, linear systems, interpolation, integration, and differential equations.