Least Squares — Observation Equations - Cheat Sheet
Your 30-minute exam companion for observation-equation (parametric) least squares. Covers the core principle, matrix solution, residuals, redundancy, and weighted-mean applications essential for the PRC Licensure Examination.
Core Principle & Concept
Formulas
Minimize Σ wᵢvᵢ²
Meaning
w = weight; v = residual (computed − observed); sum over all observations
Watch out
Sign of residual: here v = computed − observed (some texts use observed − computed; be consistent!)
When to use
Whenever you need to adjust redundant observations; it's the definition of least squares
Important facts
- Least squares is the rigorous, most-probable method when errors are random and normally distributed.
- Observation form (parametric): each observation is a function of unknown parameters; alternative is condition form (constraint equations).
- For one unknown, least-squares solution IS the weighted mean; don't over-complicate.
- Weights are typically w ∝ 1/σ² (inverse variance); equivalent to weight matrix P = diagonal with pᵢᵢ = 1/σᵢ².
- Redundancy must be positive (n > u) to perform adjustment; n = u gives unique solution but no error check.
Key definitions
Residual (v)
Example
If computed elevation = 152.40 m, observed = 152.36 m, then v = +0.04 m
The correction to computed value to match observation; v = Axˆ − ℓ in observation form.
Observation Equation
Example
Three level routes give three equations; each relates observed height to single unknown elevation H
Linear model ℓ + v = f(xˆ), rearranged to v = Axˆ − ℓ, relating each observed value to unknown parameters.
Redundancy (Degrees of Freedom)
Example
12 observations, 8 unknowns → redundancy = 4; allows adjustment and error detection
Number of excess observations: redundancy = n − u (observations − unknowns).
Design Matrix (A)
Example
For one unknown observed n times, A is a column vector of 1's; for two unknowns, A has two columns
Matrix of partial derivatives ∂f/∂xⱼ; each row is one observation, each column one unknown parameter.
Diagrams to know
- Flow: Observation → Linearization → Design Matrix (A) → Normal Equations → Solution xˆ → Residuals v
- Matrix relationship: v = Axˆ − ℓ (parametric form)
- Weight-matrix structure: P = diag(1/σ₁², 1/σ₂², …, 1/σₙ²)
Matrix Formulation & Solution
Formulas
v = Axˆ − ℓ
Meaning
v = residuals (n×1); A = design matrix (n×u); xˆ = parameter corrections (u×1); ℓ = observed − computed (n×1)
Watch out
Sign: some texts define ℓ = computed − observed (opposite sign). Check problem setup. Here ℓ = observed − computed.
When to use
Starting point for all observation-equation least-squares problems; linearized form of observations
(AᵀPA)xˆ = AᵀPℓ
Meaning
Normal equations (u×u system); P = weight matrix (n×n, diagonal); solution gives parameter corrections
Watch out
Matrix (AᵀPA) must be non-singular (full rank); verify by checking if all unknowns appear in at least one observation
When to use
Solve for xˆ directly (board exam) when you don't have matrix inverse formulas; rearrange to isolate xˆ
xˆ = (AᵀPA)⁻¹AᵀPℓ
Meaning
Explicit least-squares solution for unknown corrections; (AᵀPA)⁻¹ is variance-covariance-like matrix
Watch out
Matrix inversion is computational; exam usually gives either the inverse or the normal equations to solve. Check matrix dimensions!
When to use
Direct formula solution (if inverse given or calculable); preferred in computer/matrix calculators
Important facts
- Weighted least squares uses P (weight matrix); unweighted is special case where all weights = 1 (same precision).
- The matrix (AᵀPA) is symmetric and typically positive-definite if problem is well-posed.
- Solution xˆ contains corrections; final parameters = a priori values + xˆ.
- Residuals v are computed after solution via v = Axˆ − ℓ; they sum (nearly) to zero in least-squares sense.
- For independent observations (no correlations), P is diagonal; otherwise, P is full symmetric matrix.
Key definitions
Normal Equations
Example
For one unknown, reduces to Σ wᵢℓᵢ = (Σ wᵢ)xˆ, i.e., xˆ = Σ wᵢℓᵢ / Σ wᵢ (weighted mean)
The system (AᵀPA)xˆ = AᵀPℓ; derived by minimizing Σ wᵢvᵢ² and setting ∂/∂xˆ = 0.
Weight Matrix (P)
Example
Three routes with σ = 10, 5, 20 mm → weights 1/100, 1/25, 1/400 (inverse variance rule)
Diagonal matrix with diagonal entries pᵢᵢ = wᵢ = 1/σᵢ²; encodes precision of each observation.
Reference Variance (σ₀²)
Example
σ̂₀² ≈ 1 (in normalized units) means weighting is realistic; σ̂₀² >> 1 flags blunders or poor weights
Estimate of observation variance; σ̂₀² = (vᵀPv) / (n − u); checks if residuals fit assumed precisions.
Diagrams to know
- Matrix structure: A (n×u), xˆ (u×1), ℓ (n×1), v (n×1), P (n×n diagonal)
- Multiplication sequence: AᵀPA produces (u×u) system; AᵀPℓ produces (u×1) right-hand side
- Solution path: Observations → A, ℓ, P → Normal Equations → xˆ → v → σ̂₀²
Reference Variance & Redundancy Check
Formulas
σ̂₀² = (vᵀPv) / (n − u)
Meaning
v = residuals; P = weight matrix; n = observations; u = unknowns; denominator = redundancy
Watch out
Must compute v first via v = Axˆ − ℓ. If redundancy ≤ 0, can't compute (no unique adjustment exists).
When to use
After computing residuals, to assess quality of fit; check if weighting was realistic
Redundancy = n − u
Meaning
Excess observations available for error checking; must be ≥ 1 for adjustment
Watch out
If n = u, solution is unique but unadjustable (no redundancy, no error estimate). If n < u, problem has infinite solutions.
When to use
First thing: count observations and unknowns; if redundancy ≤ 0, problem is under-determined
Important facts
- Reference variance σ̂₀² is the squared standard error of unit weight (per weight unit); often called post-fit variance.
- If σ̂₀² is much > 1, investigate outliers, blunders, or systematic errors in observations.
- If σ̂₀² is much < 1, weights may be over-pessimistic (observations better than assumed) or model is over-fitting.
- Redundancy needed ≥ 1; typical geodetic surveys aim for redundancy ≥ 3–5 for robust error detection.
- For network adjustment, σ̂₀² ≈ 1 is ideal; σ̂₀² used to scale cofactor matrix for parameter uncertainties.
Key definitions
Degrees of Freedom
Example
12 observations, 8 unknowns → 4 degrees of freedom allow chi-squared test on σ̂₀²
Synonym for redundancy; the number of independent constraints on residuals after solving for unknowns.
Chi-Squared Test (σ̂₀² Check)
Example
σ̂₀² = 0.9 (good); σ̂₀² = 5.2 (suggests blunder or over-optimistic weights)
Compare σ̂₀² to expected value ~1 (if weights are correct units); large σ̂₀² suggests outliers or poor weighting.
Diagrams to know
- Decision tree: redundancy ≤ 0? → Cannot adjust. redundancy > 0? → Compute σ̂₀² → Check fit quality
Single Unknown = Weighted Mean (Exam Gold Standard)
Formulas
xˆ = Σ(wᵢℓᵢ) / Σwᵢ
Meaning
Weighted mean; wᵢ = weight of observation i; ℓᵢ = observed − computed for observation i
Watch out
Weights must be consistent (all proportional to 1/σ² or all relative). If one observation has weight = 1, scale all weights accordingly.
When to use
ONE unknown observed multiple times (most common exam case); quickest, most intuitive solution
wᵢ = 1/σᵢ² (or proportional)
Meaning
Weight inversely proportional to variance (or squared error); higher precision → higher weight
Watch out
If σᵢ² is given in different units (e.g., one in mm², another in m²), convert to consistent units first!
When to use
Assigning weights from measurement uncertainties; standard rule for independent observations
A = [1, 1, …, 1]ᵀ (column of 1s for single unknown)
Meaning
Design matrix for one unknown observed n times; each observation equation: ℓᵢ + vᵢ = xˆ
Watch out
If unknown appears with different coefficients in observations (e.g., ℓ₁ + v₁ = 2xˆ), A is not a column of 1s; revert to general solution.
When to use
Verifying matrix form of single-unknown case; recognize it reduces to weighted mean
Important facts
- For one unknown, least-squares always reduces to the weighted mean; no matrix inversion needed.
- Weights proportional to 1/distance (in leveling) or 1/variance (general); normalize so Σwᵢ = some convenient number or leave as ratio.
- Most common exam case: 2–4 observations of same quantity (height, distance, angle) with different precisions.
- Residuals: vᵢ = xˆ − ℓᵢ; sum of weighted residuals Σ wᵢvᵢ = 0 (always true for least-squares).
- Reference variance: σ̂₀² = Σ wᵢvᵢ² / (n − 1) for single unknown (redundancy = n − 1)
Key definitions
Weighted Mean
Example
Three elevation routes: 152.34 m (weight 0.5), 152.30 m (weight 1.0), 152.36 m (weight 0.25) → weighted mean = 152.32 m
Average value computed with weights proportional to precision; least-squares solution for single unknown.
Diagrams to know
- Single-unknown flow: Multiple observations with weights → Compute Σ wᵢℓᵢ and Σ wᵢ → Divide to get xˆ → Compute residuals vᵢ = xˆ − ℓᵢ → Check Σ wᵢvᵢ ≈ 0
Reactions or equations
xˆ = (Σ wᵢℓᵢ) / (Σ wᵢ) = Σ (wᵢ / Σ wⱼ) · ℓᵢ
Note
Second form shows xˆ as weighted sum of observations; equivalent to normal equations (AᵀPA)xˆ = AᵀPℓ for A = [1,1,…,1]ᵀ
Conditions
All observations of same unknown; weights wᵢ proportional to precision (1/σᵢ²)
Board-Exam Workflow & Common Pitfalls
Important facts
- Step 1: Define unknowns (parameters to adjust) and count them (u).
- Step 2: Count observations (n); compute redundancy = n − u.
- Step 3: If one unknown → use weighted mean directly.
- Step 4: If multiple unknowns → set up design matrix A, observation vector ℓ, weight matrix P.
- Step 5: Solve normal equations (AᵀPA)xˆ = AᵀPℓ (by hand or given).
- Step 6: Compute residuals v = Axˆ − ℓ.
- Step 7: Verify Σ wᵢvᵢ ≈ 0 (weighted residuals sum to zero).
- Step 8: Compute reference variance σ̂₀² = (vᵀPv) / (n − u); check fit quality.
- Pitfall: confusing ℓ = observed − computed vs. computed − observed; define at start and stick with it.
- Pitfall: forgetting to normalize weights or mixing weight units; ensure consistency.
Key definitions
Observation Form (Parametric)
Example
Level network: each sight difference = function of station heights (unknowns); observation equation is linear
Least-squares method modeling each observation as function of unknown parameters; contrast with condition form (constraints).
Linearization
Example
Distance observation d₀ + Δd = √((ΔE)² + (ΔN)²) is nonlinear; linearize around approximate positions
Approximation of nonlinear observation function via Taylor series (first-order only); required for observation equations.
Diagrams to know
- PRC Exam Problem-Solving Checklist: Understand problem → Count n, u → Check redundancy → Set up A, ℓ, P → Solve → Verify → Report
Worked Examples (Exam-Style)
Important facts
- Example 1 (Weighted mean, 1 unknown): 3 level routes to elevation. ℓ₁ = 152.34 m (k₁ = 2 km), ℓ₂ = 152.30 m (k₂ = 1 km), ℓ₃ = 152.36 m (k₃ = 4 km). Weights ∝ 1/k. Solution: w₁ = 0.5, w₂ = 1.0, w₃ = 0.25; Σw = 1.75; xˆ = (0.5·152.34 + 1.0·152.30 + 0.25·152.36)/1.75 = 152.32 m.
- Example 2 (Redundancy check): Network with 12 observations, 8 unknowns. Redundancy = 12 − 8 = 4 degrees of freedom. Adjustment is possible; error detection enabled.
- Example 3 (Reference variance concept): After adjustment, σ̂₀² = 1.3. Interpretation: residuals slightly larger than assumed precisions; possible minor blunder or over-optimistic weights. Check observations #2 and #5.
- Example 4 (Single unknown, 4 observations): Heights h₁ = 100.12 m (σ = 0.05 m), h₂ = 100.08 m (σ = 0.10 m), h₃ = 100.10 m (σ = 0.05 m), h₄ = 100.15 m (σ = 0.20 m). Weights w ∝ 1/σ²: w₁ = 400, w₂ = 100, w₃ = 400, w₄ = 25; Σw = 925. xˆ = (400·100.12 + 100·100.08 + 400·100.10 + 25·100.15)/925 = 100.11 m (approx).
Must remember
- 1. Least squares minimizes Σ wᵢvᵢ² (weighted sum of squared residuals); observation form is v = Axˆ − ℓ.
- 2. Solution to normal equations: xˆ = (AᵀPA)⁻¹AᵀPℓ; for one unknown, this is the weighted mean.
- 3. One unknown observed n times → xˆ = Σ(wᵢℓᵢ)/Σwᵢ; fastest exam solution.
- 4. Weights: wᵢ ∝ 1/σᵢ² (inverse variance rule); higher precision → higher weight.
- 5. Redundancy = n − u (observations − unknowns); must be > 0 for adjustment; enables error checking.
- 6. Residuals: v = Axˆ − ℓ (computed after solving); check Σ wᵢvᵢ ≈ 0 as verification.
- 7. Reference variance: σ̂₀² = (vᵀPv)/(n−u); near 1.0 = good fit; >> 1.0 = investigate blunders.
- 8. Design matrix A has rows = observations, columns = unknowns; partial derivatives ∂f/∂xⱼ.
- 9. Weight matrix P = diagonal with pᵢᵢ = wᵢ = 1/σᵢ² (for independent observations).
- 10. Before solving: always verify redundancy > 0 (enough observations); define ℓ = observed − computed consistently.
Last minute tips
- Tip 1: If problem mentions 'one point observed multiple ways' (multiple leveling routes, GPS fixes, theodolite sightings), it's always a weighted-mean case → use xˆ = Σ(wᵢℓᵢ)/Σwᵢ directly (fastest exam method).
- Tip 2: Sign of ℓ: define once, use consistently. Here ℓ = observed − computed. If confused, check the worked example in your reference and stick with it for the entire problem.
- Tip 3: Weight units must match: if one route is in km and you assign weight 1/distance, scale all weights to same unit or normalize Σwᵢ to a round number (e.g., 1.0 or 10) before dividing.
- Tip 4: Always check redundancy = n − u first. If ≤ 0, stop and state 'problem under-determined' or 'unique solution exists but no adjustment/error estimate'. If > 0, proceed to least-squares.
- Tip 5: After computing xˆ, verify by checking Σ wᵢvᵢ ≈ 0 (should be nearly zero due to normal-equation derivation). Large value = arithmetic error in vᵢ or weights.
Comparison tables
Observation Form vs. Condition Form (Context Only)
| Aspect | Observation Form (Parametric) | Condition Form (Not This Chapter) |
|---|---|---|
| Equation Type | Each observation = f(parameters) | Constraint between observations |
| Design Matrix | A = partials of f; n×u | B = constraint partials; c×n (c = conditions) |
| When to Use | Observations are primary; unknowns are target | Constraints are primary; measurements are secondary |
| Exam Focus | Most common for surveying adjustments | Less common in introductory courses |
Weight Assignment Rules (Key Reference)
| Observation Type | Standard Weight Formula | Example |
|---|---|---|
| Leveling by distance | w ∝ 1/distance (or 1/sqrt if route) | 2 km route: w = 0.5; 1 km route: w = 1.0 |
| Angle measurement | w ∝ 1/σ² (sigma from instrument spec) | σ = 5 arcsec: w = 1/25; σ = 10 arcsec: w = 1/100 |
| Distance by EDM | w ∝ 1/σ² (proportional to range × constant) | σ = 5 mm at 100 m: w = 1/25; σ = 10 mm at 200 m: w = 1/100 |
| Equal precision | w = 1 for all (unweighted case) | All observations assumed same σ; simple mean |
Residual Interpretation (σ̂₀² Values)
| σ̂₀² Range | Interpretation | Action |
|---|---|---|
| 0.5–1.5 | Good fit; weighting is realistic | Accept solution; report standard errors |
| 1.5–3.0 | Acceptable; minor inconsistencies | Check for systematic errors; review observations |
| > 3.0 | Poor fit; probable blunders or bad weights | Investigate outliers; re-measure suspect observations |
| < 0.5 | Unusually good; weights may be pessimistic | Verify weights; solution is likely valid but optimistic |
Diagrams
Least Squares Problem-Solving Flowchart
Complete workflow from problem statement to final check. Shows decision points (redundancy test, single-unknown shortcut) and verification steps. Critical for PRC exam time management.
Least Squares Observation Equations — Complete Concept Map
Hierarchical concept map showing how observation-equation least squares is organized: principle → equation form → solution methods (general and single-unknown shortcut) → quality checks. Essential study map for PRC Licensure Exam.
Single-Unknown Weighted-Mean Decision & Calculation
Detailed step-by-step flowchart for the most common PRC exam case: one unknown observed multiple times. Shows weight assignment, weighted-mean computation, residual check, and optional variance assessment. Use this as your personal exam template.