The wall (constraint)
A guard stands at w = c. Crossing is impossible. The optimizer presses against the wall and stops at w = c, wanting more.
Lesson 0014 · Refresher
The Lagrangian: Prices Replace Walls
Every constraint is a wall. The Lagrangian tears the wall down and charges rent for standing where it stood — and at exactly one rent, the optimizer rebuilds the wall out of pure self-interest. That rent is the dual variable you have been reading since Lesson 0004.
The One-Step Skill
Given any constrained problem, write its Lagrangian and say in plain words what the multiplier is doing: it is the per-unit price of violating the constraint. Then predict the price at which the optimizer obeys the constraint voluntarily. Lesson 0013's grouping trick — and all of duality — is this one move, repeated.
The Picture First: Tear Down the Wall, Charge Rent
Take the wall geometry from Lesson 0004: the bowl f0(w) wants w = 1.00, but a mandate says w <= c. Two ways to enforce the mandate:
The toll (Lagrangian)
Fire the guard. Anyone may cross — but standing at w costs λ per unit past the line. The rule became a price.
Written out, the toll version is just the objective plus the priced violation:
wall: minimize f0(w) subject to w - c <= 0
toll: minimize L(w, λ) = f0(w) + λ (w - c)
— no constraint left; λ >= 0 is the price per unit of violation
That is the entire mechanism of “moving a constraint into the objective”: replace the rule with its price. The term λ(w - c) is positive when you violate (you pay) and negative when you have slack (curiously, you get credit — hold that thought for the certificate below).
Now the key question: can a price really substitute for a wall? Watch what the toll does to the picture. Adding λ(w - c) tilts the whole bowl up on the right — the more you charge, the more the bowl's minimum slides left, away from the forbidden zone.
Memory hook: the multiplier is the rent at which you would obey the rule voluntarily. Too cheap — you cheat. Too dear — you over-comply. At
λ* — you stand exactly where the wall stood, by choice.The Theory: Why One Price Rebuilds the Wall
The toll-payer (call him the renter) minimizes the tilted bowl without any constraint. Smooth optimality from Lesson 0005:
renter's choice: L'(w) = f0'(w) + λ = 0
with our bowl: 0.30 w - 0.30 + λ = 0
w(λ) = 1 - λ/0.30 ← each price buys a position
Every price λ produces a position w(λ), sliding from the frictionless target (free) back toward zero (punitive). Demand that the renter lands exactly on the old wall w(λ) = c and solve for the price:
λ* = -f0'(c) = 0.30 (1 - c) ← the push at the wall cap c = 60%: λ* = 0.12 loose cap (c >= 100%): λ* = 0 (nobody pays for a wall behind them)
Recognize both facts. The price that rebuilds the wall equals the objective's push against the wall — the same shadow price Lessons 0004 and 0008 taught you to read off a solver. And a slack constraint commands price zero — complementary slackness, now obvious: rent for space nobody wants is zero.
This is also Lesson 0012 wearing a suit: there, a fee κ was a price on trading, and trades happened only when the push beat the price. Here, λ is a price on violating, and violations happen only when the push beats the price. Fees, tolls, multipliers — one idea: prices channel pushes.
Why does anyone bother, beyond elegance? Because the priced problem is a bound certificate. For any feasible w (so w - c <= 0) and any λ >= 0, the toll term is <= 0, hence:
L(w, λ) = f0(w) + λ(w - c) <= f0(w) for every feasible w so: min over ALL w of L(w, λ) <= best feasible f0(w) every price you name yields a floor under the true optimum; the best price λ* raises the floor until it touches (convexity).
Name any rent, let the renter optimize freely, and his score is a guaranteed floor under the honest constrained optimum — that is weak duality. Solvers hunt for the rent with the highest floor, and for convex problems the floor meets the ceiling: the wall problem and the best-priced toll problem have the same answer. That is the license behind Lesson 0013's move of treating the budget's dual as a fee.
Lab: Find the Rent That Rebuilds the Wall
The dashed bowl is f0; the solid bowl is the Lagrangian, tilted by the toll. The shaded zone is where the wall used to be — nothing stops the renter now except price. Three experiments: (1) set λ = 0 — the renter squats at the frictionless target, deep in forbidden territory; (2) raise λ and watch him retreat; with the cap at 60% he lands exactly on the toll line at λ = 0.120 = 0.30(1 - 0.60), matching the metric; (3) keep raising — he over-complies, leaving objective value on the table, which is why the best price stops exactly at λ*. Then slide the cap to 100% and confirm no positive rent makes sense.
Cap c-
Rent λ-
Renter's choice-
Violation-
Verdict-
λ* (wall rent)-
The grey dot is where the wall would have forced you; the black dot is where the price sends you. The lesson is the moment they coincide.
Reread Lesson 0013 With This Picture
Lesson 0013's Lagrangian was L(w, λ) = f0(w) + λ(|w - w_prev| - τ). You can now read it as: fire the turnover guard, charge λ per unit of turnover beyond the budget. Grouping the terms showed the priced problem is Lesson 0012's fee problem with κ = λ — of course it is: a price on excess turnover and a fee on turnover differ only by the constant -λτ, which moves no minimizer. The penalty–budget equivalence isn't a trick; it's the Lagrangian doing the only thing it ever does.
One sentence per object
Lagrangian: the objective after every wall becomes a toll. Multiplier: the toll. Dual problem: shopping for the toll with the highest floor. λ*: the toll that rebuilds the wall.
Sign discipline
Write inequalities as g(w) <= 0; then L = f0 + λ g with λ >= 0. Violation costs, slack credits. Wrong signs are the #1 way to garble a Lagrangian at a whiteboard.
Many walls, many rents
With several constraints, each gets its own price: L = f0 + Σ λ_i g_i. KKT stationarity (Lesson 0004) just says: at the optimum, the objective's push is exactly paid for by the active rents.
Interview shape
“Why can you move a constraint into the objective?” Answer: at the optimal multiplier, the priced problem has the same solution as the walled one (convexity for exactness); the multiplier is the constraint's shadow price.
Retrieval Practice
Cap at c = 60%, so λ* = 0.12. You price violations at λ = 0.06. The renter does what?
What is the price λ* that makes the renter voluntarily stop exactly at the wall?
You pick some rent λ >= 0 and let the renter minimize freely. His score is guaranteed to be?
Primary Source
Chapter 5.1–5.2 of Boyd & Vandenberghe, Convex Optimization builds the Lagrangian and weak duality exactly along these lines (their “least-squares interpretation of the dual” is the floor-hunting story). Keep the new Lagrangian recipe card at hand, next to the KKT reference and the budget–penalty bridge.
Questions?
Ask your agent anything unclear — good prompts: “walk the toll story through an equality constraint,” “why can λ be any sign for equalities but not inequalities?”, or “show the floor rising as λ sweeps from 0 to 0.2 with the cap at 60%.”