Convex Optimization Taster

What does it mean for a function to be convex?

If you like optimization, chances are you’ve heard of convex optimization, the study of optimization of convex functions. In this post, we’ll look at what it means to be convex, and why we should care.

import matplotlib.pyplot as plt
import numpy as np

fig, ax = plt.subplots(figsize=(6, 6))
ax.plot(np.linspace(-5, 5, 100), np.linspace(-5, 5, 100)**2)
ax.set_title("$x^2$ - a convex function")
pass

A convex function is a function (of potentially multiple variables) with the following property: any line drawn between two points on the function will never pass below the function. Note that this forces the function to have a single local mimimum¹ - think about it!². If you consider the shape formed by all the points above the function, you will have a convex shape - hence the name.

Footnotes

1. Well, a flat line is convex but it has infinitely many minima - so we should in theory be more precise about our wording. Note, however that all minima are just as good as any other.

2. Another aside; the convex function \(e^x\) has no minima, unless we count the minimum at \(-\infty\). The real benefit is that we no longer have to worry about getting stuck in suboptimal local mimima.