Unit 8, lesson 41

Cyclic Subspaces and Companion Matrices

How the orbit of one vector spans a cyclic subspace, why its first dependence gives an orbit basis, a companion matrix and a linear recurrence, and when an operator has a cyclic vector.

  • Video 41
  • Polynomial Tools and Cyclic Structure

Start with e1=(1,0)e_1 = (1, 0) and keep applying the matrix with rows (0,−2)(0, -2) and (1,3)(1, 3), and the vectors (0,1)(0, 1), then (−2,3)(-2, 3), then (−6,7)(-6, 7) appear. The starting vector and its first image are independent, the second image (−2,3)(-2, 3) is already a combination of them, and from then on no new direction can ever appear. This lesson shows that this first dependence settles a lot at once, namely an invariant subspace with a natural basis, the matrix of the operator restricted to that subspace, and a recurrence that every later vector obeys. It also explains why some starting vectors reach the whole space and others do not. Throughout, F\mathbb{F} is R\mathbb{R} or C\mathbb{C}, VV is a finite-dimensional vector space over F\mathbb{F}, a square matrix AA stands for the operator x↦Axx \mapsto Ax, and a tuple such as (1,0)(1, 0) stands for a column vector.

Key terms

Orbit of a vector

Definition Let T ⁣:V→VT \colon V \to V be linear and let v∈Vv \in V. The orbit of vv under TT is the sequence

v, Tv, T2v, T3v, …v, \ Tv, \ T^2 v, \ T^3 v, \ \dots

with T0=IT^0 = I. Suppose v≠0v \ne 0. Any dim⁡V+1\dim V + 1 vectors of VV are dependent, so there is a smallest positive integer dd for which v,Tv,…,Tdvv, Tv, \dots, T^d v are linearly dependent, and d≤dim⁡Vd \le \dim V. By the choice of dd the vectors v,Tv,…,Td−1vv, Tv, \dots, T^{d-1} v are independent. In a nontrivial dependence among v,Tv,…,Tdvv, Tv, \dots, T^d v the coefficient of TdvT^d v is not zero, since otherwise the earlier vectors would be dependent. Dividing by it gives the first dependence

Tdv=−a0v−a1Tv−⋯−ad−1Td−1v,\begin{aligned} T^d v &= -a_0 v - a_1 Tv - \cdots \\[0.4em] &\quad - a_{d-1} T^{d-1} v, \end{aligned}

and the scalars a0,…,ad−1∈Fa_0, \dots, a_{d-1} \in \mathbb{F} are unique because the earlier vectors are independent. Equivalently, p(T)v=0p(T)v = 0 for the monic polynomial

p(t)=td+ad−1td−1+⋯+a1t+a0.\begin{aligned} p(t) &= t^d + a_{d-1} t^{d-1} + \cdots \\[0.4em] &\quad + a_1 t + a_0. \end{aligned}

In plain words Apply TT to one vector again and again and keep the outputs in order. Each new output either lies outside the span of the earlier vectors or is a combination of them. The first time it is a combination, the coefficients of that combination define the polynomial pp.

Example Let

A=[0−213]A = \begin{bmatrix} 0 & -2 \\ 1 & 3 \end{bmatrix}

act on R2\mathbb{R}^2 and take v=e1v = e_1. The orbit starts (1,0)(1, 0), (0,1)(0, 1), (−2,3)(-2, 3), (−6,7)(-6, 7). The first two vectors are independent and (−2,3)=−2(1,0)+3(0,1)(-2, 3) = -2(1, 0) + 3(0, 1), so d=2d = 2 and the first dependence is A2v=−2v+3AvA^2 v = -2v + 3Av. Watch the signs here. The coefficients are a0=2a_0 = 2 and a1=−3a_1 = -3, because moving everything to one side gives A2v−3Av+2v=0A^2 v - 3Av + 2v = 0, so p(t)=t2−3t+2p(t) = t^2 - 3t + 2.

Cyclic subspace

Definition Let T ⁣:V→VT \colon V \to V be linear and v∈Vv \in V. The cyclic subspace generated by vv is the span of its orbit,

CT(v)=span⁡(v,Tv,T2v,… ).\mathcal{C}_T(v) = \operatorname{span}(v, Tv, T^2 v, \dots).

For v=0v = 0 it is {0}\{0\}. For v≠0v \ne 0, let dd and pp come from the first dependence. Then the orbit basis v,Tv,…,Td−1vv, Tv, \dots, T^{d-1} v is a basis of CT(v)\mathcal{C}_T(v), so dim⁡CT(v)=d\dim \mathcal{C}_T(v) = d. The subspace is invariant under TT, and it is contained in every TT-invariant subspace that contains vv. The minimal polynomial of the restriction of TT to U=CT(v)U = \mathcal{C}_T(v), written T∣UT|_{U}, is pp. A polynomial qq satisfies q(T)v=0q(T)v = 0 exactly when pp divides qq, so in particular pp divides the minimal polynomial μT\mu_T.

For the proof, let UU be the span of the orbit basis. The operator sends TjvT^j v to Tj+1vT^{j+1} v, which lies in UU for j<d−1j < d - 1, and it sends Td−1vT^{d-1} v to TdvT^d v, which lies in UU by the first dependence. By linearity TT maps UU into UU, and induction on kk puts every TkvT^k v in UU. Hence CT(v)=U\mathcal{C}_T(v) = U, and any invariant subspace containing vv contains every TkvT^k v and therefore UU. Polynomials in TT commute, so p(T)Tjv=Tjp(T)v=0p(T)T^j v = T^j p(T) v = 0 for every jj, and p(T)p(T) vanishes on UU. A nonzero polynomial of degree less than dd that vanished on UU would vanish at vv, which would be a dependence among the independent vectors v,…,Td−1vv, \dots, T^{d-1} v. So pp is the monic annihilating polynomial of least degree for T∣UT|_{U}. Finally, if q(T)v=0q(T)v = 0, the same commuting argument shows that q(T)q(T) vanishes on UU, so pp divides qq. Conversely q=hpq = hp gives q(T)v=h(T)p(T)v=0q(T)v = h(T)p(T)v = 0.

In plain words The cyclic subspace is everything you can build from vv by applying TT repeatedly and taking linear combinations. It is the smallest invariant subspace containing vv, and the vectors before the first dependence already span it, so drawing more orbit points never adds anything. The word cyclic refers to the repeated action, not to an orbit that closes up into a loop.

Example For the matrix AA above and v=e1v = e_1 the orbit basis is e1,e2e_1, e_2, so CA(e1)=R2\mathcal{C}_A(e_1) = \mathbb{R}^2 and p(A)=0p(A) = 0 on the whole plane. Indeed

A2=[−2−637],A^2 = \begin{bmatrix} -2 & -6 \\ 3 & 7 \end{bmatrix},

and the entries of A2−3A+2IA^2 - 3A + 2I are −2−0+2-2 - 0 + 2, −6+6+0-6 + 6 + 0, 3−3+03 - 3 + 0 and 7−9+27 - 9 + 2, all zero. A different seed can see less. The vector w=(1,−1)w = (1, -1) has Aw=(2,−2)=2wAw = (2, -2) = 2w, so CA(w)\mathcal{C}_A(w) is the line spanned by ww, with d=1d = 1 and p(t)=t−2p(t) = t - 2. That polynomial divides μA(t)=(t−1)(t−2)\mu_A(t) = (t - 1)(t - 2), as it must.

Cyclic vector

Definition Let n=dim⁡V≥1n = \dim V \ge 1 and let T ⁣:V→VT \colon V \to V be linear. A vector v∈Vv \in V is a cyclic vector for TT if CT(v)=V\mathcal{C}_T(v) = V, that is, if v,Tv,…,Tn−1vv, Tv, \dots, T^{n-1} v is a basis of VV. If TT has a cyclic vector vv with first dependence pp, then

χT=μT=p.\chi_T = \mu_T = p.

Indeed, pp is the minimal polynomial of the restriction of TT to CT(v)=V\mathcal{C}_T(v) = V, which is TT itself, so μT=p\mu_T = p has degree nn. By the Cayley-Hamilton theorem μT\mu_T divides χT\chi_T, which is also monic of degree nn, so the two are equal. Consequently an operator whose minimal polynomial has degree less than nn has no cyclic vector. The converse is also true over R\mathbb{R} and C\mathbb{C}, so TT has a cyclic vector exactly when deg⁡μT=n\deg \mu_T = n, but that direction needs a longer argument and is not proved here.

In plain words A cyclic vector reaches the whole space by repeated action. When one exists, a single polynomial of degree nn records how TT acts, and the characteristic and minimal polynomials cannot differ.

Example For the matrix AA above, e1e_1 is a cyclic vector for AA on R2\mathbb{R}^2 and (1,−1)(1, -1) is not. Now let D=diag⁡(1,2,2)D = \operatorname{diag}(1, 2, 2) act on R3\mathbb{R}^3. Its minimal polynomial (t−1)(t−2)(t - 1)(t - 2) has degree 2<32 < 3, so DD has no cyclic vector. The seed v=(1,1,1)v = (1, 1, 1) gives Dv=(1,2,2)Dv = (1, 2, 2) and D2v=(1,4,4)=3Dv−2vD^2 v = (1, 4, 4) = 3Dv - 2v, so it generates a plane, and the seed (0,1,−1)(0, 1, -1) is an eigenvector for 22 and generates only a line. A scalar matrix cIcI on a space of dimension at least 22 is the extreme case, since every nonzero seed generates just its own line.

Companion matrix

Definition Let p(t)=td+ad−1td−1+⋯+a1t+a0p(t) = t^d + a_{d-1} t^{d-1} + \cdots + a_1 t + a_0 be a monic polynomial of degree d≥1d \ge 1 with coefficients in F\mathbb{F}. Its companion matrix is the d×dd \times d matrix with ones directly below the diagonal, the negated coefficients in the last column, and zeros elsewhere,

Cp=[00⋯0−a010⋯0−a101⋯0−a2⋮⋮⋱⋮⋮00⋯1−ad−1].C_p = \begin{bmatrix} 0 & 0 & \cdots & 0 & -a_0 \\ 1 & 0 & \cdots & 0 & -a_1 \\ 0 & 1 & \cdots & 0 & -a_2 \\ \vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & 0 & \cdots & 1 & -a_{d-1} \end{bmatrix}.

If v≠0v \ne 0 has first dependence pp, then the matrix of T∣UT|_{U} on U=CT(v)U = \mathcal{C}_T(v) in the ordered orbit basis (v,Tv,…,Td−1v)(v, Tv, \dots, T^{d-1} v) is CpC_p. Column jj holds the coordinates of the image of the jjth basis vector. The operator sends each orbit basis vector to the next one, which gives the ones below the diagonal, and it sends the last one to TdvT^d v, whose coordinates −a0,…,−ad−1-a_0, \dots, -a_{d-1} come from the first dependence. Since Cpej=ej+1C_p e_j = e_{j+1} for j<dj < d and CpedC_p e_d is the last column, the vector e1e_1 is a cyclic vector for CpC_p on Fd\mathbb{F}^d with first dependence pp, so

χCp=μCp=p.\chi_{C_p} = \mu_{C_p} = p.

Some books use the transpose, with the negated coefficients in the last row. It has the same characteristic polynomial.

In plain words In an orbit basis the operator just moves each basis vector to the next one, until the last, where the first dependence takes over. The last column stores the coefficients a0,…,ad−1a_0, \dots, a_{d-1} of pp with their signs reversed.

Example For p(t)=t2−3t+2p(t) = t^2 - 3t + 2 we have a0=2a_0 = 2 and a1=−3a_1 = -3, so the last column is (−2,3)(-2, 3) and CpC_p is the matrix AA from the start. The same polynomial appears for D=diag⁡(1,2,2)D = \operatorname{diag}(1, 2, 2) and the seed v=(1,1,1)v = (1, 1, 1), so the restriction of DD to the plane spanned by vv and DvDv also has matrix CpC_p in the basis (v,Dv)(v, Dv). The orbit matrix with columns vv and DvDv is 3×23 \times 2 and has no inverse, so the restricted matrix comes from coordinates in that basis instead. The image DvDv has coordinates (0,1)(0, 1), and the image D(Dv)=−2v+3DvD(Dv) = -2v + 3Dv has coordinates (−2,3)(-2, 3). For degree three take

p(t)=(t−1)(t−2)(t−3)=t3−6t2+11t−6,\begin{aligned} p(t) &= (t - 1)(t - 2)(t - 3) \\[0.4em] &= t^3 - 6t^2 + 11t - 6, \end{aligned}

whose companion matrix is

Cp=[00610−11016].C_p = \begin{bmatrix} 0 & 0 & 6 \\ 1 & 0 & -11 \\ 0 & 1 & 6 \end{bmatrix}.

Linear recurrence

Definition Let p(t)=td+ad−1td−1+⋯+a1t+a0p(t) = t^d + a_{d-1} t^{d-1} + \cdots + a_1 t + a_0 be monic with coefficients in F\mathbb{F}. A sequence x0,x1,x2,…x_0, x_1, x_2, \dots of scalars, or of vectors in VV, satisfies the linear recurrence with polynomial pp if

xk+d=−ad−1xk+d−1−⋯−a1xk+1−a0xk\begin{aligned} x_{k+d} &= -a_{d-1} x_{k+d-1} - \cdots \\[0.4em] &\quad - a_1 x_{k+1} - a_0 x_k \end{aligned}

for every k≥0k \ge 0. The orbit xk=Tkvx_k = T^k v of a nonzero vector satisfies the recurrence of its first dependence pp, because applying TkT^k to p(T)v=0p(T)v = 0 gives Tkp(T)v=0T^k p(T) v = 0, which is the recurrence at step kk. For every linear functional ℓ∈V∨\ell \in V^{\vee}, the scalar sequence sk=ℓ(Tkv)s_k = \ell(T^k v) satisfies the same recurrence, because ℓ\ell is linear. The orbit also satisfies the recurrence of any monic polynomial qq with q(T)v=0q(T)v = 0, such as μT\mu_T or χT\chi_T, and among these the recurrence of pp has the least order.

In plain words Once one dependence among consecutive images is known, every later image follows from the previous dd images with the same coefficients at every step. Any fixed linear measurement of the images inherits the same rule.

Example For AA and v=e1v = e_1 the recurrence is xk+2=3xk+1−2xkx_{k+2} = 3x_{k+1} - 2x_k. From (0,1)(0, 1) and (−2,3)(-2, 3) it gives

3(−2,3)−2(0,1)=(−6,7),3(-2, 3) - 2(0, 1) = (-6, 7),

which agrees with A(−2,3)A(-2, 3), and the next vector is (−14,15)(-14, 15). The second coordinates 0,1,3,7,150, 1, 3, 7, 15 obey sk+2=3sk+1−2sks_{k+2} = 3s_{k+1} - 2s_k, and induction shows sk=2k−1s_k = 2^k - 1, because 3(2k+1−1)−2(2k−1)=2k+2−13(2^{k+1} - 1) - 2(2^k - 1) = 2^{k+2} - 1. The recurrence belongs to the seed. The orbit of w=(1,−1)w = (1, -1) satisfies the shorter rule xk+1=2xkx_{k+1} = 2x_k.

Check yourself

  1. Let BB be the matrix with rows (2,1)(2, 1) and (0,2)(0, 2). Find the cyclic subspaces generated by e1e_1 and by e2e_2, and the matrix of BB in the orbit basis of e2e_2.
Show the answer

The seed e1e_1 is an eigenvector, since Be1=(2,0)=2e1Be_1 = (2, 0) = 2e_1. So d=1d = 1, the cyclic subspace is the line spanned by e1e_1, and p(t)=t−2p(t) = t - 2. For e2e_2 the orbit starts (0,1)(0, 1), (1,2)(1, 2), (4,4)(4, 4). The first two are independent, and

(4,4)=4(1,2)−4(0,1),(4, 4) = 4(1, 2) - 4(0, 1),

so B2e2=4Be2−4e2B^2 e_2 = 4Be_2 - 4e_2, the vector e2e_2 is cyclic with CB(e2)=R2\mathcal{C}_B(e_2) = \mathbb{R}^2, and p(t)=t2−4t+4=(t−2)2p(t) = t^2 - 4t + 4 = (t - 2)^2. With a0=4a_0 = 4 and a1=−4a_1 = -4 the matrix in the orbit basis (e2,Be2)(e_2, Be_2) is

Cp=[0−414].C_p = \begin{bmatrix} 0 & -4 \\ 1 & 4 \end{bmatrix}.

As a check, its trace 44 and determinant 44 match those of BB.

  1. Write the companion matrix of p(t)=t3−4t2+t+6p(t) = t^3 - 4t^2 + t + 6 and check its trace and determinant against pp. What goes wrong if the signs in the last column are not reversed?
Show the answer

Here a0=6a_0 = 6, a1=1a_1 = 1 and a2=−4a_2 = -4, so the last column is (−6,−1,4)(-6, -1, 4) and

Cp=[00−610−1014].C_p = \begin{bmatrix} 0 & 0 & -6 \\ 1 & 0 & -1 \\ 0 & 1 & 4 \end{bmatrix}.

For a 3×33 \times 3 matrix the coefficient of t2t^2 in the characteristic polynomial is minus the trace, and the constant term is χCp(0)=det⁡(−Cp)=−det⁡Cp\chi_{C_p}(0) = \det(-C_p) = -\det C_p. The trace is 44 and expanding along the first row gives det⁡Cp=−6\det C_p = -6, which match the coefficient −4-4 and the constant term 66. Copying the coefficients without reversing their signs gives the last column (6,1,−4)(6, 1, -4), a matrix with trace −4-4 whose characteristic polynomial is t3+4t2−t−6t^3 + 4t^2 - t - 6, a different polynomial.

  1. Does D=diag⁡(1,2,3)D = \operatorname{diag}(1, 2, 3) have a cyclic vector? Does E=diag⁡(2,2,3)E = \operatorname{diag}(2, 2, 3)? For the first, find the recurrence satisfied by the orbit of v=(1,1,1)v = (1, 1, 1).
Show the answer

For DD the orbit of vv starts (1,1,1)(1, 1, 1), (1,2,3)(1, 2, 3), (1,4,9)(1, 4, 9). The matrix with these columns has determinant (2−1)(3−1)(3−2)=2≠0(2 - 1)(3 - 1)(3 - 2) = 2 \ne 0, so they form a basis and vv is a cyclic vector. Then χD=μD=(t−1)(t−2)(t−3)=t3−6t2+11t−6\chi_D = \mu_D = (t - 1)(t - 2)(t - 3) = t^3 - 6t^2 + 11t - 6, and the orbit satisfies

xk+3=6xk+2−11xk+1+6xk.x_{k+3} = 6x_{k+2} - 11x_{k+1} + 6x_k.

For instance 6(1,4,9)−11(1,2,3)+6(1,1,1)=(1,8,27)=D3v6(1, 4, 9) - 11(1, 2, 3) + 6(1, 1, 1) = (1, 8, 27) = D^3 v. Not every seed works, though. The seed e1e_1 is an eigenvector and generates only a line. For EE the minimal polynomial is (t−2)(t−3)(t - 2)(t - 3), of degree 2<32 < 3, so EE has no cyclic vector, even though its characteristic polynomial (t−2)2(t−3)(t - 2)^2(t - 3) has degree 33.

How this lesson connects