Linear Algebra – Eigenvalues and eigenvectors

Definition: An eigenvector of an \(n\times n\) matrix \(A\) is a vector \(\mathbf{x}\neq\mathbf{0}\) such that \(A\mathbf{x}=\lambda\mathbf{x}\) for a certain \(\lambda\in\mathbb{R}\). A scalar \(\lambda\in\mathbb{R}\) is called an eigenvalue of \(A\) if there exists a solution \(\mathbf{x}\neq\mathbf{0}\) of \(A\mathbf{x}=\lambda\mathbf{x}\). Such a vector \(\mathbf{x}\) is called an eigenvector of \(A\) corresponding to the eigenvalue \(\lambda\).

The characteristic equation

A scalar \(\lambda\in\mathbb{R}\) is an eigenvalue of an \(n\times n\) matrix \(A\) if and only if \(\lambda\) satisfies the characteristic equation

\[\det(A-\lambda I)=0.\]

The \(n^{\text{th}}\) degree polynomial \(\det(A-\lambda I)\) in \(\lambda\) is called the characteristic polynomial of \(A\).

Proof: If \(A\mathbf{x}=\lambda\mathbf{x}\) with \(\mathbf{x}\neq\mathbf{0}\), then we have: \((A-\lambda I)\mathbf{x}=\mathbf{0}\) is a homogeneous vector equation with a nontrivial solution \(\mathbf{x}\neq\mathbf{0}\). This implies that the matrix \(A-\lambda I\) is not invertible and therefore: \(\det(A-\lambda I)=0\).

The fundamental theorem of algebra implies that

\[\det(A-\lambda I)=(\lambda_1-\lambda)(\lambda_2-\lambda)\cdots(\lambda_n-\lambda),\quad\lambda_1\lambda_2,\ldots,\lambda_n\in\mathbb{C}.\]

If \(A\) is a real matrix, then nonreal eigenvalues only occur in complex conjugate pairs.

The (algebraic) multiplicity of an eigenvalue \(\lambda\) is its multiplicity as a root of the characteristic equation.

If \(\lambda\in\mathbb{R}\) is an eigenvalue of an \(n\times n\) matrix \(A\), then \(\text{E}_{\lambda}:=\{\mathbf{x}\in\mathbb{R}^n\,|\,A\mathbf{x}=\lambda\mathbf{x}\}\) is called the eigenspace of \(A\) corresponding to the eigenvalue \(\lambda\). Note that \(\text{E}_{\lambda}=\text{Nul}(A-\lambda I)\).

The geometric multiplicity of an eigenvalue \(\lambda\) is the dimension of the eigenspace \(\text{E}_{\lambda}\).

Theorem: The eigenvalues of an (upper or lower) triangular matrix are the entries on its main diagonal.

Theorem: If \(\mathbf{v}_1,\ldots,\mathbf{v}_r\) are eigenvectors of a matrix \(A\) corresponding to the distinct eigenvalues \(\lambda_1,\ldots,\lambda_r\) respectively, then the set \(\{\mathbf{v}_1,\ldots,\mathbf{v}_r\}\) is linear independent.

Proof: Suppose that \(\{\mathbf{v}_1,\ldots,\mathbf{v}_r\}\) is linear dependent, then one of the vectors can be written as a linear combination of the other vectors. Suppose that \(\mathbf{v}_{p+1}=c_1\mathbf{v}_1+\cdots+c_p\mathbf{v}_p\) and that \(\{\mathbf{v}_1,\ldots,\mathbf{v}_p\}\) is linear independent. Then we have:

\[A\mathbf{v}_{p+1}=c_1A\mathbf{v}_1+\cdots+c_pA\mathbf{v}_p\quad\Longrightarrow\quad\lambda_{p+1}\mathbf{v}_{p+1}=c_1\lambda_1\mathbf{v}_1+\cdots+c_p\lambda_p\mathbf{v}_p.\]

However, we also have that \(\lambda_{p+1}\mathbf{v}_{p+1}=c_1\lambda_{p+1}\mathbf{v}_1+\cdots+c_p\lambda_{p+1}\mathbf{v}_p\). Subtracting the last two equations, we obtain:

\[\mathbf{0}=c_1(\lambda_1-\lambda_{p+1})\mathbf{v}_1+\cdots+c_p(\lambda_p-\lambda_{p+1})\mathbf{v}_p.\]

Since \(\{\mathbf{v}_1,\ldots,\mathbf{v}_p\}\) is linear independent, this implies that all weights are equal to zero. Since all eigenvalues are different, we have that \(\lambda_i-\lambda_{p+1}\neq0\) for all \(i=1,2,\ldots,p\). Hence: \(c_i=0\) for all \(i=1,2,\ldots,p\). However, this implies that \(\mathbf{v}_{p+1}=c_1\mathbf{v}_1+\cdots+c_p\mathbf{v}_p=\mathbf{0}\). This is impossible because \(\mathbf{v}_{p+1}\) is an eigenvector. This implies that \(\{\mathbf{v}_1,\ldots,\mathbf{v}_r\}\) is linear independent.

Theorem: If \(A\) is a square matrix, then \(A\) and \(A^T\) have the same characteristic polynomials and therefore the same eigenvalues.

Proof: \(\det(A^T-\lambda I)=\det(A^T-\lambda I^T)=\det((A-\lambda I)^T)=\det(A-\lambda)\).

Definition: Two \(n\times n\) matrices \(A\) and \(B\) are called similar if there exists an invertible matrix \(P\) such that \(A=PBP^{-1}\).

Note that if \(A=PBP^{-1}\), then we have: \(B=QAQ^{-1}\) with \(Q=P^{-1}\).

Theorem: If the \(n\times n\) matrices \(A\) and \(B\) are similar, then they have the same characteristic polynomial and therefore the same eigenvalues.

Proof: If \(A=PBP^{-1}\), then we have:

\[A-\lambda I=PBP^{-1}-\lambda PP^{-1}=P(BP^{-1}-\lambda P^{-1})=P(B-\lambda I)P^{-1}.\]

This implies that

\[\det(A-\lambda I)=\det(P(B-\lambda I)P^{-1})=\det(P)\cdot\det(B-\lambda I)\cdot\det(P^{-1})=\det(B-\lambda I),\]

since \(\det(P)\cdot\det(P^{-1})=\det(PP^{-1})=\det(I)=1\).

Theorem: If \(\lambda_1,\ldots,\lambda_n\in\mathbb{C}\) are all eigenvalues of an \(n\times n\) matrix \(A\), counted with multiplicity, then we have:

\[\lambda_1\lambda_2\cdots\lambda_n=\det(A)\quad\text{en}\quad\lambda_1+\lambda_2+\cdots+\lambda_n=\text{tr}(A),\]

where \(\text{tr}(A)=a_{11}+a_{22}+\cdots+a_{nn}\) is the trace of the matrix \(A\).

---

Last modified on April 5, 2021