Vector Space and Geometry¶
Vector Space¶
Let \(\mathcal{F}\) be a scalar field (such as \(\mathbb{R}\) or \(\mathbb{C}\)).
Attention
Field refers to the algebraic definition with properly defined addition and multiplication operators on them.
Not to be confused with scalar fields which represents functionals that maps vectors into scalers.
[Definition] Vector Space
\(V_\mathcal{F}\) is a vector space over \(\mathcal{F}\) with \(0\in \mathcal{F}\) iff:
\(\forall a\in \mathcal{F},\mathbf{u}\in V_\mathcal{F}\implies a\cdot\mathbf{u}\in V_\mathcal{F}\)
\(\mathbf{u},\mathbf{v}\in V_\mathcal{F}\implies \mathbf{u}+\mathbf{v}\in V_\mathcal{F}\)
with the following properties:
[Commutative Addition]: \(\mathbf{u}+\mathbf{v}=\mathbf{v}+\mathbf{u}\)
[Identity Element]: \(\exists\mathbf{0}\in V_\mathcal{F}\) such that
\(0\cdot\mathbf{u}=\mathbf{0}\)
\(\mathbf{u}+\mathbf{0}=\mathbf{0}+\mathbf{u}=\mathbf{u}\)
[Inverse Element]: \(\forall\mathbf{u}\in V_\mathcal{F},\exists\mathbf{v}\in V_\mathcal{F}\) (represented as \(-\mathbf{u}\)) such that
\(\mathbf{u}+\mathbf{v}=\mathbf{0}\)
Attention
Vector spaces are Abelian groups w.r.t \(+\) but the addition of scalar multiplication provides an even richer structure.
Tip
Elements of vector space are called vectors.
We often omit the underlying scalar field \(\mathcal{F}\) and write the vector space as \(V\).
Example of finite dimensional vectors: Euclidean vectors \(\mathbb{R}^n\) where the scalar field is \(\mathbb{R}\) or complex vectors \(\mathbb{C}^n\) over the scalar field \(\mathbb{C}\).
Euclidean Vector Space¶
These defintions and theorems are based on Graybill.
Vector Space¶
[Definition] n-component Vector
Let \(n\) be a positive integer and let \(a_1,\cdots,a_n\) be elements from \(\mathcal{F}\). The ordered \(n\)-tuple \(\mathbf{a}=(a_1,\cdots,a_n)^T\) is defined as n-component (or \(n\times 1\) vector)
See also
Vector spaces with \(n\times 1\) vectors are denoted here by \(V_n\).
Theorem
Let \(R_n\) be the set of all \(n\times 1\) vectors for a fixed positive integer \(n\). Then \(R_n\) is a vector space.
Vector Subspace¶
[Definition] Vector Subspace
Let \(S_n\) be the subset of vectors in the vector space \(V_n\). If the set \(S_n\) itself is a vector space, then \(S_n\) is called a vector subspace of \(V_n\).
Theorem
If \(S_n\) is a subset of the vector space \(V_n\) such that, for every two vectors, \(\mathbf{s}_1\) and \(\mathbf{s}_2\) in \(S_n\), \(a_1\mathbf{s}_1+a_2\mathbf{s}_2\) is in \(S_n\) for all real numbers \(a_1\) and \(a_2\), then \(S_n\) is a vector subspace of \(V_n\).
Theorem
The set \(\{\mathbf{0}\}\) where \(\mathbf{0}\) is the \(n\times 1\) null-vector, is a subspace of every vector space \(V_n\). Every vector space \(V_n\) is a subspace of itself.
Linear Dependence and Independence¶
[Definition] Linear Dependence and Independence
Let \(\{\mathbf{v}_1,\cdots,\mathbf{v}_m\}\) be a set of \(m\) vectors each with \(n\) components, so that \(\mathbf{v}_i\in R_n;i=1,\cdots,m\). This set is defined to be linearly dependent if and only if there exists a set of scalars \(\{c_1,\cdots,c_m\}\), at least one of which is not equal to zero, such that
If the only set of scalars that satisfies the above is \(\{0,\cdots,0\}\), then the set of vectors is called linearly independent.
Theorem
If the vector \(\mathbf{0}\) is included in a set of vectors, the set is linearly dependent.
Theorem
If \(m > 1\) vectors are linearly dependent, it’s always possible to express at least one of them as a linear combination of the others.
Theorem
In the set of \(m\) vectors \(\{\mathbf{v}_1,\cdots,\mathbf{v}_m\}\), if there are \(s\) vectors, \(s\le m\), that are linearly dependent, then the entire set of vectors is linearly dependent.
Theorem
If the set of \(m\) vectors \(\{\mathbf{v}_1,\cdots,\mathbf{v}_m\}\) is a linearly independent set, while the set of \(m+1\) vectors \(\{\mathbf{v}_1,\cdots,\mathbf{v}_{m+1}\}\) is a linearly dependent set, then \(\mathbf{v}_{m+1}\) can be expressed as a linear combination of \(\mathbf{v}_1,\cdots,\mathbf{v}_m\).
Theorem
A necessary and sufficient condition for thet set of \(n\times 1\) vectors \(\{\mathbf{v}_1,\cdots,\mathbf{v}_m\}\) to be linearly dependent set is that the rank of the matrix formed by the vectors (as columns) is less than the number of vectors \(m\); that is \(r < m\).
Theorem
If the rank of the matrix of a set of \(n\times 1\) vectors \(\{\mathbf{v}_1,\cdots,\mathbf{v}_m\}\) is \(r\), then \(r\) must be less than or equal to \(m\), and if \(r > 0\), theree exists exactly \(r\) of those vectors that are linearly independent, while each of the other \(m-r\) (if \(m-r > 0\)) vectors expressible as a linear combination of these \(r\) vectors.
Theorem
The set of \(n\times 1\) vectors \(\{\mathbf{v}_1,\cdots,\mathbf{v}_m\}\) is always linearly independent if \(m > n\).
Basis of a Vector Space¶
Theorem
Let \(\{\mathbf{v}_1,\cdots,\mathbf{v}_m\}\) be a set of vectors in \(V_n\) and let \(V\) be the set of vectors defined by
then \(V\) is a subspace of \(V_n\).
[Definition] Basis
Let \(\{\mathbf{v}_1,\cdots,\mathbf{v}_m\}\) be a set of linearly independent set of vectors that spans \(V_n\). Then this set is called a basis of \(V_n\). For the special vector space \(\{\mathbf{0}\}\), we shall say that \(\mathbf{0}\) is the basis (even though it’s not linearly independent).
Theorem
If \(\{\mathbf{v}_1,\cdots,\mathbf{v}_m\}\), \(\{\mathbf{u}_1,\cdots,\mathbf{u}_q\}\) are two bases for \(V_n\), then \(m=q\).
[Definition] Dimensions
Let \(V_n\) be any vector space except \(\{\mathbf{0}\}\). Let the number of vectors in the basis of \(V_n\) be \(m\). Then \(m\) is defined to be the dimension of \(V_n\). The dimension of \(\{\mathbf{0}\}\) is defined to be 0.
Theorem
Let \(\{\mathbf{v}_1,\cdots,\mathbf{v}_m\}\) be a basis for the vector space \(V_n\) (neq \(\{\mathbf{0}\}\)). Let \(\mathbf{v}\) be any vector in \(V_n\). There is oen and only one ordered set of scalars \(\{c_1,\cdots,c_m\}\) such that
In other words, \(\mathbf{v}\) is a unique linear combination of a given basis.
Theorem
If \(r\) is the rank of the matrix of vectors \(\mathbf{v}_1,\cdots,\mathbf{v}_m\) that span the vector space \(V_n\), then there are exactly \(r\) independent vectors in that set and every vector in \(V_n\) can be expressed uniquely as a linear combination of these \(r\) vectors.
Theorem
If the vector space \(V_n\) is spanned by a set of \(m\) vectors, and if the matrix of those vectors has a rank \(r\), then any set of \(r+1\) vectors in \(V_n\) is linearly dependent.
Theorem
Let \(\mathbf{V}=\begin{bmatrix}\mathbf{v}_1&\cdots&\mathbf{v}_m\end{bmatrix}\) be a matrix containing a set of vectors that is a basis for \(V_n\), and let \(\mathbf{U}=\begin{bmatrix}\mathbf{u}_1&\cdots&\mathbf{u}_q\end{bmatrix}\) be a matrix that is any set of vectors in \(V_n\). The set of vectors in \(\mathbf{U}\) is a basis set for \(V_n\) if and only if \(m=q\) and there exists a non-singular \(m\times m\) matrix \(\mathbf{A}\) such that \(\mathbf{U}=\mathbf{V}\mathbf{A}\).
Theorem
Let \(\{\mathbf{v}_1,\cdots,\mathbf{v}_m\}\), \(m > 1\), be a basis for the vector space \(V_n\) and let \(\mathbf{v}\) be any vector in \(V_n\) such that \(\mathbf{v}=\sum_{i=1}^m c_i\mathbf{v}_i\). If \(c_t\neq 0\) for some \(t\), then the set \(\{\mathbf{v}_1,\cdots\mathbf{v}_{t-1},\mathbf{v},\mathbf{v}_{t+1},\cdots\mathbf{v}_m\}\) is a basis for \(V_n\). However, if \(c_t=0\), then the set \(\{\mathbf{v}_1,\cdots\mathbf{v}_{t-1},\mathbf{v},\mathbf{v}_{t+1},\cdots\mathbf{v}_m\}\) is a linearly dependent set and hence not a basis for \(V_n\).
Theorem
Let \(\{\mathbf{v}_1,\cdots,\mathbf{v}_q\}\) be a linearly independent vectors of \(V_n\). Then this set is a subset of a basis for \(V_n\).
Inner Product and Orthogonality of Vectors¶
[Definition] Inner Product
Let \(\mathbf{x}\) and \(\mathbf{y}\) be two vectors in \(V_n\). The inner product of \(\mathbf{x}\) and \(\mathbf{y}\), \(\mathbf{x}\cdot\mathbf{y}\) is defined to be the scalar \(\sum_{i=1}^nx_iy_i\). It is the scalar that is the element in the \(1\times 1\) matrix \(\mathbf{x}^T\mathbf{y}\).
[Definition] Orthogonal Vectors
Let \(\mathbf{x}\) and \(\mathbf{y}\) be two vectors in \(V_n\). \(\mathbf{x}\) and \(\mathbf{y}\) are defined to be orthogonal if and only if the inner product is 0.
Tip
\(\mathbf{0}\) vector is orthogonal to every vector in \(V_n\).
[Definition] Normal Vectors
A vector \(\mathbf{x}\) in \(V_n\) is defined to be a normal vector if and only if the inner product of \(\mathbf{x}\) with itself is equal to \(+1\).
[Definition] Orthogonal and Orthonormal Basis
If \(\{\mathbf{v}_1,\cdots,\mathbf{v}_m\}\) is a basis for \(V_n\) such that \(\mathbf{v}_i^T\mathbf{v}_j\) for all \(i\neq j=1,\cdots m\), then the basis is defined to be orthogonal basis for \(V_n\). If in addition, \(\mathbf{v}_i^T\mathbf{v}_i=1\) for all \(i=1,\cdots m\), the basis is defined to be orthonormal basis.
Theorem
Every vector space has an orthogonal basis.
Theorem
Every vector space except \(\{\mathbf{0}\}\) has an orthonormal basis.
Theorem
Let \(\{\mathbf{v}_1,\cdots,\mathbf{v}_m\}\) be a set of vectors in \(V_n\) such that each and every distinct pair of vectors is orthogonal; that is \(\mathbf{v}_i^T\mathbf{v}_j=0\) for all \(i\neq j\). If none of the vectors is the zero vector, then the set of vectors is a linearly independent set.
Theorem
Any set of \(q\) nonzero pairwise orthogonal vectors in \(V_n\) is a subset of a basis for \(V_n\).
Theorem
Let \(\{\mathbf{v}_1,\cdots,\mathbf{v}_q\}\) be the set of basis for the vector space \(V_n (\neq \{\mathbf{0}\})\). Then the set of \(q\) vectors \(\{\mathbf{z}_1,\cdots,\mathbf{z}_q\}\) is also a basis vector for \(V_n\) and they are an orthonormal set where they are defined as