Linear Combinations

Definition: Linear Combination

Let ${\mathbf{v}}_1,\cdots ,{\mathbf{v}}_n$ be vectors in a vector space $(V,F,+,\cdot )$.

We say that $\mathbf{v}\in V$ is a linear combination of ${\mathbf{v}}_1,\cdots ,{\mathbf{v}}_n$ if there exist $c_i\in F$ such that

$$\mathbf{v}=c_1{\mathbf{v}}_1+\cdots +c_n{\mathbf{v}}_n=\sum _{i=1}^n{c}_i{\mathbf{v}}_i$$

Span

Definition: Span

Let ${\mathbf{v}}_1,\cdots ,{\mathbf{v}}_n$ be vectors in a vector space $(V,F,+,\cdot )$.

The span of ${\mathbf{v}}_1,\cdots ,{\mathbf{v}}_n$ is the set of all linear combinations which can be constructed from them.

$$\{c_1{\mathbf{v}}_1+\cdots +c_n{\mathbf{v}}_n\mid c_i\in F\}$$

Notation

$$⟨\{{\mathbf{v}}_1,{\mathbf{v}}_1,\cdots \}⟩⟨{\mathbf{v}}_1,{\mathbf{v}}_1,\cdots ⟩\mathrm{span}(\{{\mathbf{v}}_1,{\mathbf{v}}_1,\cdots \})\mathrm{span}({\mathbf{v}}_1,{\mathbf{v}}_1,\cdots )$$

Definition: Spanning Set (Generator)

Let $(V,F,+,\cdot )$ be a vector space.

A set of vectors $S=\{{\mathbf{v}}_1,{\mathbf{v}}_2,\cdots \}$ is a spanning set (or a generator) of $(V,F,+,\cdot )$ if the span of $S$ is $V$

$$\mathrm{span}({\mathbf{v}}_1,{\mathbf{v}}_2,\cdots )=V$$

TIP

This essentially means that each vector in $V$ can be expressed as a linear combination of some vectors in $S$.

Definition: Minimality

A spanning set $S$ is minimal if there is no vector ${\mathbf{v}}_i\in S$ such that $S\setminus \{{\mathbf{v}}_i\}$ is still a spanning set.

TIP

This means that there is no way to remove a vector from a minimal spanning set and still obtain a spanning set of the vector space.

Definition: Finitely Generated Vector Space

A vector space is finitely generated if there exists a finite spanning set for it.

Linear Independence

Definition: Linear Independence

Let ${\mathbf{v}}_1,\cdots ,{\mathbf{v}}_n$ vectors in a vector space $(V,F,+,\cdot )$.

We say that ${\mathbf{v}}_1,\cdots ,{\mathbf{v}}_n$ are linearly independent if

$$c_1{\mathbf{v}}_1+\cdots +c_n{\mathbf{v}}_n=0⟹c_1=\cdots =c_n=0$$

Tip

Essentially, the vectors ${\mathbf{v}}_1,\dots ,{\mathbf{v}}_n$ are linearly independent if the only way to express the zero vector $0$ as a linear combination of ${\mathbf{v}}_1,\dots ,{\mathbf{v}}_n$ is when the coefficients before ${\mathbf{v}}_1,\dots ,{\mathbf{v}}_n$ are all zero.

Definition: Maximality

A set of linearly independent vectors $S=\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$ from a vector space $(V,F,+,\cdot )$ is maximal if there is no $\mathbf{v}\in V\setminus S$ such that $S\cup \{\mathbf{v}\}$ the union is still linearly independent.

TIP

This means that no matter how much we try, we cannot find any vector outside $S$ such that if we add it to $S$, we would still end up with a linearly independent set.

Theorem: Size Limit for Linearly Independent Sets

The number of elements in any set $I$ of linearly independent vectors from a finitely generated.md) Vector Spaces $(V,F,+,\cdot )$ is always less than or equal to the Bases of $V$.

$$|I|\le \dim (V)$$

PROOF

Let $B=\{{\mathbf{b}}_1,\cdots ,{\mathbf{b}}_n,\}$ be a Bases of $V$ and let $I=\{{\mathbf{v}}_1,\cdots ,{\mathbf{v}}_m\}$ be a set of linearly independent vectors.

According to the Bases there are $n-m$ vectors in $B$ which form a basis with the vectors from $I$. This means that $n-m$ cannot be negative and thus the proof is complete.

Linear Dependence

Definition: Linear Dependence

Let ${\mathbf{v}}_1,\cdots ,{\mathbf{v}}_n$ be vectors in some Vector Spaces $(V,F,+,\cdot$).

We say that ${\mathbf{v}}_1,\cdots ,{\mathbf{v}}_n$ are linearly dependent iff they are not linearly independent, i.e. there exist $c_1,\cdots ,c_n$ with at least one $c_i\ne 0$ such that

$$c_1{\mathbf{v}}_1+\cdots +c_n{\mathbf{v}}_n=0$$

Theorem: Linear Dependence $⟹$ Linear Combination

If $\{{\mathbf{v}}_1,\cdots ,{\mathbf{v}}_n\}(n\ge 2)$ are linearly dependent vectors, then there is at least one ${\mathbf{v}}_i\in \{{\mathbf{v}}_1,\cdots ,{\mathbf{v}}_n\}$ which can be expressed as a Linear Combinations of the rest of the vectors:

$${\mathbf{v}}_i=\sum _{\begin{array}{rl}j& =1\\ j& \ne i\end{array}}^n{c}_j{\mathbf{v}}_j$$

PROOF

According to the definition of linear dependence, there are coefficients $h_1,\cdots ,h_n\in F$ with at least one $h_i\ne 0$ such that

$$h_1{\mathbf{v}}_1+\cdots +h_i{\mathbf{v}}_i+\cdots +h_n{\mathbf{v}}_n=0$$

Let’s move everything except $h_i{\mathbf{v}}_i$ to the other side of the equation:

$$h_i{\mathbf{v}}_i=-h_1{\mathbf{v}}_1+\cdots -h_{i-1}{\mathbf{v}}_{i-1}-h_{i+1}{\mathbf{v}}_{i+1}+\cdots +-h_n{\mathbf{v}}_n$$

Since $h_i\ne 0$, we can divide both sides by it:

$${\mathbf{v}}_i=c_1{\mathbf{v}}_1+\cdots +c_{i-1}{\mathbf{v}}_{i-1}+c_{i+1}{\mathbf{v}}_{i+1}+\cdots +c_n{\mathbf{v}}_n$$

We have thus obtained ${\mathbf{v}}_i$ as a linear combination of the other vectors, where $c_j=-\frac{h_j}{h_i}$