> For the complete documentation index, see [llms.txt](https://ee16a.gitbook.io/studee16a/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ee16a.gitbook.io/studee16a/linear-algebra/linear-equations/example-problems.md).

# Example Problems

### Examples

| Linear equations                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | Non-linear equations                                                                                                                                                                                                                 |
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| <p><span class="math">y = \frac{3}{2}x</span> <br> <span class="math">\frac{5}{4}x - \frac{1}{6}y = 3</span> <br> <span class="math">6\sqrt{2}a - 3.3b = c</span> <br> <span class="math">e^{1.72} \cdot \frac{\sqrt{\pi} - 1.6 + \frac{5}{6}}{44.5 - 5\sqrt{6.6}} \cdot x\_1 + \int\_{0}^{5} e^{-t^2 + \sqrt{5}t}dt \cdot x\_2 - \sum\_{i = 0}^{100} \sin(i) \cdot x\_3 = 2</span> <br> <span class="math">\begin{bmatrix}1.08 & 6.2 & 2.6\end{bmatrix}\begin{bmatrix}x\_1 \ x\_2 \ x\_3\end{bmatrix} = 4</span></p> | <p><span class="math">y = x^2</span> <br> <span class="math">a \cdot b - 5.3c = 5</span> <br> <span class="math">3x^3 - \frac{5}{2}y^5 - 54xy = \sqrt{z}</span> <br> <span class="math">\int\_{-5}^{x} 5\csc(t^2) dt = 2y</span></p> |

## System of linear equations

A system of linear equations consists of multiple equations that are solved simultaneously. The easiest method to solve systems of linear equations is through **Gaussian elimination**.

### Examples

$$\left{\begin{array}{l} \frac{1}{3}x\_1+\frac{1}{4}x\_2+\frac{1}{2}x\_3=\frac{15}{4} \ \frac{7}{2}x\_1+\frac{2}{3}x\_2+\frac{1}{3}x\_3=\frac{16}{3} \ \frac{1}{5}x\_2+\frac{3}{7}x\_3=\frac{34}{5} \ \end{array}\right.$$

$$a\_1\begin{bmatrix}1 \ 2 \ 1\end{bmatrix} + a\_2\begin{bmatrix}0 \ 1 \ 1\end{bmatrix} + a\_3\begin{bmatrix}2 \ 0 \ 0\end{bmatrix} = \begin{bmatrix}5 \ 2 \ -4\end{bmatrix}$$

$$\left\[\begin{array}{ccc|c}1 & -2 & 1 & 0 \ 0 & 5 & -2 & -1 \ 4 & 1 & 3 & 2\end{array}\right]$$

**Overview for the next topics:**

{% embed url="<https://youtu.be/k7RM-ot2NWY>" %}

## Linear combinations

A linear combination of a set of terms is a **sum** of each term **multiplied by a constant**. Since a linear combination is basically a weighted sum of each term, the constant coefficients can be thought of as “weights.”

### Examples

1. $$5\vec{a} + 7\vec{b}$$ is a linear combination of $$\vec{a}$$ and $$\vec{b}$$.
2. $$3\vec{a} + 0\vec{b} + 1\vec{c} = 3\vec{a} + \vec{c}$$ is a linear combination of $$\vec{a}, \vec{b}, \vec{c}$$.  (Note that we can multiply a vector by $$0$$ as well.)

### Span

Sometimes, we want to denote **all** linear combinations of a set of vectors $${\vec{a}\_1, \vec{a}\_2, \ldots, \vec{a}\_n}$$. To do this, we define the **span** of these vectors to be the set of all linear combinations of these vectors:

$$\text{span}{\vec{a}\_1, \vec{a}\_2, \ldots \vec{a}\_n} = {c\_1\vec{a}\_1 + c\_2\vec{a}\_2 + \cdots + c\_n\vec{a}\_n \mid c\_1, c\_2, \ldots, c\_n \in \mathbb{R}}$$

> **Note**: The zero vector $$\vec{0} \in \text{span}{\vec{a}\_1, \vec{a}\_2, \ldots \vec{a}\_n}$$ for any $$n$$ vectors because we can multiply all of these vectors by $$0$$ to obtain $$\vec{0}$$.

### Examples

$$\text{span}\left{\begin{bmatrix}1 \ 0 \ 0\end{bmatrix}, \begin{bmatrix}0 \ 1 \ 0\end{bmatrix}\right}$$ is basically the $$x$$-$$y$$ plane in $$\mathbb{R}^3$$ because every vector lying on this plane can be expressed as a sum of each vector multiplied by a constant, e.g. $$\begin{bmatrix}5 \ 3 \ 0\end{bmatrix} = 5\begin{bmatrix}1 \ 0 \ 0\end{bmatrix} + 3\begin{bmatrix}0 \ 1 \ 0\end{bmatrix}$$.

> **Note**: This plane is **not** the same as $$\mathbb{R}^2$$. Each point on the $$x$$-$$y$$ plane in $$\mathbb{R}^3$$ has 3 components while all points in $$\mathbb{R}^2$$ only have 2. In general, $$\mathbb{R}^2 \not\subset \mathbb{R}^3$$.

Similarly, $$\text{span}\left{\begin{bmatrix}1 \ 0\end{bmatrix}, \begin{bmatrix}0 \ 1\end{bmatrix}\right} = \mathbb{R}^2$$ because every vector in $$\mathbb{R}^2$$ can be written as a linear combination of $$\begin{bmatrix}1 \ 0\end{bmatrix}$$ and $$\begin{bmatrix}0 \ 1\end{bmatrix}$$.

### Calculation

Suppose we wanted to find out whether a vector $$\vec{b}$$ is in the span of a set of vectors $${\vec{a}\_1, \vec{a}\_2, \ldots, \vec{a}\_n}$$.

Basically, we are solving trying to find constants $$c\_1, c\_2, \ldots, c\_n$$ such that $$c\_1\vec{a}\_1 + c\_2\vec{a}\_2 + \cdots + c\_n\vec{a}\_n = \vec{b}$$

This looks like a system of linear equations, so we can rewrite this in matrix notation: $$\begin{bmatrix}| & | & & | \ \vec{a}\_1 & \vec{a}\_2 & \cdots & \vec{a}\_n\ | & | & & |\end{bmatrix}\begin{bmatrix}c\_1 \ c\_2 \ \vdots \ c\_n\end{bmatrix} = \vec{b}$$

We can then transform the equation into an augmented matrix: $$\left\[\begin{array}{cccc|c}| & | & & | & | \ \vec{a}\_1 & \vec{a}\_2 & \cdots & \vec{a}\_n & \vec{b} \ | & | & & | & |\end{array}\right]$$

Therefore, solving a system of linear equations is basically finding how we can express the vector $$\vec{b}$$ as a linear combination of the column vectors $$\vec{a}\_1, \vec{a}\_2, \ldots, \vec{a}\_n$$. We can conclude:

1. If the system of linear equations is consistent, then $$\vec{b}$$ is in the span of the vectors $$\vec{a}\_1, \vec{a}\_2, \ldots, \vec{a}\_n$$, so $$\vec{b}$$ is a linear combination of the vectors $$\vec{a}\_1, \vec{a}\_2, \ldots, \vec{a}\_n$$.
2. If the system of linear equations is inconsistent, then $$\vec{b}$$ is **not** in the span of the vectors $$\vec{a}\_1, \vec{a}\_2, \ldots, \vec{a}\_n$$, i.e. $$\vec{b}$$ is **not** a linear combination of the vectors $$\vec{a}\_1, \vec{a}\_2, \ldots, \vec{a}\_n$$.

### Example 1

Determine whether $$\begin{bmatrix}5 \ 4 \ 3\end{bmatrix}$$ is in the span of $$\begin{bmatrix}1 \ 0 \ 1\end{bmatrix}$$ and $$\begin{bmatrix}1 \ 2 \ 0\end{bmatrix}$$.

We can just solve the augmented matrix:

$$\left\[\begin{array}{cc|c}1 & 1 & 5 \ 0 & 2 & 4 \ 1 & 0 & 3\end{array}\right] \sim \left\[\begin{array}{cc|c}1 & 1 & 5 \ 0 & 1 & 2 \ 0 & 1 & 2\end{array}\right] \sim \left\[\begin{array}{cc|c}1 & 0 & 3 \ 0 & 1 & 2 \ 0 & 0 & 0\end{array}\right]$$

Since the system of linear equations is consistent, $$\begin{bmatrix}5 \ 4 \ 3\end{bmatrix}$$ is in the span of $$\begin{bmatrix}1 \ 0 \ 1\end{bmatrix}$$ and $$\begin{bmatrix}1 \ 2 \ 0\end{bmatrix}$$. In fact, $$\begin{bmatrix}5 \ 4 \ 3\end{bmatrix} = 3\begin{bmatrix}1 \ 0 \ 1\end{bmatrix} + 2\begin{bmatrix}1 \ 2 \ 0\end{bmatrix}$$.

### Example 2

Determine whether $$\begin{bmatrix}1 \ 0 \ 1\end{bmatrix}$$, $$\begin{bmatrix}1 \ 2 \ 0\end{bmatrix}$$, and $$\begin{bmatrix}5 \ 4 \ 3\end{bmatrix}$$ span $$\mathbb{R}^3$$.

Let's row-reduce the matrix:

$$\begin{bmatrix}1 & 1 & 5 \ 0 & 2 & 4 \ 1 & 0 & 3\end{bmatrix} \sim \begin{bmatrix}1 & 1 & 5 \ 0 & 2 & 4 \ 0 & 1 & 2\end{bmatrix} \sim \begin{bmatrix}1 & 1 & 5 \ 0 & 1 & 2 \ 0 & 0 & 0\end{bmatrix}$$

Observe that there is no pivot in the third row. This means that this system of linear equations is not consistent for all vectors in $$\mathbb{R}^3$$, so $$\begin{bmatrix}1 \ 0 \ 1\end{bmatrix}$$, $$\begin{bmatrix}1 \ 2 \ 0\end{bmatrix}$$, and $$\begin{bmatrix}5 \ 4 \ 3\end{bmatrix}$$ do not span $$\mathbb{R}^3$$.

In other words, we could have row-reduced the following augmented matrix:

$$\left\[\begin{array}{ccc|c}1 & 1 & 5 & b\_1 \ 0 & 2 & 4 & b\_2 \ 1 & 0 & 3 & b\_3\end{array}\right] \sim \left\[\begin{array}{ccc|c}1 & 1 & 5 & b\_1 \ 0 & 2 & 4 & b\_2 \ 0 & 1 & 2 & b\_1 - b\_3\end{array}\right] \sim \left\[\begin{array}{ccc|c}1 & 1 & 5 & b\_1 \ 0 & 1 & 2 & \frac{b\_2}{2} \ 0 & 0 & 0 & b\_2 - 2(b\_1 - b\_3)\end{array}\right]$$

Let's pick a vector $$\vec{b} = \begin{bmatrix}b\_1 \ b\_2 \ b\_3\end{bmatrix} \in \mathbb{R}^3$$, such that $$b\_2 - 2(b\_1 - b\_3) \neq 0$$, such as $$\vec{b} = \begin{bmatrix}1 \ 0 \ 0\end{bmatrix}$$. After row reduction, you will see that $$\begin{bmatrix}1 \ 0 \ 0\end{bmatrix} \notin \text{span}\left{\begin{bmatrix}1 \ 0 \ 1\end{bmatrix}, \begin{bmatrix}1 \ 2 \ 0\end{bmatrix}, \begin{bmatrix}5 \ 4 \ 3\end{bmatrix}\right}$$.

In general, there needs to be **pivots in every row** for a set of vectors to span $$\mathbb{R}^n$$.

## Linear independence

A set of vectors $${\vec{a}\_1, \vec{a}\_2, \ldots, \vec{a}\_n}$$ is said to be **linearly independent** if and only if **the only solution** to the equation

$$c\_1\vec{a}\_1 + c\_2\vec{a}\_2 + \cdots + c\_n\vec{a}\_n = \vec{0}$$

is the trivial solution, $$c\_1 = c\_2 = \cdots = c\_n = 0$$.

In other words, a set of vectors is linearly independent if and only if **none** of its elements is a linear combination of the other elements. Else, the set is considered **linearly dependent**.

> **Note**: A set containing the zero vector $$\vec{0}$$ is automatically linearly dependent because there is always a non-trivial set of scalars (the constant corresponding to the zero vector can be anything).

### Calculation

To determine whether a set of vectors is linearly independent, we find solutions to the equation

$$c\_1\vec{a}\_1 + c\_2\vec{a}\_2 + \cdots + c\_n\vec{a}\_n = \vec{0}$$.

$$\begin{bmatrix}| & | & & | \ \vec{a}\_1 & \vec{a}\_2 & \cdots & \vec{a}\_n \ | & | & & |\end{bmatrix}\begin{bmatrix}c\_1 \ c\_2 \ \vdots \ c\_n\end{bmatrix} = \vec{0}$$

$$\left\[\begin{array}{cccc|c}\vec{a}\_1 & \vec{a}\_2 & \cdots & \vec{a}\_n & \vec{0}\end{array}\right]$$

Therefore, we just row-reduce the augmented matrix $$\left\[\begin{array}{cccc|c}\vec{a}\_1 & \vec{a}\_2 & \cdots & \vec{a}\_n & \vec{0}\end{array}\right]$$. Since every row operation will preserve the $$0$$ in the last column of the augmented matrix, we usually omit the last column and only row-reduce the matrix $$\begin{bmatrix}\vec{a}\_1 & \vec{a}\_2 & \cdots & \vec{a}\_n\end{bmatrix}$$.

Since $$c\_1 = c\_2 = \cdots = c\_n = 0$$ if the set if linearly independent, we should not find any free variables (which could be any value). Therefore, we should find **pivots in every column**.

> **Note**: For a set of $$m$$ vectors of size $$n$$:
>
> * If we have pivots in every **row**, we know that the vectors span the entire subspace $$\mathbb{R}^n$$.
> * If we have pivots in every **column**, we know that the vectors are linearly independent.
>
> **Note**: An $$m \times n$$ matrix, where $$m < n$$, has at most $$m$$ pivots. Therefore, there can never be pivots in each columns (or there would have to be $$n$$ pivots), so the column vectors of such a matrix are automatically linearly dependent. In other words, if the number of vectors in a set exceeds the number of elements in each vector, then this set is automatically linearly dependent. Furthermore, having a “wide” matrix does not guarantee that the set of vectors span $$\mathbb{R}^m$$.

### Example 1

Determine whether $$\begin{bmatrix}1 \ 0 \ 1\end{bmatrix}$$, $$\begin{bmatrix}1 \ 2 \ 0\end{bmatrix}$$, and $$\begin{bmatrix}5 \ 4 \ 3\end{bmatrix}$$ are linearly independent.

We first row-reduce the matrix:

$$\begin{bmatrix}1 & 1 & 5 \ 0 & 2 & 4 \ 1 & 0 & 3\end{bmatrix} \sim \begin{bmatrix}1 & 1 & 5 \ 0 & 1 & 2 \ 0 & 0 & 0\end{bmatrix}$$

We see that there is no pivot in the third column. Therefore, the given vectors are **linearly dependent**.

### Example 2

Determine whether $$\begin{bmatrix}1 \ 0 \ 1\end{bmatrix}$$ and $$\begin{bmatrix}1 \ 2 \ 0\end{bmatrix}$$ are linearly independent.

Let's row-reduce the matrix:

$$\begin{bmatrix}1 & 1 \ 0 & 2 \ 1 & 0\end{bmatrix} \sim \begin{bmatrix}1 & 1 \ 0 & 2 \ 0 & 1\end{bmatrix} \sim \begin{bmatrix}1 & 1 \ 0 & 1 \ 0 & 0\end{bmatrix}$$

We observe that there is a pivot in every column of the matrix. Therefore, the given vectors are **linearly independent**.

### Extra Resources

#### Links to Guerrilla Section Problems and solutions

#### Guerilla section 1 Problems

<https://d1b10bmlvqabco.cloudfront.net/attach/j6h2cdky5zk7c6/hz1x03ot4mb2mj/j7f2i2m14rgn/gs1_final.pdf>

#### Guerilla section 1 Solutions

<https://d1b10bmlvqabco.cloudfront.net/attach/j6h2cdky5zk7c6/hz1x03ot4mb2mj/j7f6yufr2rvo/gs1_solutions.pdf>

#### Links to Exam Problems

<https://tbp.berkeley.edu/courses/ee/16A/>

<https://hkn.eecs.berkeley.edu/exams/course/ee/16a>

#### Vidoes elaborating guassian elimination

<https://www.youtube.com/watch?v=2j5Ic2V7wq4> <https://www.youtube.com/watch?v=xCIXkm3-ocQ> <https://www.youtube.com/watch?v=g0h_s1p0ax0>