June 2022 edits: I wrote this page 2 years during my Advanced Linear Algebra course at Penn ago and gave gotten much better at math, exposition, and math typesetting since then. I am slowly working through it to make it more readable

Composition of Linear Maps
- Properties of Compsition of Linear Maps
Dual Spaces and Isomorphism

Composition of Linear Maps

Recall that if $X,Y,Z$ are all sets, and $f: X \mapsto Y$ and $g: Y \mapsto Z$ if we have a mapping $f$ from $X$ to $Y$ and a mapping $g$ from $Y$ to $Z$, we can form a composite map:

$(g \circ f): X \mapsto Z$ that is defined by $g \circ f = g(f(x)) \) for all $ x \in X $
we already used the notion of composition to define invertible/bijective maps of sets.
composition also preserves the property of maps being linear!! verycool we state it formally and prove it:
Claim: The composition of two linear maps is also linear. Let U, V, W be $ \mathbb{F} $ vector spaces.
- Let these be linear maps:
- $ T: U \mapsto V $ and $ S: V \mapsto W $
- the compositon of these is also linear:
- $ S \circ T : U \mapsto W $
proof: Let $ x,y \in V $, $ c \in K $. Then:
- $ S \circ T(x+y) = S(T(x+y)) = S(T(x) + T(y)) = S(T(y)) + S(T(y)) = S \circ T(x) + S \circ T(y) $
- also, $ S \circ T(cx) = S(T(cx)) = S(cT(x)) = cS(T(x)) = cS \circ T(x) $
- QED
Thus the operation of compositions gives yet another way of constructing new linear maps out of known ones.
Under our dictionary that uses bases to convert linear maps to matrices and matrices back to linear maps (QUESTION: lets see an example)
- => the natural operations on linear maps that produce new linear maps just become the natural operations of matrices (that produce new matrices?)
- Theorem: Let U, V, W, be $ \mathbb{F} $ -vector spaces with bases:
  - $ \mathbb{E} = { e_{1},…….,e_{n} } \subset U $
  - $ \mathbb{F} = { f_{1},…….,f_{n} } \subset V $
  - $ \mathbb{G} = { g_{1},…….,g_{n} } \subset W $
- (1) : If $ T_{1}: U \mapsto V, T_{2}: U \mapsto V $ are linear mappings, $ c \in \mathbb{F} $
  - and $ A_{1}, A_{2} \in Mat_{mxn}\mathbb{F}$ are matrices of $ T_{1}, T_{2} $ in the bases $ \mathbb{E}, \mathbb{F} $
  - then matrices of :
    - $ T_{1} + T_{2}: U \mapsto V $ == $ A_{1} + A_{2} $
    - $ cT_{1}: U \mapsto V $ == $ cA_{1} $
  - in our bases $ \mathbb{E} $ and $ \mathbb{F} $
- (2) : (Linear mapping composition corresponds to matrix multiplcation): if $ T: U \mapsto V $ and $ S: V \mapsto W $ are linear mappings and
  - T has an (m x n) matrix A with bases E (from U) and F (from V) [bases from domain and codomain]
  - S has a (k x m) matrix B with bases F (from V) and G (from W) [bases from domain and codomain]
- Then $ S \circ T $ has an (k x n) matrix B*A in the bases E (from domain U) and G (from codomain W)
- proof:
  - (1) follows from the definition of the matrix of a linear map
  - (2) Consider the linear map $ f:= S \circ T: U \mapsto W $ and let C be the matrix of of that operator $ S \circ T $ in the bases E (from domain U) and G (from codomain W)
    - every entry of C $ c_{pq} $ is given by:
    - $ c_{qp} = $ the q-th coordinate of the vector $ f_{e_{p}} $ in the basis $ \mathbb{G} = (g_{1}…g_{n}) $
    - we can actually compute each entry $ c_{qp} = f_{e_{p}} $ directly:
    - $ f_{e_{p}} = S \circ T(e_{p}) $
    - = $ S(T(e_{p})) $
    - = $ S(A_{1p}f_{1} +…+ A_{mp}f_{m}) $ QUESTION: where does m come from?
    - = $ \sum_{r=1}^{m} A_{rp} (S_{f_{r}}) $
    - = $ \sum_{r=1}^{m} A_{rp} (\sum_{q=1}^{k} B_{qr} g_{q})$
    - = $ \sum _{1}^{m} (\sum_{1}^{m} B_{qr}A_{rp}) g_q{} $
    - so we have $ c_{qp} = \sum_{1}^{m} B_{qr}A_{rp} $
    - = (row q of B)(column p of A) = $ (BA)_{qp} $
- Example 1 : (General Reflections)
  - Let $ L \in \mathbb(R)^{2}$ be any line through the origin and let the transformaition $s_{L} : R^{2} \mapsto R^{2}$ be the reflection across L
  - so $s_{L}$ send a point in R^{2} to its mirrror image, where them mirror is L
  - Claim : $s_{L}$ is a linear map.
  - proof: you can prove this directly using difficult geometric constructions, or you can use the fact that compositions of linear maps are linear:
  - S1: show that the reflection across the x-axis is linear:
    - $s = s_{x-axis} : \mathbb{R^{2} \mapsto \mathbb{R^{2}}}$
    - \s(x, y)^{t} = (x -y)^{t}\) for all vectors x,y
    - reflection over x axis matrix ==
    - A = $ \begin{bmatrix} 1 & 0 \
      0 & -1 \
      \end{bmatrix}
      $
  - S2: notice that reflection across L can be viewed as a composition of three linear maps:
    - 1. rotate the plane so that L becomes the x-axis
    - 1. reflect the plane across the x axis
    - 1. rotate the plane so that x-axis becomes L
  - each of these is linear, so $s_{L}$ is linear.
  - S3: in concrete terms, $\theta : $ measured in radians, counter closckwise direction, Then,
    - $s_{L} = rot_{\theta} \circ s \circ rot_{-\theta}$
    - use the previous theorem, and we can get matrices for these transformations:
    - $A_{rot_{\theta}}$ = $ \begin{bmatrix} cos(\theta) & -sin(\theta) \
      sin(\theta) & cos(\theta) \
      \end{bmatrix}
      $
    - $A_{rot_{\theta}}$ = $ \begin{bmatrix} cos(\theta) & sin(\theta) \
      -sin(\theta) & cos(\theta) \
      \end{bmatrix}
      $ QUESTION + $A_{s}$ = reflection over x axis matrix
    - EXERCISE: derive these matrices
    - and so $A_{s_{L}}$ = multiplication/composition of the three: = $ \begin{bmatrix} cos(2\theta) & sin(2\theta) \
      sin(2\theta) & -cos(2\theta) \
      \end{bmatrix}
      $
- EXERCISE: Compute the matrix of the reflection over the line L with respect to the line L: y =7x in the standard basis of $R^{2}$

Properties of Compsition of Linear Maps

(1): if $T \in L(V,W)$ and $S \in L(U,V)$ - then $T \circ S \in L(U,W)$

(2) Composition is a linear map in each argument. if
- $ S \in L(U,V) $
- then the map $ S \circ (-): L(V,W) \mapsto L(U,W) $ is a linear map between the two vector spaces L(V,W), L(U,V)
- example: if $a, b \in \mathbb{F}$, $T_{1}, T_{2} \in L(V,W)$
- then $S \circ (aT_{a} + b_{T_{2}})$
- = $ aS \circ T_{1} + bS \circ T_{2} $.
- check: we can check this directly! just check both sides are equal to the same thing when evaluated on any vect $u \in U$ (because U is domain of S? That’s where we start)
- from the definition of addition and scaling in the vector space of linear maps, here it is!
- = $S \circ (aT_{1}+bT_{2})(u)$
- = $S \circ ((aT_{1}+bT_{2})(u))$
- = $S \circ (aT_{1}(u) + bT_{2}(u))$
- = $ aS(T_{1}(u)) + bS(T_{2}(u))) $
- = $ a S \circ T_{1}(u) + b S \circ T_{2}(u)$
- QED
- QUESTION: CHECK WHETHER LINEAR MAPS IN QUESTION ARE CORRECT: THEY MIGHT BE FLIPPED IN THE NOTES! for this next one too
- similarly: $T \in L(V,W)$, check directly that $(-) \circ T : L(U,V) \mapsto L(U,W)$ is linear
  - because $(aS_{1} + bS_{2}) \circ T = aS_{1} \circ T + b S_{2} \circ T$ for all a,b scalars and $S_{1}, S_{2} \in L(U,V)$
(3): Inverse maps of linear maps are linear.
- Suppose $T: V \mapsto W$ is al inear map which is bijective (is invertible as a map of sets + is injective and surjective + is one to one and onto)
- consider the inverse map $T_{-1}: W \mapsto V$,
- then $T_{-1}$ is also linear.
- proof: let $y_{1}, y_{2} \in W$ - the domain of inverse map, let $a_{1}, a_{2} be two scalars in \mathbb{F}$
  - WTS: $ T_{-1}(a_{1}y_{1} + a_{2}y_{2}) = a_{1}T^{-1}(y_{1}) + a_{2}T^{-1}(y_{2}) $
  - assign some names: $ x_{1} = T^{-1}(y_{1}) \in V $ and $x_{2} = T^{-1}(y_{2}) \in V $ - V =codomain of $T^{-1}$
  - then the RHS $ a_{1}T^{-1}(y_{1}) + a_{2}T^{-1}(y_{2}) $ is equal to $a_{1}x_{1} + a_{2}x_{2} \in V$
  - Let’s evaluate the original map T on this vector. Remember T is linear and we get:
  - $ T(a_{1}x_{1} + a_{2}x_{2}) = a_{1}T(x_{1}) + a_{2}T(x_{2}) $
  - = $ a_{1}T \circ T^{-1}(y_{1}) + a_{2}T \circ T^{-1}(y_{2})$
  - = $ a_{1}y_{1} + a_{2}y_{2} $
  - Thus, $T(a_{1}x_{1} + a_{2}x_{2}) = a_{1}y_{1} + a_{2}y_{2}$
  - Now let’s evaluate $T^{-1}$ on both sides of this identity. Literally just:
  - $T^{-1} \circ T(a_{1}x_{1} + a_{2}x_{2})$ = $ T^{-1} (a_{1}y_{1} + a_{2}y_{2})$
  - => $ a_{1}x_{1} + a_{2}x_{2}$ = $ T^{-1} (a_{1}y_{1} + a_{2}y_{2})$
  - $ a_{1}T^{-1}(y_{1}) + a_{2}T^{-1}(y_{2})$ = $ T^{-1} (a_{1}y_{1} + a_{2}y_{2})$
  - we have shown the linearity of the inverse map $T^{-1}$
isomorphism (def): the linear map $T : V \mapsto W$ is an isomorphism if there exists a linear map $S: W \mapsto V$ s.t $S \circ T = id_{v}$ and $T \circ S = id_{v}$ (also think about corresponding definition in terms of matrices)more info
- note: we checked above that a linear map is an isomorphism iff it is a bijection
- note: we also checked that the inverse linear map is simply the inverse set theoretic map.
isomorphism (def): two vector spaces V,W over $\mathbb{F}$ are isomorphic if there exists a linear map $T: V \mapsto W$ which is an isomorphism (so if it is bijective).
- isomorphic means “has same shape”. useful for turning undamiliar algebraic objects into familiar ones, making them easier to work with.
- note: as with sets we will not distinguish isomorphic (vector) spaces.
  - the rational is tht using the isomorphism and its inverse we can transport any property of V to W and back again
  - these preperties and feauees that only involce teh addition and scaling of vectors are matched in V and W via the isomorphism and its inverse.
Examples of Isomorphisms and Isomorphic Spaces:
more examples
Example (1): (iso that sends linear combination vector to its coordinates and back)If V is a finite dimensional vector space over $\mathbb{F}$
- then every choice of a basis $\mathbb{B} = \{ b_{1},..,b_{n} \}$ gives you an isomorphism, namely: $ [-]_{\mathbb{B}} : V \mapsto \mathbb{F} ^{n}$ between V and coordinate n space
- so what does this map do?
- $[-]_{\mathbb{B}}$ assigns to each $ v \in V$ the column vector of its coordinates in the basis $\mathbb{B}$
- what this means is that each $ v \in V$ has a unique representation as $v = \sum_{i=1}^{n} x_{i}b_{i}$ with $ (x_{1},..,x_{n}) \in V, b_{1}…b_{n} \in \mathbb{B} $ (in the vector space V!!)
- this vector gets sent to a column vector of the coordinates from the field. $ [v]_{B} := (x_{1}…x_{n})^{T} $
- conversely, the inverse map $[-]^{-1} : \mathbb{F} ^{n} \mapsto V$ is also easy to write explicitly. Sends a column vector of coordinates from the scalar field to the linear combination representation of the vector v.
Notation: we can write and see the linear combination $\sum_{i}^{n} x_{i}b_{i}$ as a formal matrix product $\mathbb{B} \cdot x$ where
- $\mathbb{B} = (b_{1},…,b_{n})$ == the row vector of vectors (in the basis)
- $x = (x_{1},…,x_{n})^{t}$ is the column vector of coordinates in $\mathbb{F} ^{n}$
- with this new notation, new view, we can look at the inverse map of $[-]_{B}: V \mapsto F^{n}$ as the “matrix multiplication by B map”: $B \cdot (-) : \mathbb{F} ^{n} \mapsto V$
further notes on example 1: note that the isomorphism [-] depends on your choice of a basis.
- different bases will give rise to different isomophism.
- another way to descrube $[-]_{\mathbb{B}} : V \mapsto \mathbb{F} ^{n}$ is to say that it is the unique linear map that sends the basis B of V to the standard basis $E = \{e_{1},…,e_{n}\} $ of $\mathbb{F} ^{n}$.
  - so the mapping $[-]_{\mathbb{B}}$ is the unique linear map for which $ [b_{i}]_{\mathbb{B}} = e_{i}$ for i = 1,…,n
  - in vector notation : $ [\mathbb{B}]_{\mathbb{B}} = E $
(2): Let V,W be vector spaces over $\mathbb{F}$ with bases
- $E = \{ e_{1},…,e_{n} \} \subset V$
- $F = \{ f_{1},…,f_{n} \} \subset W$
- now given a linear map $T: V \mapsto W$ let $A_{T} \in Mat(mxn)(\mathbb{F})$ be its matrix (T’s matrix) in the bases E and F.
- Then the assignment $A_{(-)}: L(V,W) \mapsto Mat(mxn)(\mathbb{F}) :: T \mapsto A_{T}$ is a linear map.
  - this follows from part (1) of the first theorem above.
- And $A_{(-)}$ is also an isomorphism. (it’s bijective)
  - check:
    - first, by definition, $ A_{T} $ is the unique matrix such that $ T(E) = T(e_{1} + .. + T(e_{n})) = F \cdot A_{T} $
      - QUESTION: why is this true? are $T(e_{1}, …)$ are column vectors?
    - This fact guides us on how to compute values of T in terms of $A_{T}$: (Here’s how):
      - if $v \in V$ and we have $x \in F^{n}$ := vector of coordinates of v in the basis E,
      - then $v = Ex$
      - by linearity of T we have:
      - $ T(v) = T(Ex) = T(E) \cdot x = F \cot A_{T}x$
      - recalling that $x = [v]_{E} \in F^{n}$ is the vector of the cordinates of v in the basis E, we now have a formula for T in terms of A_{T}, its matrix:
        
        $T(v) = F A_{T}[v]_{E}$
      - this derivation also gives us the inverse of the map $A_{(-)} : T \mapsto A_{T}$,
        
        $A_{(-)} ^{-1} : Mat(mxn)(\mathbb{F}) \mapsto L(V,W) :: A \mapsto FA[-]_{\mathbb{E}}$
        
        QUESTION: so what does this actually look like? need examples
Example (3): variant of example 1 (T that sends bases to bases is an iso)
Let V,W be vector spaces over $\mathbb{F}$ with the bases:
- $E = \{ e_{1},…,e_{n} \} \subset V $
- $F = \{ f_{1},…,f_{n} \} \subset W $
Then the unique linear map $T: V \mapsto W$ which sends the basis E to the basis F is an isomorphism!
Proof: Let T be as defined above with bases E and F, for which $T(E) = F$
and let $S: W \mapsto V$ be another such that $S(F) = E$
both T and S exist and are unique thanks to the theorem we proved above and lectures 7,8.
Then $S \circ T : V \mapsto V$ is a linear map such that
- $S \circ T(E) = S(T(E)) = S(F) = E$
But we already have such a map that maps from V to V and sends E to E. It’s the identity map! $id_{v}$
by the uniqueness part of the above theorem and the last lecture, we get that this $S \circ T = id_{v}$, they are one and the same.
the same argument applies to $T \circ S$ on F, it equals $id_{w}$
Corollary: Let V,W be finite dimensional vector spaces over F. Then
- (V and W are isomorphic) iff (dim V = dim W) nice
- proof: => follows from isomorphism. look:
  - If $T: V \mapsto W$ is an isomorphism then for any basis B of V the vectors $T(B) \subset W$ form the basis of W (important point!)
  - We can check this directly:
    - if $w \in W$ then we can consider $v = T^{-1}(w) \in V$
    - by definition, $T(v) = T(T^{-1})(w) = w$.
    - but, $v \in (V = span(B))$
    - So by the linearity of T we have
      - $w = (T(v) \in span(T(B)))$ QUESTION: is spanT(B) = W? oh wait thats the conclusion we are trying to reach.
    - This shows that T(B) is a generating set of W
    - But if $B = \{ b_{1}…b_{n} \} $ and $ \{c_{1},..,c_{n}\} $ are scalars in the field such that the linear combination of these with the bases vectors is in W and equals 0, or $ c_{1}T(b_{1})+…+c_{n}T(b_{n}) = \vec{0} \in W$
    - then we can apply $T^{-1}$ to both sides and get:
    - $ T^{-1}(\vec{0}) = T^{-1}(\sum_{i=1}^{n} c_{i}T(b_{i})) $
    - = $ \sum_{i=1}^{n} c_{i}T^{-1}T(b_{i}) $ (linearity of T)
    - = $ \sum_{i=1}^{n}c_{i}b_{i} $
    - since B is a basis, this implies that $c_{1}=…=c_{n} = 0$ and hence that T(B) is a linearly independent set of W
    - Since it is both spanning and linearly in dependent,
    - T(B) is a basis of W and thus…
    - dim V = number of vectors in B = number of vectors in T(B) = dim W
  - (<=): WTS: dim V = dim W => V and W and isomorphic
    - supposed dim V = dim W = n (so there’s n vectors in the basis)
    - and let
      - $ E = {\ e_{1},..,e_{n} }\ \subset V$
      - $ F = {\ f_{1},..,f_{n} }\ \subset W$ be bases
    - Then by the previous claim(what is it!) the unique linear map $T: V ]mapsto W$ such that T(E) = F (sends one basis to another) is an isomorphism. nice.
notation: two isomorphic spaces over the same field:= $V \simeq W$
caution: the previous claim just says if dim V = dim W there exists an isomorphism between them. It is not the case that any linear map between the two spaces is an isomorphism.
- ex: $0 : v \mapsto W$ is a linear map between the two that is not an isomorphism. Not bijective when dim V is greater than 0.
- the size of the Kernel helps tell if a mapping is an iso.
Kernel (definition) : the subspace of the domain of a linear mapping $T : v \mapsto W$ between two vector spaces
- ker(T) = $ \{ v \in V s.t. T(v) = 0 \} $
Theorem ( equal dimensions and kernel has zero vector implies iso ):
- given a linear map T between V and W, F vector spaces, that have the same finite dimension,
- T is an iso iff <=> Ker(T) = {$ \vec{0} $}
- proof:
  - S1: ( => ) T is an isomorphism / bijective, so from the “onto” criteria we have
    - there exists (for sure) a vector v in V that maps to 0 in W. This is because W needs to have the 0 vector to be a vector space.
    - there exists $v \in V$ s.t. $T(v) = \vec{0} \in W$
  - S2: But T is linear, so $T(0) = T(0v) = 0 T(v) = 0$
  - S3: This shows that $\vec{0} \in V$ is the uniqe vector in V that is mapped to 0 in W by T.
    - This proves that $Ker(T) = 0$
    - -(QUESTION: does this mean that the solution set has only the trivial solutin in an isomorphism?) - the id matrix is an iso, and when set = 0 its only solution is 0, think about this, so it seems to be the case, at least for the id matrix
  - S1: (<=) “kernel only contains 0 implies T is an iso” (harder!)
    - suppose $Ker(T) = \{\vec{0}\}$
    - Let $\{ e_{1},…,e_{n} \}$ be a basis of V
    - S2: so the vectors $\{ f_{1} = T(e_{1} … f_{n} = T(e_{n})) \} \in W$ are linearly independent. (how do we know this?)
      - well, if we have scalars $a_{n} \in \mathbb{F}$ s.t. $a_{1}f_{1},…,a_{n}f_{n} = \vec{0} \in W$ then we get
      - $\vec{0} = \sum_{i=1}^{n}a_{i}f_{i}$
      - $= \sum_{i=1}^{n}a_{i}T(e_{i}) $
      - = $ \sum_{i=1}^{n}T(a_{i}e_{i})$ (linearity of T)
      - = $ T(\sum_{a=1}^{n}a_{i}e_{i}) $ = $\vec{0}$, (linearity of T)
    - S3: but since we have that $Ker(T) = \{\vec{0}\}$,
      - $Ker(T) = \{ \sum_{a=1}^{n}a_{i}e_{i} = \vec{0}\}$, and kernel is a subspace of the domain T, V, so $\sum_{a=1}^{n}a_{i}e_{i} \in V $
    - S4: since {e}’s were a basis of V, we have $a_{2} = …= a_{n} = 0$
    - S5: we have now shows that S2 is true.
      - This face, that the {f} are linearly independent + dim V = dim W = n (bases of V and W have the same number of vectors) shows that $f_{1},…,f_{n}$ is a basis of W.
    - S6: Since T sends the basis of V to the basis of W, it is an isomorphism.
    - QED
Image of T(definition): For a linear mapping $T: V \mapsto W$,
- im(T) = $ \{w \in W : \exists v \in V s.t. T(v) = w\} $
- EXERCISE: check that im(T) is always a subspace of W.
Theorem: V,W, F-vector spaces with equal finite dimension. dim V = dim W < $\infty$. Then a linear map $T: V \mapsto W$ is an iso <=> (iff) im(T) = W
QUESTION: is this any different than the def of iso? being onto => range(T) = W? oh i see this only works for the first direction.
- proof : ( => )If T is an isomorphism, then
  - T is bijective (in particular onto/surjective:). Thus image(T) = W.
- (<=) “if im(T) = W then T is an isomorphism.”
- Suppose image(T) = W,
- let $\{ f_{1},…,f_{n}\}$ is a basis of W.
- since im(T) = W we can find vectors $e_{1},..,e_{n} \in V$ such that $T(e_{i}= f_{i})$ , $i = 1,…,n$
- Then the vectors $\{e_{1},…,e_{n}\}$ are linerly independent in V. how so? well :
  - suppose we have scalars $a_{1},…,a_{n} \in K$
  - so that $ a_{1}e_{1} + .. + a_{n}e_{n} = \vec{0} $
  - then, $ T(\sum_{i=1}^{n} a_{i}e_{i}) = T(\vec{0})$
  - $ \sum_{i=1}^{n} a_{i}T(e_{i}) = T(\vec{0})$
  - $ \sum_{i=1}^{n} a_{i}f_{i} = T(\vec{0})$
  - since $ \{ f_{1},…,f_{n} \} $ is a basis in V, then we can conclude that the scalars $ a_{1}=…=a_{n} = 0$
  - thus, $ \{e_{1},…,e_{n}\} $ is linearly independent.
- Since we have dim V = dim W = n, this implies that $ \{ e_{1},…,e_{n} \} $ is a basis of V
- and because we have a linear mapping T that sends a basis to another it is an isomorphism.
- QED
The argument we used to prove the previous two theorems can be refined to show that Ker(T) and im(T) always have complementary dimensions.
Theorem: (Rank-Nullity?) Let V,W be F-vector spaces. $ T:V \mapsto W$ is a linear map. Dimension V < infinity (it’s finite). note that this only talks about the dimentison of the domain Then:
- (a): dim Ker(T), dim Im(T) finite.
- (b): dim Ker(T), dim Im(T) = dim V
proof: for part (a), note that $Ker(T) \subset V$ is a subspace of V and by the monotonicity of dimension of spaces we get that
- $ dim(Ker(T) \leq dim(V) \leq \infty)$ (both finite, dim ker is less)
Also if $ \{ e_{1},…,e_{n} \} $ is a basis of V
- then V = span$\{e_{i}\}$
so then the image of T, which is all of T applied to every vector in V, is the span of T applied to every basis vector, by linearity of T:
- $Im(T) = T(V) = span(T(e_{1}),…,T(e_{n})) $
from this fact it follows that $im(T)$ is spanned by finitely many vectors, and so the dim $im(T) \leq \infty$
For part (b) :
- choose a basis of the Ker(T), $\{ e_{1},…,e_{k} \}$.
- now choose a completion to a basis of V to a basis of V : $ \{ e_{1},…,e_{k},e_{k+1},…,e_{n} \} $
- QUESTION: get an explicit example of doing this!
- Then we get that $T(e_{1}),…,T(e_{n})$ span im(T)! QUESTION: why? from part a!
- But $T(e_{1}) = …= T(e_{k}) = 0$, from the basis of Ker(T)
- so we get that really, $T(e_{k+1}), …, T(e_{n})$ will span im(T)
- but the vectors $T(e_{k+1}), … , T(e_{n})$ will be linearly independent in $im(T) \subset W$ (how so)? :
  - well, if we have $a_{k+1},…,a_{n} \in \mathbb{F}$ s.t.
  - $\sum_{i=k+1}^{n} a_{i}T(e_{i}) = \vec{0}$
  - then $ T(\sum_{i = k+1}^{n}a_{i}e_{i}) = \vec{0}$ (by linearity of T)
  - and so $ \sum_{i = k+1}^{n}a_{i}e_{i} \in Ker(T) $
    - because its the argument in T(0) = 0, do you see it?
  - so $\sum_{i = k+1}^{n}a_{i}e_{i} = \sum _{j=1}^{k}b_{j}e_{j}$
  - since$\{e_{1},…,e_{n}\}$ is a basis of V it follows that
    - all coefficients must be equal to zero, $a_{k+1},…,a_{n} = 0$, (which implies linear independence?)
  - Thus $T(e_{k+1}),…,T(e_{n})$ is a basis of im(T).
  - This shows that dim im(T) = $n-k$ = dim V - dim ker(T).
  - QED

Dual Spaces and Isomorphism

We can use the criteria for isomorphic spaces to uncover more truths about the relationship between $V $ and $V^{v}$

Dimension of $V^{v}$ and $V$ Claim:
- if V is a finite dimensional vector space over F,
- then (=>) $dimV^{v} = dim V$
  - proof: Let $E = \{e_{1},…,e_{n}\} $ be a basis of V.
  - Then for all $i$ consider this unique linear operator / function:
    - ${e_{i}}^{v}: V \mapsto \mathbb{F}$ (from the vector space to the field) s.t.
    - ${e_{i}}^{v}{e_{j}}$ = $\begin{cases} 0 & j \neq 1 \\\ 1 & j = i \end{cases}$
  - we claim that the functions: $e_{1}^{v},…,e_{n}^{v} \in V^{v}$ form a basis of the dual space $V^{v}$ of $V$ how so you ask!
  - well, let $f \in V^{v}$ be any element in the dual space, so it is a linear functional that sends vectors in v to the field.
  - the function $f: V \mapsto \mathbb{F}$ is unqiely determined by its values on the spanning set E of V.
    - QUESTION: so the lemma works for spanning sets, not just bases?????
  - these values are: $ f(e_{1}),…,f(e_{n}) \in \mathbb{F} $. (recall they send vectors in v, in this case e, to a scalar in the field)
  - consider the linear function:
    - g = $ f(e_{1})e_{1}^{v},…,f(e_{n})e_{n}^{v}$
    - $g(e_{j}) = f(e_{1})e_{1}^{v}(e_{j}) + … + f(e_{n})e_{n}^{v}(e_{j})$ = $f(e_{j})$, because it becomes $ f(1*e_{j})$ for all j.
    - Thus, $f : V \mapsto F$ and $g : V \mapsto F$, take the same values on the basis $e_{1},…,e_{n}$
    - since linear transformations are characterized by what they do to a generating set, then $f = g$ (remeber E is a spanning set of V)
  - This shows that $V^{v} =$ span$ (e_{1}^{v}, …, e_{n}^{v}) $
  - Next, let scalars $c_{1},…,c_{n}$ are such that $c_{1}e_{1}^{v},…,c_{n}e_{n}^{v} = 0 \in V^{v}$
  - so this $c_{1}e_{1}^{v},…,c_{n}e_{n}^{v} : V \mapsto F$ is the zero function.
  - Evaluate the function on $ (e_{j})$, gives $ c_{1}e_{1}^{v}(e_{j}),…,c_{n}e_{n}^{v}(e_{j}) $
  - but value of zero mapping on any vector just gives you zero back, so
    - $ c_{1} = … = c_{n} =0$
  - Hence, $ \{ e_{1}^{v},…,e_{n}^{v} \}$ is linearly independent, thus they form a basis, since they are also spanning, of the dual space.
  - Hence, $dim V^{v} = n = dim V$
  - QED
Definition Dual Basis: (not complete lol)
- $E^{v}= e_{1}^{v},…,e_{n}^{v} $, as defined above, is the dual basis of E (of V)
note: dual spaces allow us (only when V is finite dimensional!) to define an isomorphism $T: V \mapsto V^{v}$.
- T is defined as the unique linear map sending the basis E of V to the dual basis $E^{v} of V^{v}$.
also works in opposite direction …
Dual Bases Claim: V is a finite dim vector space over F, let
- $ F = \{ f_{1},…,f_{n} \} $ be a basis of $V^{v}$
- Then there exists a unique basis $E = \{ e_{1},…,e_{n} \}$ of V s.t.$E^{v} = F$
- (a basis for the dual space can always constructed from a basis for the space, so we can find the “origin” basis for any basis in $V^{v}$)
- proof:
  - note that each vector $ x \in V $ defines a linear function
    - $ev_{x}: V^{v} \mapsto \mathbb{F}$ alt: $f \mapsto f(x)$ (the evaluation map! kirillov was talking about this!)
  - This gives a map:
    - $ev : V \mapsto V^{vv}$ alt: $ x \mapsto ev_{x} $
  - this map is linear:
    - if $x_{1},x_{2} \in V$, $a_{1}, a_{2} \in F$
    - then for every $ f \in V^{v}$ we have:
    - $ev_{a_{1}x_{1} + a_{2}x_{2}}(f) =V s\mapsto F f(a_{1}x_{1}+a_{2}x_{2}) = a_{1}f(x_{1}) + a_{2}f(x_{2}) = a_{1}ev_{x1}(f) + a_{2}ev_{x2}(f)$
  - By the previous claim on $dim V^{vv} = dim V^{v} = n$,
  - so $ev: V \mapsto V^{vv}$ is a linear map between vector spaces of the same dimension.
  - now a brief interruption for a lemma we need to prove to finish proving this claim:
- Evaluation Map is an Isomorphism Lemma:
  - For a finite dimensional vector space V over F the map $ ev: V \mapsto V^{vv} $ is an isomorphism. (a natural one!)
  - proof: Let $\{ b_{1},..,b_{n} \}$ be a basis of $V$.
    - let $\{b_{1}^{v},…,b_{n}^{v}\}$ be a basis $V^{v}$.
    - let $ \{ b_{1}^{vv},…,b_{n}^{vv} \} \subset V^{vv}$ be a basis dual to the dual basis.
    - Compute $ev_{b_{i}}(b_{j}^{v})$.
    - By definition, we have
    - $ ev_{b_{i}}(b_{j}^{v}) = b_{j}^{v}(b_{i}) = $ $\begin{cases} 0 & j \neq 1 \\\ 1 & j = i \end{cases}$
    - Thus, we can see that $ ev_{bi} $ and $ b^{vv} $ take on the same values on the dual basis $ \{ b_{1}^{v},…,b_{n}^{v} \} $
    - since every linear map,function is uniquely detemined by its values on a spanning set, it follows that:
      - $ ev_{bi} = b_{i}^{vv}$ for i =1,..,n
    - Conclusion: $ev$ sends a basis of V toa basis of $V^{vv}$ and so it is an isomorphism.
- back to the proof….
- Now let us start wuth the basis
  - $ F = \{ f_{1},…,f_{n}\} $ of $V^{v}$
- Let $F^{v} = \{ f_{1}^{v},…,f_{n}^{v} \}$ be the dual basis of $v^{vv}$
- Since we have $ev: V \mapsto V^{vv}$ is an isomorphism (proved above)
- for each $i = 1,…,n$ we have a unique vector $e_{i} \in V$ s.t.
  - $ev_{e_{i}} = f_{i}^{v}$ (because an iso sends a basis to a basis?)
- But then, $f_{i}(e_{j}) = ev_{ej}(f_{i}) = f_{j}^{v}(f_{i}) = \begin{cases} 0 & j \neq 1 \
  1 & j = i \end{cases} $
- Now consider the collection of vectors $ \{ e_{1},…,e_{n} \} \subset V$
  - It is a basis of V since it is the image of the basis of the double dual: $\{ f_{1}^{v},..,f_{n}^{v} \}$ of $V^{vv}$ under the isomorphism: $ev^{-1}: V^{vv} \mapsto V$
- The formula $ f_{i}(e_{j}) = \begin{cases} 0 & j \neq 1 \
  1 & j = i \end{cases} $ implies that ..
- $f_{1},..,f_{n}$ is the dual basis of $ e_{1},…,e_{n} $ and this proves our claim.
- QED
Remark: If $V$ is an n-dimensional space over F, we now know that
- $V \simeq V^{v}$
- $V \simeq V^{vv}$
However, the first one is a isomorphism that depends on the choice of the basis, the second is a canonical isomorphism.
To construct the first, we need to find a basis of V.
To construct the second, there is a canonical isomorphism $ev: V \mapsto V^{vv}$ which is deinfed only in terms of V and doesn’t depend on a basis.

** Exercises:

(Duality for Linear Maps): Let U, V, W be vector spaces over F
- (1) Show that if $T: V \mapsto W$ is a linear map, then the map $T^{v}: W^{v} \mapsto V^{v}$ defined by $T^{v}(f) = f \circ T$, for any linear function $f: W \mapsto F$ is also linear.
- (2) Show that if $S: U \mapsto V$, $T: V \mapsto W$ are linear maps, then $ (T \circ S)^{v}: S^{v} \circ T^{v}$
- (3) Suppose that V,W are finite dimensional and let
  - $ E = \{ e_{1},…,e_{n} \} \subset V $
  - $ F = \{ f_{1},…,f_{n} \} \subset W $
- be bases of V and W.
- Let $S \in Mat(mxn)(F)$ be the matrix of T in the bases $E \subset V$ and $ F \subset W $.
- Show that the matrix of $T^{v}: W^{v} \mapsto V^{v}$ in the dual bases $ F^{v} \subset W^{v} $ and $ E^{v} \subset V^{v} $ is the matrix $ A^{T} \in Mat(nxm)(F) $
- (4) Prove that if $A \in Mat(nxm)(F)$ and $B \in Mat(mxk)(F)$
  - then $(AB)^{T} = B^{T}A^{T}$.

Lectures 12-13

####Traces:
It is useful to have numerical invariants measuring the complexity of linear maps
we already have some discrete (= integer invariants)
- for every linear map $ T: V \mapsto W $
- we have two integers cpaturing information about T (transformation)
  - Definition: nullity of T:
    - dim Kernel(T) = dim Nullspace(T) = dim of the solution set to $Ax=0$
  - Definition: Nullspace(T):
    - set of all n-dimensional column vectors such that $Ax=0), the solution set of the homog linear system.
      - Theorem: The nullspace N(A) is a subspace of the vector space \(\mathbb{R^{n}}$
      - proof: WTS: N(A) is nonempty, closed under addition, closed under scalar multiplication:
      - S1: the trivial solution is always in N(A)- so it’s nonempty. $\vec{x}=\vec{0}$
      - S2: WTS: $ x,y \in N(A) => x+y \in N(A)$
        
        Well, $ Ax = 0, Ay = 0, A(x+y) = A(x) + A(y) = 0 + 0 = 0 $
      - S3: $c \in \mathbb{R}, x \in N(A) => cx \in N(A)$
        
        Well, $A(cx)=c*A(x) = c * 0 = 0$
      - QED
  - Definition: rank of T:
    - dim image(T) = …QUESTION: any other defs? yes, see lec 14 for longer discussion
turns out that for linear operators $T: V \mapsto V$ we also have defined invariants which are scalars of the field $\mathbb{F}$
- example of invariants:
- Definition: Trace:
  - $tr: L(V,V) \mapsto \mathbb{F}$ is :
  - the sum of elements on the main diagonal of a square matrix A
  - the sum of its complex eigenvalues
  - invariant with respect to change of basis
  - trace with this def applies to linear operators in general
  - is a linear mapping: $ tr(T + S) = tr(T) + tr(S)$ and $ tr(cT)= c * tr(T) $
  - notice inside L(V,V) (linear maps from V to V) we have a natural collection of linear operators, from each one we can get a scalar back.
    - how can we get this scalar?
    - given any pair (f,v) where
      - $v \in V$ is a vector
      - $f \in V^{v}$ is a linear functional in the dual space = the space of all linear functionals from V to the scalar field
    - we can construct a linear operator:
      - $s_{f,v}: V \mapsto V, x \mapsto f(x)v $
        
        QUESTION: doesnt this give me a vector back?
    - but given (f,v) we can also get a natural scalar:
      - $ f(v)\in \mathbb{F} $
    - with this in mind we can form and prove the existence statement:
    - Existence Lemma:
      - Suppose V is finite dimensional vector space over $\mathbb{F}$
      - Then there exists a unique linear function:
        
        $tr: L(V,V) \mapsto \mathbb{F}$
        
        such that for all $v \in V$ and $f \in V^{v}$
        
        $tr(s_{f,v}) = f(v)$
    - proof of lemma:
      - fundamental fact: every linear function (any linear transformation) is uniquely determined by what it does to a basis (by its values on a basis)
      - from this fact, it suffices to constrct a basis of all linear functions from V to V, $L(V,V)$ that consists of operators of the form $s_{f,v}$ for the chosen f’s and v’s
        
        more here
      - Let $ \mathbb{B} = { b_{1}…….b_{n} } \subset V $ be any basis of V
      - Let $ \mathbb{B}^{v} = {b_{1}^{v}…….b_{n}^{v} } \subset V $ be its dual basis
      - Then we can say that the collection of operators
        
        $ \mathbb{S} = { s_^{v},b_{1}}…….s_^{v},b_{n}} } $ is a basis of $L(V,V)$ the set of all linear functions from V to V
        
        basis = spanning + linearly independent.
        
        here, each ${b_{i}}^{v}$ is a linear functional from the dual basis, and each $b_{i}$ is a vector from the basis of V. Each gets plugged into the linear operator s and spits out a and spits out a $ {b_{i}}^{v} * b_{i} $, which is QUESTION: a vector in V?
        
        proof that $ \mathbb{S} $ is a basis for L(V,V):
        
        S1: $ T: V \mapsto V $ is a linear map.
        
        Let $ A \in Mat_{nxn}\mathbb{F} $ be the matrix of T in the basis $ \mathbb{B} $
        
        note: we can always represent a linear transformation/mapping by a matrix in its
        
        S2: Then $ T = \sum_{i,j=1}^{n} a_{ij} * s_(^{v}, b_{i}}) $
        
        for every $ k = 1,….,n $, we have
        
        $ T(b_{k}) = \sum_{i=1}^{n} a_{ik} * b_{i} $
        
        and we also have:
        
        $ (\sum_{ij} a_{ij}*s_^{v}b_{i}} )(b_{k}) $
        
        $ = {\sum_{ij}} a_{ij}*s_^{v}b_{i}} (b_{k}) $
        
        $ = \sum_{ij} a_{ij} {b_{j}}^{v}(b_{k})b_{i} $
        
        $ = \sum_{i=1}^{n} a_{ik} * b_{i} $
        
        S3: Thus, $ T = \sum_{ij}a_{ij}s_^{v}, b_{i}} $
        
        This representation is unique since the matrix of T in the basis $ \mathbb{B} $ is uniquely determined by T and $ \mathbb{B} $ *(“linear extension theorem” - a linear transformation is uniqely determined by what it does to a basis.) ?
        
        by the characterization of linear maps (the one descirbed above?) we then have a unique linear function, trace:
        
        $ tr: L(V,V) \mapsto \mathbb{F} $
        
        ENDED PG 7 REVISIT AFTER LEC 9-11

Lecture 14: Row Reduction

Outline:

Simplifying Linear Systems
Row Reduction and Echelon Forms
Solving Systems with Row Reduction
Corollaries
####Solving a Linear System
using row and column operations we can convert every linear system into a system in which all variables separate
- row operation:
- column operation:
$Ax=b$ where
- $A \in Math(mxn)(F)$ is a given coefficient matrix.
- $b \in F^{m}$ is a given vector of right hand sides.
- $x \in F^{n}$ is an unknown vector.
we proved (QUESTION: where b?) that we can find invertible matrices
- $ R \in Mat(mxm)(F) $ - number of eqs
- $ C \in Mat(nxn)(F) $ - number of unkowns
- such that after performing on A the row operation corresponding to R, and then the column operation corresponding to C we get the simplest possible matrix:
- $ \tilde{A} = RAC =$
  - where r = rank of matrix (see discussion on rank below!!)
- Using R and C we can simplify the system Ax=b.
- Left multiplying (Ax=b) by R gives an equivalient system
  - $RAx = Rb <=> RACC^{-1}=Rb$
    - so the id matrix is written in the form $CC^{-1}$ which doesnt change anything, but we have our simple matrix RAC to work with.
  - given the notation:
    - $\tilde{A} = RAC \in Mat(mxn)F$
    - $\tilde{b} = Rb \in (F^{m})$
    - $\tilde{x} = C^{-1}x \in F^{n} $ = column vector with n entries.
  - write system as $\tilde{A}\tilde{x}=\tilde{b}$, but since we used only invertible operations to get this new system, we can answer quesetions about the old system with this one.
  - in terms of the new variables $\tilde{x} = (\tilde{x}_{1},…,\tilde{x}_{n})^{T}$ = $C^{-1}x$
  - so it becomes : \[ \tilde{x}_{1} = \tilde{b}_{1} \
    … \
    \tilde{x}_{r} = \tilde{b}_{r} \
    0 = \tilde{b}_{r+1}\
    …\
    0 = \tilde{b}_{m} \]
  - note that it is m equations. we got this very simple system we alluded to using the block matrix previously.
  - So we conclude that we can get the solutions of Ax=b through $\tilde{A}\tilde{x}=\tilde{b}$
    - proof:
    - S1: The tilde system and Ax = b are both consistent iff $\tilde{b}_{r+1}= … = \tilde{b}_{m}=0$
    - S2: If $ \tilde{b}_{r+1}=…=\tilde{b}_{m}=0$ then the solutions of $ \tilde{A}\tilde{x}=\tilde{b} $ are vectors of the form: \[ \tilde{b}_{1} \
      … \
      \tilde{b}_{r} \
      \tilde{x}_{r+1} \
      … \
      \tilde{x}_{n} \]
    - with $ \tilde{x}_{r+1},…,\tilde{x}_{n} \in \mathbb{F} $ being free variables
    - S3: If $\tilde{b}_{r+1}= … =\tilde{b}_{m}=0$ then the solutions of $Ax=b$ are of the form: x = \[ \tilde{b}_{1} \
      … \
      \tilde{b}_{r} \
      \tilde{x}_{r+1} \
      … \
      \tilde{x}_{n} \]
    - with $ \tilde{x}_{r+1},…,\tilde{x}_{n} \in \mathbb{F} $ being free variables.\, and the whole column vector of solutions is multiplied by $C$. (why? because solutions $\tilde{x} were our solutions x multiplied by C^{-1}$ !)
  - but this method is not constructive, we had to pick bases in $F^{n}, F^{m}$ to fit with A. We had actually construct R and C by choosing these bases, adopted to map T, or matrix of map A. Row reduction algorithm solves the issue.
    - the algo simplifies the system systematically, uses only row ops, done in simple steps, and allows us to get close enough to solve the system.
    - also they solve the systm in finitely many steps!
####Elementary Row Operations:
- note that Ax=b is the matrix equation for our equation $F(v) = b$, once we choose a basis in the vector space V.
- If A is (mxn), the augmented matrix is (m x n+1).
- Elemtary Row Operations are just left multiplication by a specific elementary matrix. We called it R above.
  - they just replace rows in the matrix with linear combinations of rows, in an invertible way (you can always work backwards).
  - The important question is - why is it that these row operatiors don’t change the solution set?? Because they are all invertible, reversible.
  - the inverses of ERO’s are just ERO’s of the same type.
  - What are these special elementary matrices? Well, they are just obtained by doing the corresponding row operation to the identity matrix.
    - Row operation 1: switch two rows:
    - Row operation 2: multiply a row by a scalar multiple:
    - Row operation 2: add a scalar multiple of a row to another row:
  - another way of showing that a row operation does not change the solution set of a system, matrix equation style :) E is the elementary matrix representating a row operation.
    - $Ax=b$
    - $EAx = Eb$ (implies any solution of this equation is a solution of the previous, because we multiplied them all by the same matrix)
    - $E^{-1}EAx = E^{-1}Eb$
    - $Ax = b$, QED.
  - claim: the composition of row operations is also a row operation!
    - proof: if $p_{1},…,p_{s}: \mathbb{F}^{n} \mapsto \mathbb{F}^{n}$
####Rank of a Matrix :
- recall that left multiplication by an mxn matrix $A$ defines a linear map:
  - $T : \mathbb{F}^{n} \mapsto \mathbb{F}^{m}$ :: $v \mapsto Av$
- the rank of the matrix A is then the rank of T.
  - rank(T) = dimension of the subspace $ im(T) \subset \mathbb{F}^{m}$ of the codomain.
- rank (A) (def): rank (A) = rank(T) = dim im(T) = dim (column space of A) = maximal number of linearly independent vectors in col space A = max number of linearly independent columns of A.
- im(T) (def): im(T) = $y \in F^{m} : y = T(v) = Av$ for some $v \in F^{n}$ = span(columns of A)
  - (we show the last part now), that im(T) = span of columns of A.
  - now note that:
  - $F^{n}$, the domain of the linear map, which is a coordinate n space, is spanned by the vectors of the cardinal basis $e_{1},…,e_{n}$ and therefore $im(T)$ is spanned by the vectors $Ae_{1},…,Ae_{n} \in F^{m}$, which as we now discuss are just the columns of A
    - we know that image of a linear map is gneerated by the image of the generators of the source space (QUESTION: - when did we discuss this?)
    - recall that each $e_{i}$ has a 1 in the ith spot, 0 everywhere else, so by definition of matrix multiplication (row by column),
      - $Ae_{i}$ = i-th column of A.
  - Thus, im(T) = span(columns of A)
- To be continued

Chapter 1

Chapter 2

Section 2.3 : Analyzing the Pivots:

All questions about the existence of a solution + uniqueness of a solution of a system can be answered by : look at the pivots in RREF form of the augmented matrix of the system. (needs to be $A^{rref}$)!
when is $Ax=b$ inconsistent?
- when it doesn’t have a solution!
- (1) a system is inconsistent (<=>) (iff)
  - there is a pivot in the last row of the augmented matrix.
  - so it looks like ($0 0 0 0 : 1$)
    - this implies the last equation is $0x_{1} + … + 0x_{n} = b, b\neq 0$,
    - => no solution.
remarks on the coefficient matrix:
- (1): a solution, if it exists, is unique $\iff$ :
  - there are no free variables
  - the echelon form of the coefficient matrix has a pivot in each column,
  - (note that this says nothing about the existence, only uniqueness of solution.)
  - QUESTION: how does this connect to rank, and the fact that rank(A) = dim column space of A?
- (2): $Ax = b$ is consistent (has a solution) for all b $\iff$ :
  - the echelon form of the coefficient matrix has a pivot in each_row_
  - note: this property makes sense considering if there is a pivot in each row of coefficint matrix, you can’t have a pivot in the last column on the augmented matrix. => never inconsistent.
- (3): Ax = b has a unique solution (and it exists..) for all b iff $\iff$:
  - echelon form of the coefficient matrix $A$ has a pivot in each row and each column.
remarks on inconsistency (very cool)
- WTS: To be continued

Section 2.7 : Fundamental Subspaces and Rank:

four fundamental subspaces:
- Ker(A), Ran(A), row space(A), Left nullspace(A)
- need to study relationships between their dimensions.
any linear transformation has subspaces : Ker(A), Range(A)
- Ker(A), subset of domain $V$ :
  - Ker(A) = Nullspace(A) := $\{ v \in V : Av = 0 \} \subset V$
  - solution set of the homogenous equation Ax = 0
  - Ker($A^{T}$) = left null space.
- Ran(A), subset of codoamin $W$ :
  - Range(A) := $\{w \in W : w = Av, v \in V\}$
  - set of all right sides $b \in W$ for which $Ax = b$ has a solution, “is consistent.”
  - Range(A) = column space(A) (a basis is pivot columns’ originals)
  - Range($A^{T}$) = row space (a basis is pivot rows)
- rank (A) = dim Range(A) = dim Colspace(A)
Computing fundamental subspaces:
- S1: reduce $A$ to $A_{e}$ (echelon form of A).
- S2: original columns of $A$ which becomes pivot columns of $A_{e}$ are a basis in Range(A).
- S3: pivot rows of $A_{e}$ are a basis for rowspace(A)
  - you can also transpose the row matrix and do row operations, but easier this way.
- S4: basis for Ker(A) = nullspace(A) comes from solving homogenous $Ax = 0$.
  - Consider $A$ and $A_{e}$:
  - A = $ \begin{bmatrix} 1 & 1 & 2 & 2 & 1\
    2 & 2 & 1 & 1 & 1 \
    3 & 3 & 3 & 3 & 2 \
    1 & 1 & -1 & -1 & 0 \end{bmatrix}$, $A_{e}$ = $ \begin{bmatrix} 1 & 1 & 2 & 2 & 1 \
    0 & 0 & -3 & -3 & -1 \
    0 & 0 & 0 & 0 & 0 \
    0 & 0 & 0 & 0 & 0 \end{bmatrix}$
  - since columns $1,3$ becomes pivot columns in $A_{e}$, then the columns 1 and 3 of $A$ are a basis for colspace(A) = Ran(A). (notice that the dimension of the colspace/ran(A) is 2.)
  - since rows $1,2$ are pivot rows of $A_{e}$, they form a basis for rowspace(A), in their REF. (notice dim rowspace(A) = 2).
  - now, solving in REF, we get the solution set of A, which can be written in vector form \[ \begin{bmatrix} -t-(1/3)r \
    t \
    -s-(1/3)r \
    s \
    r \end{bmatrix} \]
  - or as $\begin{bmatrix} -1 \\ 1 \\ 0 \\ 0 \\ 0 \end{bmatrix}$ t + $\begin{bmatrix} 0 \\ 0 \\ -1 \\ 1 \\ 0 \end{bmatrix}$s + $\begin{bmatrix} -(1/3) \\ 0 \\ (1/3) \\ 0 \\ 1 \end{bmatrix}$r.
    - these are then the three basis vectors, and the dimension of the kernel is 3.
    - note that there is no shortcut for computing Ker($A^{T}$), you need to solve $A^{T}x = 0$ by hand.
Explanation for computing bases of fundamnetal subspaces: why do these methods work ?

Chapter 3

Chapter 4

Spectral Theory
####notes for hw 10 - on spectral theory
1. Eigenvectors are those vectors parallel to vector x,
- just means that Ax is some multiple of x , Ax = $\lamba$x
- lambda can be anything, negative, or even zero, just as long as it makes lamba x parallel to x.
- eigenvectors are the vectors that dont get knowcked off their span in the space after the transformation is applied - their egeinvalues is how much they are stretcehd by the transformation (matrix multiplication)
  - so Av, where v is eigenvector = lambda v, just says multiplying your special eigencector by a matrix A is the same effect as scalar multiplication on that matrix, it’s just scaled
  - scaling by $\lambda$ is the same as having an Id matrix with only $\lambda$ down the diagonal, this is lambda times the Id matrix times the eigenvector
  - important for finding shortcuts to rotation matrices
  - another way to say this is that the vector x is in the ker(A-lambda I)
    - now youve sset it up as a homogenous linear system to solve for the kernel, you only have two options: infintely many sols or the trivial sol, you’re not interested in the trivial sol because you want to form an eigenbasis, so interested in when it has infinitely many sols
    - RECALL: unique solution exists IFF full rank IFF M is invertible IFF det M nonzero
      - so, there are infinitely many solutions IFF det M is zero
      - thats why we solve for zero when taking eigenvalues, M = A-labdaI
    - this determinant of M is the characteristic polynomial, its roots are eigenvalues of A
    - to find the eigenbectors, plug into ker M. why?
  - the nullspace M + 0 vector = eigenspace
  - the set of all eigenvalies of an operator or matrix A is called the spectrum of A, and denoted o(A)
  - how to find the eigenvalues for an abstract operator A?
    - well just take an arbritary basis and compute eigenbalies of the abstract operator in the matrix of the operator wrt the basis,
    - this works because it doesnt depend on choice of basis - see similar matrices explanation : characteristic polynomails of similar matrices coincide
  - algebraic multiplicity of a root lambda - # of times it shows up in sol
  - geometric multiplicity of the eigenvalue lambda = dimension of the eigenspace Ker(A-LI)
    - dimension of eigenspace / geo multiplicity of eigenvalue cannot exceed its algebraic multiplicity
  - trace and det (new interpretations: , L= $\lamda$
    - trace(A): L1 + L2 + … + Ln
    - det(A): L1L2…Ln
  - shortcut: eigenvalues of traingular matrices:
    - eigenvalues of a triangular matrix are exactly the diagonal entries of the original matrix A
    - => so entries of a diagonal matrix are its diagonal entries
  - similar matrices have the same eigenvalues, different vectors
  - of course, if matrices have the same eigenvals and same eigenv’s theyre the same matrix
  - proof: Similar matrix def: A, B similar if $B = M^{-1}AM$
    - $Bx = M^{-1}AMx = \lambda x = MM^{-1}AMx = M\lambda x = A(Mx) = \lambda (Mx) $
      - notice they have the same eigenvalues but different eigenvectors! $i.e: Mx vs x$
  - smilar matrices represent same linear transformation in different bases.
  - a matrix relating two similar matrices (two matrices with same eigenval
  - is the eigenvector matrix
  - $A^{T}$ is similar to $A$ ** Section 4.2:
- task of diagonalization: find a basis in which the matrix is diagonal
- not all matrices/operators are diagonalizable
  - we only care because if matrices/operators are diagonalizable then their powers or functions on them are easy to compute. applications explained here: diagonalizable applications
  - necessary and sufficient condition for diagonalization:

Chapter 9

Cayley Hamilton Theorem: (how to use it)
- plugging the original matrix A into the characteristic polynomial (def of A-lambdaxI determinant) gives you the zero matrix
- every matrix “solves” / is a root of its characteristic polynomial. cool.

Chapter 8: Dual Spaces and Tensors

all spaces here are finite dimensional
might wanna refer to Dual Spaces and Isomorphisms

Section 8.1 : Dual Spaces

Definition: Linear Functional :
- a special type of linear transformation on a vector space $V$ that sends a vector from the vector space to a scalar in the field, $L: V \mapsto \mathbb{F}$
Examples?
- a velocity vector in a given direction (physical object) getting mapped to its magnitude (a scalar) - a linear measurement
- a force vector getting mapped to its magnitude

Extra notes/defs to categorize later

Dual Spaces and Dual Basis

The dual space of V is the set of all linear functionals from V to \(\mathbb{F}\( , so : \(V^{v} = T: V \mapsto \mathbb{F} \(
- all such elements of dual space are linear functionals
if dim(V) < $ \infty$ => $ V $ and $ V^{v} $ are isomorphic
- to show this is true, show that they have the same dimension
- another way to show the isomorphism is to use the dual basis
  - linear extension theorem: says if you know what T does on basis vectos, you know what T does on every vector:
    - Let \(\mathbb{B} = { v_{1}….v_{n} }\(
    - enough to know what \(f(v_{1}),…..,f(v_{n})\( is
    - usually hard to graphically represent linear transformations, but since the codoamin of all such linear functions are scalars, \(f: V \mapsto \mathbb{F}\(, you can draw a graph of your linear functionals (where your inputs end up.)

Isomorphism

mapppings that are injective and surjective (1:1 and onto)
watch this
How to check if a mapping is an isomorphism between two vector spaces:
- a linear transformation $T: V \mapsto W$ is one-to-one if:
  - $T$ maps distinct vectors in V to distinct vectors in W
  - check if two distinct vectors in the domain give you two different vectors in the codomain when you apply the linear transformation to it.
- it is onto if:
  - range(T) = W
  - so take any vector in W and you can find some vector in V that maps to it

Determinants

For an n×n matrix, each of the following is equivalent to the condition of the matrix having determinant 0

The columns of the matrix are dependent vectors in Rn
The rows of the matrix are dependent vectors in Rn
The matrix is not invertible.
The volume of the parallelepiped determined by the column vectors of the matrix is 0
The volume of the parallelepiped determined by the row vectors of the matrix is 0
The system of homogenous linear equations represented by the matrix has a non-trivial solution.
The determinant of the linear transformation determined by the matrix is 0

The free coefficient in the characteristic polynomial of the matrix is 0 Depending on the definition of the determinant you saw, proving each equivalence can be more or less hard.

exercises

prove tr$(AB)^{-1}$ = $B^{-1}A^{-1}$
prove inverse matrix is unique
1. here
theorems in this vid
exercises in this vid
exercises in this vid
vid
vid
intersection of subspaces is a subspace, union?
[more info] (https://math.stackexchange.com/questions/1722921/injective-or-surjective-and-same-dimension-implies-vector-space-isomorphism)
[do this] (https://math.stackexchange.com/questions/2521291/proof-that-f-is-an-isomorphism-if-and-only-if-f-carries-a-basis-to-a-basis)
look at this!
exericse of double dual
eigenvales and eigenvectors
similar matrices
eigenspaces
interesting