This is perhaps one of the more fundamentally important topics in modern advanced Mathematics. It is not uncommon for most graduate level Maths courses to either begin with or contain the fundamentals of Linear Algebra, either as a refresher, or as a brief introduction. Whether you study Analysis, Computational Mathematics, Discrete Mathematics, Geometry; Linear Algebra will always be needed.
The reason for this is that Linear Algebra deals with vectors and matrices. These are precisely the Mathematical constructs which we use to understand concepts of size and direction, as well as those that encode any concept we have of multiple measurements in a given scenario. For instance, we may need to consider a problem where a number of goods need to be shipped from various factories to various stores. These shipments come with specific costs and our goal may be to minimise these costs, whilst providing each store with all the goods they will need. This type of question is usually given in a form known as a Linear Problem. Certain values need to be selected (amount of each good going from each factory to each store), so that the costs are minimised and a certain calculation (generally a matrix multiplied by the vector of these values) meets certain conditions. In this post, I will try to explain the concepts of vectors and vector spaces. In a subsequent post, I hope to explore the basic ideas of matrices, but a good understanding of vectors will be required first.
In simple terms, a vector is nothing more than a collection of values, in a given order. Suppose you have a box of toy cars. You may count the number of cars it contains, of each different colour. You can then code your findings as a vector, by saying that you will give the number of red, green, blue and yellow cars in the box, in that order. If there are two cars of each colour, your box can be represented by the vector (2, 2, 2, 2). If you then buy 2 new green cars and one new red car, but lose on of your blue cars, the new vector representing the current state of affairs will be (3, 4, 1, 2). Note that the order of the values (or variables) must remain the same throughout. We could have selected a different order to begin with, but once this choice is made, we must keep it consistent.
Once we have the above concept, we can consider all vectors whose entries represent the same things. For instance, all collections of toy cars of these four colours can be thought of as a single family of objects. Such a family of vectors is called a vector space (or linear space), when certain properties are met. The first of these is known as closure in addition. That is, when we add two vectors in our vector space, we need to be able to get a new vector, which will also be in the same space. The second is closure to scalar multiplication. That is, if we take a number and multiply it with one of our vectors, we need to get a new vector, in the same space.
First, we will deal with vector addition. Suppose you buy one blue toy car and two red ones. Then, some time later, you may decide to buy another blue car and three green ones. The number of cars of each colour that you have does not depend on the order you acquired them in. It only depends on the sum of the individual purchases. Using the notation above, we have (2, 0, 1, 0) + (0, 3, 1, 0) = (2, 3, 2, 0). That is, the addition of vectors is done separately for each component. This establishes the first property of closure that we needed.
Next, we would like to see how scalars (i.e. numbers) multiply vectors. What if a standard box has two toy cars of each colour? How many cars would I have if I bought three such boxes? The calculation would be 3 * (2, 2, 2, 2) = (6, 6, 6, 6). That is, the scalar (in this case 3) multiplies each of the components of the vector, individually. This gives us the second property and ensures that our family of vectors is, in fact, a vector space
The final property we will consider is that of size. We know how to compare numbers to each other (for instance, 1 is less than 2). But we cannot compare vectors in this manner. We need to create some notion of a vector's "bigness"; if this "bigness" is represented by a number, we will be able to compare vectors in terms of this "bigness". We call these ideas of size, which allow us to compare vectors, norms. As it turns out, there are many different norms for vectors, and some will make more sense in some situations than others. For example, in our toy car scenario, we can say that a box of cars will be assigned a size, (or norm, symbolised with || . ||), based on the number of cars it contains. That is, the vector (a, b, c, d) will have norm ||(a, b, c, d)|| = a + b + c + d. Alternatively, your nephew may like green cars a lot, but not like yellow cars as much. Then, perhaps, the norm of the same vector could be ||(a, b, c, d)|| = a + 2b + c + d/2. In geometry, one of the most commonly used and most useful sizes of vectors is the square root of the sum of the squares of the components ||(a, b, c, d)|| = √(a^2 + b^2 + c^2 +d^2). (Think of what this would be if there were only two components. Does it remind you of something similar you may have seen before, in geometry or trigonometry?) Once we have selected a norm, we say that the set of all our vectors, with this norm, forms a normed vector space, or simply normed space.
The types of functions which can be vector norms are beyond the scope of this blog, but, in brief, the properties that make a function be a norm are based on how it behaves on the zero vector (a vector in which every component is zero), how it deals with scalar multiples of vectors and with sums of vectors.
