A Simple Understanding of Linear Independence and Linear Correlation

Abstract

There are many important concepts in linear algebra, such as linear correlation and linear independence, eigenvalues and eigenvectors, and so on. Among them, linear correlation and linear independence have irreplaceable importance, and have important applications in fields such as algebra, signal processing, and artificial intelligence. This article provides a graphical explanation of how to distinguish between the concepts of linear correlation and linear independence, the method provided in the paper is easy to grasp. The conclusion points out that linear independence means that there are no two (base) vectors with the same direction in a vector graph; otherwise, it is a linear correlation.

Share and Cite:

Wu, W. and Yuan, X. (2023) A Simple Understanding of Linear Independence and Linear Correlation. Creative Education, 14, 2333-2337. doi: 10.4236/ce.2023.1411147.

1. Introduction

Linear algebra is a compulsory course for engineering majors in universities. There are many important concepts in linear algebra. For example, linear correlation and linear independence, solution space of equations, eigenvalues and eigenvectors, similarity transformation of matrices, congruent transformation, orthogonal transformation; diagonalization of matrices, and so on.

For beginners, it is difficult to truly understand the differences between the concepts of linear correlation and linear independence without specific methods. The intuitive form provided by graphics usually helps learners better understand the differences between these two concepts. So the way to learn linear algebra well is to accurately grasp the connections and differences between the above concepts through graphics.

This article mainly introduces how to distinguish between the concepts of linear correlation and linear independence (Buffa, Cho, & Sangalli, 2010; Floater & Quak, 2000; Farnoosh & Haibe-Kains, 2021; Veiga et al., 2013; Bownik & Speegle, 2013; Magalhes, 2021; Ashur, Khan, & Nyberg, 2022; Esmi et al., 2023) .

Linear independence refers to the fact that no quantity in a set of data can be represented by other quantities, corresponding to linear correlation. In linear algebra, if there is no vector in a set of elements of a vector space that can be represented by a finite linear combination of other vectors, it is called linear independence. On the contrary, it is called linear correlation.

Let’s start with the simplest two-dimensional plane, as shown in Figure 1.

The decomposition of forces in a two-dimensional plane is a simple operation, where force F can be decomposed along the X and Y axes.

The same is true in three-dimensional space (in Figure 2), where point p in the figure above can be represented as OP = 4x + 5y + 3z.

Expand this concept to n-dimensional space and imagine each coordinate axis as a vector, resulting in Figure 3.

So, point x (actually an n-dimensional vector) in Figure 3 can be represented as:

X = k1x1 + k2x2 + … + Knxn, where k1, k2, Kn is the coordinate value of the coordinate axis. This expression is called a linear space, which means decomposing any vector x in an n-dimensional space to obtain the corresponding coordinate values.

So, what do the so-called linear independence and linear correlation mean?

{ a b } = a { 1 0 } + b { 0 1 }

From Figure 4, it can be seen that any point in the two-dimensional plane can be decomposed along the X (1, 0) and Y (0, 1) axes, or by the following two column vectors:

{ a b } = a 1 { 1 3 } + a 2 { 2 4 }

But if you replace the X and Y axes with the X (1, 0) and Y (2, 0) axes:

{ a b } = a { 1 0 } + b { 2 0 }

Figure 1. Two-dimensional plane.

Figure 2. Three-dimensional space.

Figure 3. Vector graph.

Figure 4. Vector in a two-dimensional space.

We see that when b is not equal to 0, the above figure is solveless, which means that the vector formed by point P (2, 3) in Figure 2 cannot be decomposed along (1, 0) and (2, 0). In fact, (1, 0) and (2, 0) are the same vector, both on the X-axis. In this case, the vector OP in Figure 2 cannot be decomposed solely along the X-axis because its vertical component cannot be obtained.

Linear independence refers to: k1x1 + k2x2, the equation system with knxn = 0 only has 0 solutions, which is k1, k2, when kn must be equal to 0, this equation will be equal to 0. At this point, we call vectors x1, x2, x3 Xn is linearly independent.

Referring to Figure 1, the so-called linear independence actually means that there are no vectors with the same direction in the n vectors in Figure 1. If there are, then these n vectors are linearly related.

For example, assuming x1 and x2 are two vectors (1, 0) and (2, 0), the equation system can be obtained from k1x1 + k2x2 = 0:

k1 + 2k2 = 0

0k1 + 0k2 = 0

The above equation system has non-zero solutions, so the vectors (1, 0) and (2, 0) are linearly correlated.

Assuming vectors x1, x2, x3. If the xn vector forms matrix A, then for the equation system Ax = 0, it is obvious that when the determinant of A is not equal to 0, there is only 0 solution, which means that x1, x2, x3. The n vectors xn are linearly independent; if the determinant of A is equal to 0, then it is linearly correlated.

For the equation system Ax = b, when the determinant of A is not equal to 0, the equation system has a unique solution, which is the vector decomposition x = k1x1 + k2x2 + … + Knxn will obtain a set of determined k1, k2, the kn value is the coordinate value of the vector x. And if the determinant of A is not equal to 0, it means that the vectors x1, x2, x3 Xn is linearly independent (Zhao, 2021; Guo, Li, & Yang, 2023; Ma et al., 2022; Aparkin, 2021) .

We know that if a determinant is equal to 0, it means that there are equal or proportional rows or columns in the determinant, and if two rows or columns are proportional, it precisely indicates that these two row or column vectors are vectors with the same direction, that is, the same vector.

2. Conclusion

Linear independence means that there are no two (basis) vectors with the same direction in Figure 1; otherwise, it is a linear correlation.

This article explains the concepts of linear correlation and linear independence through graphical methods. The intuitive nature of graphics makes abstract mathematical concepts more concrete, thereby reducing learning difficulty and enabling learners to gain a deeper understanding while learning these concepts, as well as enhancing memory and understanding.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1] Aparkin, A. M. P. V. A. (2021) Linear Correlation between Kovats Retention Indices I and the Sum of C-13 Nuclear Magnetic Resonance Chemical Shifts in the Structural Isomers of Saturated Hydrocarbons. Russian Journal of Physical Chemistry A, 95, 101-105.
https://doi.org/10.1134/S0036024421010027
[2] Ashur, T., Khan, M., & Nyberg, K. (2022). Structural and Statistical Analysis of Multidimensional Linear Approximations of Random Functions and Permutations. IEEE Transactions on Information Theory, 68, 1296-1315.
https://doi.org/10.1109/TIT.2021.3128618
[3] Bownik, M., & Speegle, D. (2013) Linear Independence of Time-Frequency Translates of Functions with Faster than Exponential Decay. Bulletin of the London Mathematical Society, 45, 554-566.
https://doi.org/10.1112/blms/bds119
[4] Buffa, A., Cho, R., & Sangalli, R. (2010) Linear Independence of the T-Spline Blending Functions Associated with Some Particular T-Meshes. Computer Methods in Applied Mechanics & Engineering, 199, 1437-1445.
https://doi.org/10.1016/j.cma.2009.12.004
[5] Esmi, E., Silva, J., Allahviranloo, T. et al. (2023) Some Connections between the Generalized Hukuhara Derivative and the Fuzzy Derivative Based on Strong Linear Independence. Information Sciences, 643, Article 119249.
https://doi.org/10.1016/j.ins.2023.119249
[6] Farnoosh, A.-A., & Haibe-Kains, B. (2021) Abstract 1356: Meta-Analysis and Lack of Independence Assumption: Application in Biomarker Discovery. AACR Annual Meeting 2021, Philadelphia, 10-15 April - 17-21 May 2021.
[7] Floater, M. S., & Quak, E. G. (2000) Linear Independence and Stability of Piecewise Linear Prewavelets on Arbitrary Triangulations. SIAM Journal on Numerical Analysis, 38.
https://doi.org/10.1137/S0036142998342628
[8] Guo, L., Li, G.X., & Yang, X. (2023) Global Convergence of Augmented Lagrangian Method Applied to Mathematical Program with Switching Constraints. Journal of Industrial and Management Optimization, 19, 3868-3882.
https://doi.org/10.3934/jimo.2022114
[9] Ma, L., Xu, F., Zhang, L. et al. (2022) Breaking the Linear Correlations for Enhanced Electrochemical Nitrogen Reduction by Carbon-Encapsulated Mixed-Valence Fe7(PO4)6. Journal of Energy Chemistry, 71, 182-187.
https://doi.org/10.1016/j.jechem.2022.03.042
[10] Magalhes, L. T. (2021) Simple Proof of Existence of a Complex Eigenvalue of a Complex Square Matrix …and yet another Proof of the Fundamental Theorem of Algebra with Linear Algebra. Linear and Multilinear Algebra, 70, 5329-5333.
https://doi.org/10.1080/03081087.2021.1913981
[11] Veiga, L.B.D., Buffa, A., Sangalli, G. et al. (2013) Analysis-Suitable T-Splines of Arbitrary Degree: Definition, Linear Independence and Approximation Properties. Mathematical Models & Methods in Applied Sciences, 23, 1979-2003.
https://doi.org/10.1142/S0218202513500231
[12] Zhao, G. (2021) Linear Independence of T-Spline Blending Functions of Degree One for Isogeometric Analysis. Mathematics, 9, 1346.
https://doi.org/10.3390/math9121346

Copyright © 2024 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.