What is Tensor?

H. Kemal Ilter

In simplifying terms, we can think of tensors as multidimensional arrays of numbers, as a generalization of scalars, vectors, and matrices. [1]

  1. Rank 0 Tensor, Scalar, [math]\mathbb{R} [/math]
  2. Rank 1 Tensor, Vector, [math]\mathbb{R}^n [/math]
  3. Rank 2 Tensor, Matrix, [math]\mathbb{R}^n \times \mathbb{R}^m[/math]
  4. Rank 3 Tensor, Tensor, [math]\mathbb{R}^n \times \mathbb{R}^m \times \mathbb{R}^p[/math]

When we describe tensors, we refer to its “dimensions” as the rank (or order) of a tensor, which is not to be confused with the dimensions of a matrix. For instance, an m × n matrix, where m is the number of rows and n is the number of columns, would be a special case of a rank-2 tensor. A visual explanation of tensors and their ranks is given is the figure below.



  1. Introduction to S. Raschka. 2017. Artificial Neural Networks and Deep Learning, A Practical Guide with Applications in Python. p.7. http://leanpub.com/ann-and-deeplearning