NumPy 矩阵的弃用状态 class
Deprecation status of the NumPy matrix class
matrix class 在 NumPy 中的状态如何？
我一直被告知我应该改用 ndarray class。在我写的新代码中 worth/safe 使用 matrix class 吗？我不明白为什么我应该改用 ndarrays。
tl; dr: numpy.matrix class 即将弃用。有一些高知名度的库依赖 class 作为依赖项（最大的是 scipy.sparse），这阻碍了 class 的适当短期弃用，但强烈鼓励用户改为使用 ndarray class（通常使用 numpy.array 便捷函数创建）。随着用于矩阵乘法的 @ 运算符的引入，矩阵的许多相对优势已被消除。
为什么（不是）矩阵 class？
numpy.matrix 是 numpy.ndarray 的子class。它最初是为了方便地用于涉及线性代数的计算，但与更一般的数组 class 的实例相比，它们的行为方式存在局限性和令人惊讶的差异。行为根本差异的示例：
- 形状：数组可以具有从 0 到无穷大（或 32）的任意维数。矩阵总是二维的。奇怪的是，虽然不能 创建 具有更多维度的矩阵，但可以将单个维度注入矩阵以在技术上最终得到多维矩阵：
np.matrix(np.random.rand(2,3))[None,...,None].shape == (1,2,3,1)（不是这具有任何实际意义）。
- 索引：索引数组可以为您提供任意大小的数组，具体取决于 how you are indexing it。矩阵上的索引表达式将始终为您提供一个矩阵。这意味着 2d 数组的
arr[:,0] 和 arr[0,:] 都给你一个 1d ndarray，而 mat[:,0] 的形状为 (N,1) 和 mat[0,:] 的形状(1,M) 如果是 matrix.
- 算术运算：过去使用矩阵的主要原因是对矩阵的算术运算（特别是乘法和幂）执行矩阵运算（矩阵乘法和矩阵幂）。数组的相同结果是逐元素乘法和幂。因此
mat1 * mat2 在 mat1.shape[1] == mat2.shape[0] 时有效，但 arr1 * arr2 在 arr1.shape == arr2.shape 时有效（当然结果意味着完全不同的东西）。此外，令人惊讶的是，mat1 / mat2 执行两个矩阵的 elementwise 除法。这种行为可能继承自 ndarray 但对矩阵没有意义，特别是考虑到 *. 的含义
- 特殊属性：矩阵除了数组还有a few handy attributes：
mat.A和mat.A1是与np.array(mat)和[=38具有相同值的数组视图=]，分别。 mat.T 和 mat.H 是矩阵的转置和共轭转置（伴随）； arr.T 是 ndarray class 中唯一存在的此类属性。最后，mat.I是mat的逆矩阵。
编写适用于 ndarray 或矩阵的代码非常简单。但是当两个 class 有机会必须在代码中交互时，事情就开始变得困难了。特别是，很多代码 可以 自然地为 ndarray 的子 class 工作，但是 matrix 是一个行为不端的子 [=184] =] 可以很容易地破坏试图依赖 duck typing 的代码。考虑以下使用形状为 (3,4):
的数组和矩阵的示例
import numpy as np
shape = (3, 4)
arr = np.arange(np.prod(shape)).reshape(shape) # ndarray
mat = np.matrix(arr) # same data in a matrix
print((arr + mat).shape) # (3, 4), makes sense
print((arr[0,:] + mat[0,:]).shape) # (1, 4), makes sense
print((arr[:,0] + mat[:,0]).shape) # (3, 3), surprising
添加两个对象的切片是灾难性的不同，具体取决于我们切片的维度。当形状相同时，矩阵和数组的加法都是按元素进行的。上面的前两种情况很直观：我们添加两个数组（矩阵），然后从每个数组中添加两行。最后一个案例真的很令人惊讶：我们可能打算添加两列并最终得到一个矩阵。原因当然是 arr[:,0] 的形状 (3,) 与形状 (1,3) 兼容，但 mat[:.0] 的形状 (3,1)。两人broadcast一起塑造(3,3).
最后，矩阵class的最大优势（即可以简明地表达涉及大量矩阵乘积的复杂矩阵表达式）在the @ matmul operator was introduced in python 3.5, first implemented in numpy 1.10时被移除。比较简单二次型的计算：
v = np.random.rand(3); v_row = np.matrix(v)
arr = np.random.rand(3,3); mat = np.matrix(arr)
print(v.dot(arr.dot(v))) # pre-matmul style
# 0.713447037658556, yours will vary
print(v_row * mat * v_row.T) # pre-matmul matrix style
# [[0.71344704]]
print(v @ arr @ v) # matmul style
# 0.713447037658556
从上面可以清楚地看出为什么矩阵 class 在处理线性代数时受到广泛青睐：中缀 * 运算符使表达式更简洁，更易于阅读。但是，我们使用现代 python 和 numpy 的 @ 运算符获得相同的可读性。此外，请注意，矩阵情况给了我们一个形状为 (1,1) 的矩阵，从技术上讲，它应该是一个标量。这也意味着我们不能将列向量与这个“标量”相乘：上例中的 (v_row * mat * v_row.T) * v_row.T 会引发错误，因为形状为 (1,1) 和 (3,1) 的矩阵不能相乘按照这个顺序。
为了完整起见，应该注意虽然 matmul 运算符修复了 ndarray 与矩阵相比次优的最常见情况，但在使用 ndarray 优雅地处理线性代数方面仍然存在一些缺点（尽管人们仍然倾向于相信总的来说最好坚持后者）。一个这样的例子是矩阵幂：mat ** 3 是矩阵的适当的三次矩阵幂（而它是 ndarray 的元素立方体）。不幸的是 numpy.linalg.matrix_power 相当冗长。此外，就地矩阵乘法仅适用于矩阵 class。相比之下，虽然 PEP 465 and the python grammar 都允许 @= 作为 matmul 的增强赋值，但从 numpy 1.15 开始，这并没有为 ndarrays 实现。
弃用历史
考虑到上述有关 matrix class 的复杂情况，长期以来一直在反复讨论其可能的弃用问题。 @ 中缀运算符的引入是这个过程 happened in September 2015. Unfortunately the advantages of the matrix class in earlier days meant that its use spread wide. There are libraries that depend on the matrix class (one of the most important dependent is scipy.sparse 的一个巨大先决条件，它同时使用 numpy.matrix 语义并且在增密时经常使用 returns 矩阵），因此完全弃用它们有总是有问题。
我已经在 a numpy mailing list thread from 2009 中找到
等评论
numpy was designed for general purpose computational needs, not any one
branch of math. nd-arrays are very useful for lots of things. In
contrast, Matlab, for instance, was originally designed to be an easy
front-end to linear algebra package. Personally, when I used Matlab, I
found that very awkward -- I was usually writing 100s of lines of code
that had nothing to do with linear algebra, for every few lines that
actually did matrix math. So I much prefer numpy's way -- the linear
algebra lines of code are longer an more awkward, but the rest is much
better.
The Matrix class is the exception to this: is was written to provide a
natural way to express linear algebra. However, things get a bit tricky
when you mix matrices and arrays, and even when sticking with matrices
there are confusions and limitations -- how do you express a row vs a
column vector? what do you get when you iterate over a matrix? etc.
There has been a bunch of discussion about these issues, a lot of good
ideas, a little bit of consensus about how to improve it, but no one
with the skill to do it has enough motivation to do it.
这些反映了矩阵class带来的好处和困难。我能找到的最早弃用建议是 from 2008，尽管部分原因是自那以后发生了变化的不直观行为（特别是，对矩阵进行切片和迭代将产生（行）矩阵，正如人们最有可能期望的那样）。该建议既表明这是一个极具争议的主题，又表明矩阵乘法的中缀运算符至关重要。
我能找到的下一个提及 is from 2014 which turned out to be a very fruitful thread. The ensuing discussion raises the question of handling numpy subclasses in general, which general theme is still very much on the table. There is also strong criticism:
What sparked this discussion (on Github) is that it is not possible to
write duck-typed code that works correctly for:
- ndarrays
- matrices
- scipy.sparse sparse matrixes
The semantics of all three are different; scipy.sparse is somewhere
between matrices and ndarrays with some things working randomly like
matrices and others not.
With some hyberbole added, one could say that from the developer point
of view, np.matrix is doing and has already done evil just by existing,
by messing up the unstated rules of ndarray semantics in Python.
随后对矩阵的可能未来进行了大量有价值的讨论。即使当时没有 @ 运算符，人们也对矩阵 class 的弃用及其对下游用户的影响进行了很多思考。据我所知，这次讨论直接导致了引入 matmul 的 PEP 465 的诞生。
In my opinion, a "fixed" version of np.matrix should (1) not be a
np.ndarray subclass and (2) exist in a third party library not numpy itself.
I don't think it's really feasible to fix np.matrix in its current state as
an ndarray subclass, but even a fixed matrix class doesn't really belong in
numpy itself, which has too long release cycles and compatibility
guarantees for experimentation -- not to mention that the mere existence of
the matrix class in numpy leads new users astray.
一旦 @ 运算符可用了一段时间 the discussion of deprecation surfaced again, reraising the topic 关于矩阵弃用和 scipy.sparse 的关系。
最终，first action to deprecate numpy.matrix was taken in late November 2017。关于 class 的家属：
How would the community handle the scipy.sparse matrix subclasses? These
are still in common use.
他们很长一段时间都不会去任何地方（直到稀疏的 ndarrays
至少实现）。因此 np.matrix 需要移动，而不是删除。
(source) 和
while I want to get rid of np.matrix as much as
anyone, doing that anytime soon would be really disruptive.
There are tons of little scripts out there written by people who
didn't know better; we do want them to learn not to use np.matrix but
breaking all their scripts is a painful way to do that
There are major projects like scikit-learn that simply have no
alternative to using np.matrix, because of scipy.sparse.
So I think the way forward is something like:
Now or whenever someone gets together a PR: issue a
PendingDeprecationWarning in np.matrix._init_ (unless it kills
performance for scikit-learn and friends), and put a big warning box
at the top of the docs. The idea here is to not actually break
anyone's code, but start to get out the message that we definitely
don't think anyone should use this if they have any alternative.
After there's an alternative to scipy.sparse: ramp up the warnings,
possibly all the way to FutureWarning so that existing scripts don't
break but they do get noisy warnings
Eventually, if we think it will reduce maintenance costs: split it
into a subpackage
(source).
现状
截至 2018 年 5 月（numpy 1.15，相关 pull request and commit) the matrix class docstring 包含以下注释：
It is no longer recommended to use this class, even for linear algebra. Instead use regular arrays. The class may be removed in the future.
同时 PendingDeprecationWarning 被添加到 matrix.__new__。不幸的是，deprecation warnings are (almost always) silenced by default，所以大多数 numpy 的最终用户不会看到这个强烈的提示。
最后，the numpy roadmap 截至 2018 年 11 月提到多个相关主题作为“任务和功能 [numpy 社区] 将在 中投入资源”之一：
Some things inside NumPy do not actually match the Scope of NumPy.
- A backend system for numpy.fft (so that e.g. fft-mkl doesn’t need to monkeypatch numpy)
- Rewrite masked arrays to not be a ndarray subclass – maybe in a separate project?
- MaskedArray as a duck-array type, and/or
- dtypes that support missing values
- Write a strategy on how to deal with overlap between numpy and scipy for linalg and fft (and implement it).
- Deprecate np.matrix
只要 libraries/many 用户（尤其是 scipy.sparse）依赖矩阵 class，这种状态就可能会一直存在。但是，有 ongoing discussion to move scipy.sparse to depend on something else, such as pydata/sparse.
在SciPy 1.8 (released February 2022) a sparse array API was introduced for early testing and feedback, with the potential to remove the np.matrix legacy eventually. This replicates the SciPy sparse containers with an interface that matches the behaviour of NumPy arrays (rather than matrices). Maintainers of downstream libraries such as NetworkX and scikit-learn are eager to switch to the new API as soon as possible.
无论弃用过程如何发展，用户都应在新代码中使用 ndarray class，如果可能，最好移植旧代码。最终矩阵 class 可能会以单独的包结束，以消除其当前形式存在所造成的一些负担。
matrix class 在 NumPy 中的状态如何？
我一直被告知我应该改用 ndarray class。在我写的新代码中 worth/safe 使用 matrix class 吗？我不明白为什么我应该改用 ndarrays。
tl; dr: numpy.matrix class 即将弃用。有一些高知名度的库依赖 class 作为依赖项（最大的是 scipy.sparse），这阻碍了 class 的适当短期弃用，但强烈鼓励用户改为使用 ndarray class（通常使用 numpy.array 便捷函数创建）。随着用于矩阵乘法的 @ 运算符的引入，矩阵的许多相对优势已被消除。
为什么（不是）矩阵 class？
numpy.matrix 是 numpy.ndarray 的子class。它最初是为了方便地用于涉及线性代数的计算，但与更一般的数组 class 的实例相比，它们的行为方式存在局限性和令人惊讶的差异。行为根本差异的示例：
- 形状：数组可以具有从 0 到无穷大（或 32）的任意维数。矩阵总是二维的。奇怪的是，虽然不能 创建 具有更多维度的矩阵，但可以将单个维度注入矩阵以在技术上最终得到多维矩阵：
np.matrix(np.random.rand(2,3))[None,...,None].shape == (1,2,3,1)（不是这具有任何实际意义）。 - 索引：索引数组可以为您提供任意大小的数组，具体取决于 how you are indexing it。矩阵上的索引表达式将始终为您提供一个矩阵。这意味着 2d 数组的
arr[:,0]和arr[0,:]都给你一个 1dndarray，而mat[:,0]的形状为(N,1)和mat[0,:]的形状(1,M)如果是matrix. - 算术运算：过去使用矩阵的主要原因是对矩阵的算术运算（特别是乘法和幂）执行矩阵运算（矩阵乘法和矩阵幂）。数组的相同结果是逐元素乘法和幂。因此
mat1 * mat2在mat1.shape[1] == mat2.shape[0]时有效，但arr1 * arr2在arr1.shape == arr2.shape时有效（当然结果意味着完全不同的东西）。此外，令人惊讶的是，mat1 / mat2执行两个矩阵的 elementwise 除法。这种行为可能继承自ndarray但对矩阵没有意义，特别是考虑到*. 的含义
- 特殊属性：矩阵除了数组还有a few handy attributes：
mat.A和mat.A1是与np.array(mat)和[=38具有相同值的数组视图=]，分别。mat.T和mat.H是矩阵的转置和共轭转置（伴随）；arr.T是ndarrayclass 中唯一存在的此类属性。最后，mat.I是mat的逆矩阵。
编写适用于 ndarray 或矩阵的代码非常简单。但是当两个 class 有机会必须在代码中交互时，事情就开始变得困难了。特别是，很多代码 可以 自然地为 ndarray 的子 class 工作，但是 matrix 是一个行为不端的子 [=184] =] 可以很容易地破坏试图依赖 duck typing 的代码。考虑以下使用形状为 (3,4):
import numpy as np
shape = (3, 4)
arr = np.arange(np.prod(shape)).reshape(shape) # ndarray
mat = np.matrix(arr) # same data in a matrix
print((arr + mat).shape) # (3, 4), makes sense
print((arr[0,:] + mat[0,:]).shape) # (1, 4), makes sense
print((arr[:,0] + mat[:,0]).shape) # (3, 3), surprising
添加两个对象的切片是灾难性的不同，具体取决于我们切片的维度。当形状相同时，矩阵和数组的加法都是按元素进行的。上面的前两种情况很直观：我们添加两个数组（矩阵），然后从每个数组中添加两行。最后一个案例真的很令人惊讶：我们可能打算添加两列并最终得到一个矩阵。原因当然是 arr[:,0] 的形状 (3,) 与形状 (1,3) 兼容，但 mat[:.0] 的形状 (3,1)。两人broadcast一起塑造(3,3).
最后，矩阵class的最大优势（即可以简明地表达涉及大量矩阵乘积的复杂矩阵表达式）在the @ matmul operator was introduced in python 3.5, first implemented in numpy 1.10时被移除。比较简单二次型的计算：
v = np.random.rand(3); v_row = np.matrix(v)
arr = np.random.rand(3,3); mat = np.matrix(arr)
print(v.dot(arr.dot(v))) # pre-matmul style
# 0.713447037658556, yours will vary
print(v_row * mat * v_row.T) # pre-matmul matrix style
# [[0.71344704]]
print(v @ arr @ v) # matmul style
# 0.713447037658556
从上面可以清楚地看出为什么矩阵 class 在处理线性代数时受到广泛青睐：中缀 * 运算符使表达式更简洁，更易于阅读。但是，我们使用现代 python 和 numpy 的 @ 运算符获得相同的可读性。此外，请注意，矩阵情况给了我们一个形状为 (1,1) 的矩阵，从技术上讲，它应该是一个标量。这也意味着我们不能将列向量与这个“标量”相乘：上例中的 (v_row * mat * v_row.T) * v_row.T 会引发错误，因为形状为 (1,1) 和 (3,1) 的矩阵不能相乘按照这个顺序。
为了完整起见，应该注意虽然 matmul 运算符修复了 ndarray 与矩阵相比次优的最常见情况，但在使用 ndarray 优雅地处理线性代数方面仍然存在一些缺点（尽管人们仍然倾向于相信总的来说最好坚持后者）。一个这样的例子是矩阵幂：mat ** 3 是矩阵的适当的三次矩阵幂（而它是 ndarray 的元素立方体）。不幸的是 numpy.linalg.matrix_power 相当冗长。此外，就地矩阵乘法仅适用于矩阵 class。相比之下，虽然 PEP 465 and the python grammar 都允许 @= 作为 matmul 的增强赋值，但从 numpy 1.15 开始，这并没有为 ndarrays 实现。
弃用历史
考虑到上述有关 matrix class 的复杂情况，长期以来一直在反复讨论其可能的弃用问题。 @ 中缀运算符的引入是这个过程 happened in September 2015. Unfortunately the advantages of the matrix class in earlier days meant that its use spread wide. There are libraries that depend on the matrix class (one of the most important dependent is scipy.sparse 的一个巨大先决条件，它同时使用 numpy.matrix 语义并且在增密时经常使用 returns 矩阵），因此完全弃用它们有总是有问题。
我已经在 a numpy mailing list thread from 2009 中找到
等评论numpy was designed for general purpose computational needs, not any one branch of math. nd-arrays are very useful for lots of things. In contrast, Matlab, for instance, was originally designed to be an easy front-end to linear algebra package. Personally, when I used Matlab, I found that very awkward -- I was usually writing 100s of lines of code that had nothing to do with linear algebra, for every few lines that actually did matrix math. So I much prefer numpy's way -- the linear algebra lines of code are longer an more awkward, but the rest is much better.
The Matrix class is the exception to this: is was written to provide a natural way to express linear algebra. However, things get a bit tricky when you mix matrices and arrays, and even when sticking with matrices there are confusions and limitations -- how do you express a row vs a column vector? what do you get when you iterate over a matrix? etc.
There has been a bunch of discussion about these issues, a lot of good ideas, a little bit of consensus about how to improve it, but no one with the skill to do it has enough motivation to do it.
这些反映了矩阵class带来的好处和困难。我能找到的最早弃用建议是 from 2008，尽管部分原因是自那以后发生了变化的不直观行为（特别是，对矩阵进行切片和迭代将产生（行）矩阵，正如人们最有可能期望的那样）。该建议既表明这是一个极具争议的主题，又表明矩阵乘法的中缀运算符至关重要。
我能找到的下一个提及 is from 2014 which turned out to be a very fruitful thread. The ensuing discussion raises the question of handling numpy subclasses in general, which general theme is still very much on the table. There is also strong criticism:
What sparked this discussion (on Github) is that it is not possible to write duck-typed code that works correctly for:
- ndarrays
- matrices
- scipy.sparse sparse matrixes
The semantics of all three are different; scipy.sparse is somewhere between matrices and ndarrays with some things working randomly like matrices and others not.
With some hyberbole added, one could say that from the developer point of view, np.matrix is doing and has already done evil just by existing, by messing up the unstated rules of ndarray semantics in Python.
随后对矩阵的可能未来进行了大量有价值的讨论。即使当时没有 @ 运算符，人们也对矩阵 class 的弃用及其对下游用户的影响进行了很多思考。据我所知，这次讨论直接导致了引入 matmul 的 PEP 465 的诞生。
In my opinion, a "fixed" version of np.matrix should (1) not be a np.ndarray subclass and (2) exist in a third party library not numpy itself.
I don't think it's really feasible to fix np.matrix in its current state as an ndarray subclass, but even a fixed matrix class doesn't really belong in numpy itself, which has too long release cycles and compatibility guarantees for experimentation -- not to mention that the mere existence of the matrix class in numpy leads new users astray.
一旦 @ 运算符可用了一段时间 the discussion of deprecation surfaced again, reraising the topic 关于矩阵弃用和 scipy.sparse 的关系。
最终，first action to deprecate numpy.matrix was taken in late November 2017。关于 class 的家属：
How would the community handle the scipy.sparse matrix subclasses? These are still in common use.
他们很长一段时间都不会去任何地方（直到稀疏的 ndarrays 至少实现）。因此 np.matrix 需要移动，而不是删除。
(source) 和
while I want to get rid of np.matrix as much as anyone, doing that anytime soon would be really disruptive.
There are tons of little scripts out there written by people who didn't know better; we do want them to learn not to use np.matrix but breaking all their scripts is a painful way to do that
There are major projects like scikit-learn that simply have no alternative to using np.matrix, because of scipy.sparse.
So I think the way forward is something like:
Now or whenever someone gets together a PR: issue a PendingDeprecationWarning in np.matrix._init_ (unless it kills performance for scikit-learn and friends), and put a big warning box at the top of the docs. The idea here is to not actually break anyone's code, but start to get out the message that we definitely don't think anyone should use this if they have any alternative.
After there's an alternative to scipy.sparse: ramp up the warnings, possibly all the way to FutureWarning so that existing scripts don't break but they do get noisy warnings
Eventually, if we think it will reduce maintenance costs: split it into a subpackage
(source).
现状
截至 2018 年 5 月（numpy 1.15，相关 pull request and commit) the matrix class docstring 包含以下注释：
It is no longer recommended to use this class, even for linear algebra. Instead use regular arrays. The class may be removed in the future.
同时 PendingDeprecationWarning 被添加到 matrix.__new__。不幸的是，deprecation warnings are (almost always) silenced by default，所以大多数 numpy 的最终用户不会看到这个强烈的提示。
最后，the numpy roadmap 截至 2018 年 11 月提到多个相关主题作为“任务和功能 [numpy 社区] 将在 中投入资源”之一：
Some things inside NumPy do not actually match the Scope of NumPy.
- A backend system for numpy.fft (so that e.g. fft-mkl doesn’t need to monkeypatch numpy)
- Rewrite masked arrays to not be a ndarray subclass – maybe in a separate project?
- MaskedArray as a duck-array type, and/or
- dtypes that support missing values
- Write a strategy on how to deal with overlap between numpy and scipy for linalg and fft (and implement it).
- Deprecate np.matrix
只要 libraries/many 用户（尤其是 scipy.sparse）依赖矩阵 class，这种状态就可能会一直存在。但是，有 ongoing discussion to move scipy.sparse to depend on something else, such as pydata/sparse.
在SciPy 1.8 (released February 2022) a sparse array API was introduced for early testing and feedback, with the potential to remove the np.matrix legacy eventually. This replicates the SciPy sparse containers with an interface that matches the behaviour of NumPy arrays (rather than matrices). Maintainers of downstream libraries such as NetworkX and scikit-learn are eager to switch to the new API as soon as possible.
无论弃用过程如何发展，用户都应在新代码中使用 ndarray class，如果可能，最好移植旧代码。最终矩阵 class 可能会以单独的包结束，以消除其当前形式存在所造成的一些负担。