ND4J 切片是否复制原始数组?
Does ND4J slicing make a copy of the original array?
ND4J INDArray
切片是通过 java - Get an arbitrary slice of a Nd4j array - Stack Overflow 中回答的重载 get()
方法之一实现的。由于 INDArray
占用了一个连续的本机内存块,使用 get()
进行切片是否会复制原始内存(尤其是行切片,其中可以创建一个新的 INDArray
相同的后备内存)?
我找到了另一个 INDArray
方法 subArray()
。这个有什么区别吗?
我问这个是因为我正在尝试创建一个可以直接从 INDArray
中提取数据的 DatasetIterator
,并且我想消除可能的开销。源码抽象太多,自己没找到实现。
在 , and the answer can be found in Indexing — NumPy v1.16 Manual:
中提出了一个关于 NumPy 的类似问题
The rule of thumb here can be: in the context of lvalue indexing (i.e. the indices are placed in the left hand side value of an assignment), no view or copy of the array is created (because there is no need to). However, with regular values, the above rules for creating views does apply.
简短的回答是:不,它在可能的情况下使用引用。要制作副本,可以调用 .dup()
函数。
引用https://deeplearning4j.org/docs/latest/nd4j-overview
Views: When Two or More NDArrays Refer to the Same Data
A key concept in ND4J is the fact that two NDArrays can actually point to the same
underlying data in memory. Usually, we have one NDArray referring to
some subset of another array, and this only occurs for certain
operations (such as INDArray.get(), INDArray.transpose(),
INDArray.getRow() etc. This is a powerful concept, and one that is
worth understanding.
There are two primary motivations for this:
There are considerable performance benefits, most notably in avoiding
copying arrays We gain a lot of power in terms of how we can perform
operations on our NDArrays Consider a simple operation like a matrix
transpose on a large (10,000 x 10,000) matrix. Using views, we can
perform this matrix transpose in constant time without performing any
copies (i.e., O(1) in big O notation), avoiding the considerable cost
copying all of the array elements. Of course, sometimes we do want to
make a copy - at which point we can use the INDArray.dup() to get a
copy. For example, to get a copy of a transposed matrix, use INDArray
out = myMatrix.transpose().dup(). After this dup() call, there will be
no link between the original array myMatrix and the array out (thus,
changes to one will not impact the other).
ND4J INDArray
切片是通过 java - Get an arbitrary slice of a Nd4j array - Stack Overflow 中回答的重载 get()
方法之一实现的。由于 INDArray
占用了一个连续的本机内存块,使用 get()
进行切片是否会复制原始内存(尤其是行切片,其中可以创建一个新的 INDArray
相同的后备内存)?
我找到了另一个 INDArray
方法 subArray()
。这个有什么区别吗?
我问这个是因为我正在尝试创建一个可以直接从 INDArray
中提取数据的 DatasetIterator
,并且我想消除可能的开销。源码抽象太多,自己没找到实现。
在
The rule of thumb here can be: in the context of lvalue indexing (i.e. the indices are placed in the left hand side value of an assignment), no view or copy of the array is created (because there is no need to). However, with regular values, the above rules for creating views does apply.
简短的回答是:不,它在可能的情况下使用引用。要制作副本,可以调用 .dup()
函数。
引用https://deeplearning4j.org/docs/latest/nd4j-overview
Views: When Two or More NDArrays Refer to the Same Data
A key concept in ND4J is the fact that two NDArrays can actually point to the same underlying data in memory. Usually, we have one NDArray referring to some subset of another array, and this only occurs for certain operations (such as INDArray.get(), INDArray.transpose(), INDArray.getRow() etc. This is a powerful concept, and one that is worth understanding.
There are two primary motivations for this:
There are considerable performance benefits, most notably in avoiding copying arrays We gain a lot of power in terms of how we can perform operations on our NDArrays Consider a simple operation like a matrix transpose on a large (10,000 x 10,000) matrix. Using views, we can perform this matrix transpose in constant time without performing any copies (i.e., O(1) in big O notation), avoiding the considerable cost copying all of the array elements. Of course, sometimes we do want to make a copy - at which point we can use the INDArray.dup() to get a copy. For example, to get a copy of a transposed matrix, use INDArray out = myMatrix.transpose().dup(). After this dup() call, there will be no link between the original array myMatrix and the array out (thus, changes to one will not impact the other).