例如,假设Dimension = 3,N = 4,我正在尝试生成以下数组:
[ [0,0,0], [0,0,1], [0,0,2], [0,0,3], [0,1,0], [0,1,1], [0,1,2], [0,1,3], [1,0,0], ... ... ..., [3,3,3] ].
(主要依靠基数N)
另一个例子,如果Dimension = 2且N = 5,则输出为:
[ [0,0], [0,1], [0,2], [0,3], [0,4], [1,0], [1,1], [1,2], [1,3], [1,4], [2,0], [2,1], ... ... ..., [4,4] ].
我目前正在使用以下代码成功完成此操作:
ParameterSpace = [ [int(i/N**j)%N for j in range(Dimension)][::-1] for i in range(N**Dimension) ]
然而,问题在于,当我尝试'大'N和Dimension时,它消耗了大量的时间和内存(特别是当N = 16而Dimension = 7时,我无法得到答案,因为它让我记忆了16Gb)
所以我想知道是否有更有效的方法。
提前致谢。
您可以使用itertools包中的产品来创建生成器对象
from itertools import product
x = [0, 1, 2, 3, 4]
a = product(x, repeat=2)
#next(a) will print (0, 0) and so on until it's exhausted
如果由于某种原因你需要整个表,你可以使用uint8
dtype和this cartesian product code的简化版本。
import numpy as np
from itertools import chain, repeat, accumulate
def cartesian_power(N, D):
dtype = f'u{2**(((N-1).bit_length() + 7) // 8 - 1).bit_length()}'
arr = np.empty((*repeat(N, D), D), dtype=dtype)
arrs = *accumulate(chain((arr,), repeat(0, D)), np.ndarray.__getitem__),
rng = np.arange(N, dtype=dtype)
idx = slice(None), *repeat(None, D-1)
for i in range(D-1, 0, -1):
arrs[i][..., i] = rng[idx[:D-i]]
arrs[i-1][1:] = arrs[i]
arr[..., 0] = rng[idx]
return arr.reshape(-1, D)
在我的8GB笔记本电脑上使用此功能16 ^ 7无疑问:
>>> cartesian_power(16, 7)
array([[ 0, 0, 0, ..., 0, 0, 0],
[ 0, 0, 0, ..., 0, 0, 1],
[ 0, 0, 0, ..., 0, 0, 2],
...,
[15, 15, 15, ..., 15, 15, 13],
[15, 15, 15, ..., 15, 15, 14],
[15, 15, 15, ..., 15, 15, 15]], dtype=uint8)
只要您想要将所有数字保存在列表中,它就会降低内存效率。查看计算以获得有关内存消耗的直觉。
Dimension = 7, N = 16
Total numbers generated = 16*16*...*16 (7 times) = 268435456
No. of elements used to represent a number = 7
Total numbers used = 268435456*7 = 268435456
Size used to represent an int in Python2 = 24 bytes
Total memory consumption = 268435456*24 = 6442450944 bytes = 6GB + Extra Overhead of each lists and lists of lists.
您可以按照@mamun的建议创建一个生成器,而不是生成整个序列,但迭代它将非常耗时。