如何生成给定维度的计数“向量空间”,以一些整数为模

问题描述 投票:0回答:3

例如,假设Dimension = 3,N = 4,我正在尝试生成以下数组:

[ [0,0,0], [0,0,1], [0,0,2], [0,0,3], [0,1,0], [0,1,1], [0,1,2], [0,1,3], [1,0,0], ... ... ..., [3,3,3] ].

(主要依靠基数N)

另一个例子,如果Dimension = 2且N = 5,则输出为:

[ [0,0], [0,1], [0,2], [0,3], [0,4], [1,0], [1,1], [1,2], [1,3], [1,4], [2,0], [2,1], ... ... ..., [4,4] ].

我目前正在使用以下代码成功完成此操作:

ParameterSpace = [ [int(i/N**j)%N for j in range(Dimension)][::-1] for i in range(N**Dimension) ]

然而,问题在于,当我尝试'大'N和Dimension时,它消耗了大量的时间和内存(特别是当N = 16而Dimension = 7时,我无法得到答案,因为它让我记忆了16Gb)

所以我想知道是否有更有效的方法。

提前致谢。

python numpy
3个回答
1
投票

您可以使用itertools包中的产品来创建生成器对象

from itertools import product
x = [0, 1, 2, 3, 4]
a = product(x, repeat=2)
#next(a) will print (0, 0) and so on until it's exhausted

1
投票

如果由于某种原因你需要整个表,你可以使用uint8 dtype和this cartesian product code的简化版本。

import numpy as np
from itertools import chain, repeat, accumulate

def cartesian_power(N, D):
    dtype = f'u{2**(((N-1).bit_length() + 7) // 8 - 1).bit_length()}'
    arr = np.empty((*repeat(N, D), D), dtype=dtype)
    arrs = *accumulate(chain((arr,), repeat(0, D)), np.ndarray.__getitem__),
    rng = np.arange(N, dtype=dtype)
    idx = slice(None), *repeat(None, D-1)
    for i in range(D-1, 0, -1):
        arrs[i][..., i] = rng[idx[:D-i]]
        arrs[i-1][1:] = arrs[i]
    arr[..., 0] = rng[idx]
    return arr.reshape(-1, D)

在我的8GB笔记本电脑上使用此功能16 ^ 7无疑问:

>>> cartesian_power(16, 7)
array([[ 0,  0,  0, ...,  0,  0,  0],
       [ 0,  0,  0, ...,  0,  0,  1],
       [ 0,  0,  0, ...,  0,  0,  2],
       ...,
       [15, 15, 15, ..., 15, 15, 13],
       [15, 15, 15, ..., 15, 15, 14],
       [15, 15, 15, ..., 15, 15, 15]], dtype=uint8)

0
投票

只要您想要将所有数字保存在列表中,它就会降低内存效率。查看计算以获得有关内存消耗的直觉。

Dimension = 7, N = 16
Total numbers generated = 16*16*...*16 (7 times) = 268435456
No. of elements used to represent a number = 7
Total numbers used = 268435456*7 = 268435456
Size used to represent an int in Python2 = 24 bytes
Total memory consumption = 268435456*24 = 6442450944 bytes = 6GB + Extra Overhead of each lists and lists of lists.

link

您可以按照@mamun的建议创建一个生成器,而不是生成整个序列,但迭代它将非常耗时。

© www.soinside.com 2019 - 2024. All rights reserved.