R mxnet 1 pass返回NAN作为丢失值

Question

这是预期的行为吗？

library(mxnet)

hidden_u_1 <- 100
activ_hidden_1 <- 'tanh'

hidden_u_2 <- 1

learn_rate <- 0.001

initializer <- mx.init.uniform(1)

optimizer <- 'rmsprop' #sgd

loss <- mx.metric.mse

device.cpu <- mx.cpu()

mini_batch <- 64 #8

rounds <- 1 #2


## data symbols

nn_data <- mx.symbol.Variable('data')
nn_label <- mx.symbol.Variable('label')


## first fully connected layer

flatten <- mx.symbol.Flatten(data = nn_data)

fc1 <- mx.symbol.FullyConnected(data = flatten
                                , num_hidden = hidden_u_1)

activ1 <- mx.symbol.Activation(data = fc1, act.type = activ_hidden_1)

## second fully connected layer

fc2 <- mx.symbol.FullyConnected(data = activ1, num_hidden = hidden_u_2)

q_func <- mx.symbol.LinearRegressionOutput(data = fc2, label = nn_label, name = 'regr')


# initialize NN

train.x <- matrix(rnorm(640, 0, 1), ncol = 10)
train.x <- t(train.x)
dim(train.x) <- c(nrow(train.x), 1, 1, ncol(train.x))
train.y = rnorm(64, 0, 1)

nn_model <- mx.model.FeedForward.create(
     symbol = q_func,
     X = train.x,
     y = train.y,
     ctx = device.cpu,
     num.round = rounds,
     array.batch.size = mini_batch,
     optimizer = optimizer,
     eval.metric = loss,
     learning.rate = learn_rate,
     initializer = initializer
)

在大小超过1的样本的小批量上计算损失值的1轮（通过）总是返回NaN，而2次和更多次通过从第2遍开始给出有限值。

如果样本的数量是小批量的n倍（例如，64/8），那么即使在1轮之后，损失也是可计算的。

Answer 1

这是MXNet中的一个问题，据报道here并且是固定的here。