简单神经网络中的问题

Question

我在张量流中尝试了以下具有不同变体的代码来解决简单的回归问题。我将数据合成为

y=10.0*x

。一种输入变量和一种结果变量。但张量流给我带来了约 2000 到约 200000 的损失。我使用 MSE 作为损失函数。也尝试过relu激活，没有任何用处。

1. 对于这样的[简单]回归问题，合适的模型应该是什么？
2. 对于像

   y=x^3

   这样的问题，模型应该是什么？

def PolynomialModel():
    inp = layers.Input((1))
    l=layers.Dense(16, activation='tanh')(inp)
    l=layers.Dense(8, activation='tanh')(l)
    l=layers.Dropout(.5)(l)
    l=layers.Dense(4, activation='tanh')(l)
    l=layers.Dropout(.5)(l)
    output=layers.Dense(1, activation='tanh')(l)
    return models.Model(inp,output)

Answer 1

事实上，您不需要隐藏层，而是需要输入多项式次数 + 1 的大小。有关更多信息，请查看

PolynomialFeatures

的文档。sklearn

输出：

import tensorflow as tf
from tensorflow.keras import layers, models, optimizers, activations

def PolynomialModel(degree):
    inp = layers.Input((degree+1))
    out = layers.Dense(1)(inp)  # activation='linear'
    return models.Model(inp, out)

# Suppose you want to fit a polynomial function of degree 3
degree = 3
model = PolynomialModel(degree)
model.compile(loss='mean_squared_error',  optimizer=optimizers.Adam(0.1))

# You need an input vector of degree+1 (here: x**0, x**1, x**2 and x**3)
x = tf.linspace(-20, 20, 1000)
X = tf.transpose(tf.convert_to_tensor([x**p for p in range(degree+1)]))
y = x**3

model.fit(X, y, epochs=200)
model.predict([[4**0, 4**1, 4**2, 4**3]])

测试：

Epoch 1/200
32/32 [==============================] - 0s 974us/step - loss: 891936.5000
Epoch 2/200
32/32 [==============================] - 0s 945us/step - loss: 25650.7812
Epoch 3/200
32/32 [==============================] - 0s 897us/step - loss: 1188.4584
Epoch 4/200
32/32 [==============================] - 0s 1ms/step - loss: 75.3480
Epoch 5/200
32/32 [==============================] - 0s 2ms/step - loss: 21.2651
...
Epoch 196/200
32/32 [==============================] - 0s 1ms/step - loss: 1.2130e-11
Epoch 197/200
32/32 [==============================] - 0s 1ms/step - loss: 1.5810e-11
Epoch 198/200
32/32 [==============================] - 0s 1ms/step - loss: 1.2358e-11
Epoch 199/200
32/32 [==============================] - 0s 1ms/step - loss: 1.2775e-11
Epoch 200/200
32/32 [==============================] - 0s 1ms/step - loss: 1.4443e-11

注意：

>>> model.predict([[4**0, 4**1, 4**2, 4**3]])
1/1 [==============================] - 0s 63ms/step
array([[64.]], dtype=float32)

>>> model.predict([[3**0, 3**1, 3**2, 3**3]])
1/1 [==============================] - 0s 51ms/step
array([[27.000002]], dtype=float32)

并不是真正必要的（只有......），但我想要与

x**0

相同的行为。所以你也可以考虑如果你的学位是 3，输入大小是 3（而不是 4）。