[我以前在预训练网络上的Tensorflow.keras中添加激活和/或内核的正则化,使用层上的循环:
if regul_what is 'kernel':
for layer in model.layers:
if isinstance(layer, DepthwiseConv2D):
layer.add_loss(regularizers.l1_l2(l1,l2)(layer.depthwise_kernel))
elif isinstance(layer, layers.Conv2D) or isinstance(layer, layers.Dense):
layer.add_loss(regularizers.l1_l2(l1,l2)(layer.kernel))
if regul_what is 'activity':
for layer in model.layers:
if isinstance(layer, Activation):
layer.add_loss(regularizers.l1_l2(l1,l2)(layer.output))
在升级到tensorflow 2.0之前,它曾经可以正常工作(据我测试)。>
现在,我需要将整个框架更新为tensorflow 2.0。执行前一个代码后,在应用add_loss()时将返回以下错误:
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-123-cc0c5783731e> in <module> 3 if ('_relu' in layer.name): #isinstance(layer, Activation): 4 #layer.activity_regularizer = regularizers.l1_l2(l1,l2) ----> 5 layer.add_loss(regularizers.l1_l2(l1,l2)(layer.output)) ~/miniconda/envs/l1l2/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py in add_loss(self, losses, inputs) 1119 if eager_losses and not in_call_context: 1120 raise ValueError( -> 1121 'Expected a symbolic Tensors or a callable for the loss value. ' 1122 'Please wrap your loss computation in a zero argument `lambda`.') 1123 ValueError: Expected a symbolic Tensors or a callable for the loss value. Please wrap your loss computation in a zero argument `lambda`.
因此,我试图引入零自变量lambda函数,如下所示:
if regul_what is 'kernel': for layer in model.layers: if isinstance(layer, DepthwiseConv2D): layer.add_loss(lambda: regularizers.l1_l2(l1,l2)(layer.depthwise_kernel)) elif isinstance(layer, layers.Conv2D) or isinstance(layer, layers.Dense): layer.add_loss(lambda: regularizers.l1_l2(l1,l2)(layer.kernel)) if regul_what is 'activity': for layer in model.layers: if isinstance(layer, Activation): layer.add_loss(lambda: regularizers.l1_l2(l1,l2)(layer.output))
随着lambda的引入,add_loss循环通过而没有错误,但是当训练开始时我得到了错误:
File "~/miniconda/envs/l1l2/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 1297, in fit_generator steps_name='steps_per_epoch') File "~/miniconda/envs/l1l2/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_generator.py", line 295, in model_iteration progbar.on_batch_end(step, batch_logs) File "~/miniconda/envs/l1l2/lib/python3.6/site-packages/tensorflow_core/python/keras/callbacks.py", line 760, in on_batch_end self.progbar.update(self.seen, self.log_values) File "~/miniconda/envs/l1l2/lib/python3.6/site-packages/tensorflow_core/python/keras/utils/generic_utils.py", line 440, in update avg = np.mean(self._values[k][0] / max(1, self._values[k][1])) File "<__array_function__ internals>", line 6, in mean File "~/miniconda/envs/l1l2/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 3257, in mean out=out, **kwargs) File "~/miniconda/envs/l1l2/lib/python3.6/site-packages/numpy/core/_methods.py", line 135, in _mean arr = asanyarray(a) File "~/miniconda/envs/l1l2/lib/python3.6/site-packages/numpy/core/_asarray.py", line 138, in asanyarray return array(a, dtype, copy=False, order=order, subok=True) File "~/miniconda/envs/l1l2/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py", line 736, in __array__ " array.".format(self.name)) NotImplementedError: Cannot convert a symbolic Tensor (truediv:0) to a numpy array.
我不知道该如何解决。。预先感谢您的帮助!
我以前在预训练的网络上的Tensorflow.keras中添加激活和/或内核的正则化,使用层循环:如果regul_what是'kernel':对于model.layers中的层:...
通过在行的开头禁用急切执行: