我使用Keras和TensorFlow作为后端来创建和训练一个简单的CNN。我能够将模型及其权重保存在.pb文件中，冻结并优化它以进行推理，但是当我尝试将其加载到OpenCV 3.4.1中时，我得到错误：
flatten/Shape:Shape(max_pooling2d/MaxPool)
T:0
out_type:[ ]
OpenCV(3.4.1) Error: Unspecified error (Unknown layer type Shape in op flatten/Shape) in populateNet, file /home/dev/opencv-3.4.1/modules/dnn/src/tensorflow/tf_importer.cpp, line 1582
Traceback (most recent call last):
File "test.py", line 67, in <module>
net = cv.dnn.readNetFromTensorflow('graph.pb')
cv2.error: OpenCV(3.4.1) /home/dev/opencv-3.4.1/modules/dnn/src/tensorflow/tf_importer.cpp:1582: error: (-2) Unknown layer type Shape in op flatten/Shape in function populateNet
这与其他问题基本上是同一个问题：How to import TensorFlow model with flatten layer in OpenCV?。
在这个帖子中很好地解释了错误的原因。建议的解决方法是直接使用tf.reshape而不是使用Keras API。
但是我不确切知道如何做到这一点。我尝试使用功能API并替换：
x = Flatten()(x)
通过：
x = tf.reshape(x, [-1, some_value])
但这不起作用，我收到以下错误：
Traceback (most recent call last):
File "test.py", line 57, in <module>
tf_out = model.predict(inp)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/_impl/keras/models.py", line 965, in predict
self.build()
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/_impl/keras/models.py", line 578, in build
self.model = Model(self.inputs, self.outputs[0], name=self.name + '_model')
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/_impl/keras/engine/topology.py", line 678, in __init__
super(Network, self).__init__(inputs, outputs, name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/layers/network.py", line 341, in __init__
'(thus holding past layer metadata). Found: ' + str(x))
ValueError: Output tensors to a Model must be the output of a TensorFlow `Layer` (thus holding past layer metadata). Found: Tensor("activation_4/Softmax:0", shape=(?, 10), dtype=float32)
知道如何在仍然使用Keras进行大部分工作时导出TensorFlow模型吗？
我终于找到了解决问题的方法。我们的想法是使用Keras声明模型，使用TensorFlow API将其转换为protobuf图形定义，然后深入生成的图形定义以删除不支持的节点。
我将所有内容封装到以下python文件中：
import tensorflow as tf
from tensorflow.core import framework
def find_all_nodes(graph_def, **kwargs):
for node in graph_def.node:
for key, value in kwargs.items():
if getattr(node, key) != value:
break
else:
yield node
raise StopIteration
def find_node(graph_def, **kwargs):
try:
return next(find_all_nodes(graph_def, **kwargs))
except StopIteration:
raise ValueError(
'no node with attributes: {}'.format(
', '.join("'{}': {}".format(k, v) for k, v in kwargs.items())))
def walk_node_ancestors(graph_def, node_def, exclude=set()):
openlist = list(node_def.input)
closelist = set()
while openlist:
name = openlist.pop()
if name not in exclude:
node = find_node(graph_def, name=name)
openlist += list(node.input)
closelist.add(name)
return closelist
def remove_nodes_by_name(graph_def, node_names):
for i in reversed(range(len(graph_def.node))):
if graph_def.node[i].name in node_names:
del graph_def.node[i]
def make_shape_node_const(node_def, tensor_values):
node_def.op = 'Const'
node_def.ClearField('input')
node_def.attr.clear()
node_def.attr['dtype'].type = framework.types_pb2.DT_INT32
tensor = node_def.attr['value'].tensor
tensor.dtype = framework.types_pb2.DT_INT32
tensor.tensor_shape.dim.add()
tensor.tensor_shape.dim[0].size = len(tensor_values)
for value in tensor_values:
tensor.tensor_content += value.to_bytes(4, 'little')
output_shape = node_def.attr['_output_shapes']
output_shape.list.shape.add()
output_shape.list.shape[0].dim.add()
output_shape.list.shape[0].dim[0].size = len(tensor_values)
def make_cv2_compatible(graph_def):
# A reshape node needs a shape node as its second input to know how it
# should reshape its input tensor.
# When exporting a model using Keras, this shape node is computed
# dynamically using `Shape`, `StridedSlice` and `Pack` operators.
# Unfortunately those operators are not supported yet by the OpenCV API.
# The goal here is to remove all those unsupported nodes and hard-code the
# shape layer as a const tensor instead.
for reshape_node in find_all_nodes(graph_def, op='Reshape'):
# Get a reference to the shape node
shape_node = find_node(graph_def, name=reshape_node.input[1])
# Find and remove all unsupported nodes
garbage_nodes = walk_node_ancestors(graph_def, shape_node,
exclude=[reshape_node.input[0]])
remove_nodes_by_name(graph_def, garbage_nodes)
# Infer the shape tensor from the reshape output tensor shape
if not '_output_shapes' in reshape_node.attr:
raise AttributeError(
'cannot infer the shape node value from the reshape node. '
'Please set the `add_shapes` argument to `True` when calling '
'the `Session.graph.as_graph_def` method.')
output_shape = reshape_node.attr['_output_shapes'].list.shape[0]
output_shape = [dim.size for dim in output_shape.dim]
# Hard-code the inferred shape in the shape node
make_shape_node_const(shape_node, output_shape[1:])
有了这个，我可以执行以下脚本：
import tensorflow as tf
from tensorflow.python.keras.models import Sequential
from tensorflow.python.keras.layers import Dense, Activation, Conv2D, MaxPooling2D, Flatten, Reshape
from tensorflow.python.keras import backend as K
import numpy as np
import graph_util
# Define the model in Keras
model = Sequential()
model.add(Conv2D(32,kernel_size=(3,3),input_shape=(28,28,1)))
model.add(Activation('relu'))
model.add(Conv2D(32,kernel_size=(3,3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Flatten())
model.add(Dense(128))
model.add(Activation('relu'))
model.add(Dense(10))
model.add(Activation('softmax'))
model.summary()
# Get Keras prediction
inp = np.random.standard_normal([1, 28, 28, 1]).astype(np.float32)
tf_out = model.predict(inp)
# Serialize and fix the graph
sess = K.get_session()
graph_def = sess.graph.as_graph_def(add_shapes=True)
graph_def = tf.graph_util.convert_variables_to_constants(sess, graph_def, [model.output.name.split(':')[0]])
graph_util.make_cv2_compatible(graph_def)
# Print the graph nodes
print('\n'.join(node.name for node in graph_def.node))
# Save the graph as a binary protobuf2 file
tf.train.write_graph(graph_def, '', 'model.pb', as_text=False)
# Get OpenCV prediction
import cv2 as cv
net = cv.dnn.readNetFromTensorflow('model.pb')
net.setInput(inp.transpose(0, 3, 1, 2))
cv_out = net.forward()
print(np.max(np.abs(tf_out - cv_out)))
输出范围为1e-7,1e-8
我有同样的问题。最后经过大量搜索并测试了很多解决方案，我可以在opencv中导入keras分类训练模型。首先，你必须用tensorflow.python.keras替换keras。只需从代码行中删除keras并将其替换为tensorflow.python.keras其次，您必须使用以下代码替换flatten layares：只需删除展平图层并复制并过去：
a,b,c,d = model.output_shape
a = b*c*d
model.add(K.layers.Permute([1, 2, 3])) # Indicate NHWC data layout
model.add(K.layers.Reshape((a,)))
现在您可以重新训练您的模型。训练结束后，以.h5格式保存模型。现在你必须将keras .h5模型转换为tensorflow .pb模型。为了转换模型我使用了这个解决方法：https://stackoverflow.com/a/53386325/5208522现在你可以轻松地在opencv中导入.pb模型，它可以很好地工作。