BERT对Colab TypeError上的TPU的估计器进行微调:* =:'NoneType'和'int'的不受支持的操作数类型

问题描述 投票:2回答:1

我在Google的colab上写了一本jupyter笔记本,以微调(用于文本分类)我只接受过阿拉伯语培训的BERT版本。当培训开始时,我无法解决这个错误。

我关注了Google在github上给的笔记本

模型构建代码:

model_fn = model_fn_builder(
  bert_config=modeling.BertConfig.from_json_file(CONFIG_FILE),
  num_labels=len(label_list),
  init_checkpoint=INIT_CHECKPOINT,
  learning_rate=LEARNING_RATE,
  num_train_steps=num_train_steps,
  num_warmup_steps=num_warmup_steps,
  use_tpu=True,
  use_one_hot_embeddings=True
)


tpu_cluster_resolver = tf.contrib.cluster_resolver.TPUClusterResolver(TPU_ADDRESS)

run_config = tf.contrib.tpu.RunConfig(
    cluster=tpu_cluster_resolver,
    model_dir=OUTPUT_DIR,
    save_checkpoints_steps=SAVE_CHECKPOINTS_STEPS,
    tpu_config=tf.contrib.tpu.TPUConfig(
        iterations_per_loop=ITERATIONS_PER_LOOP,
        num_shards=NUM_TPU_CORES,
        per_host_input_for_training=tf.contrib.tpu.InputPipelineConfig.PER_HOST_V2))

estimator = tf.contrib.tpu.TPUEstimator(
    use_tpu=USE_TPU,
    model_fn=model_fn,
    config=run_config,
    train_batch_size=TRAIN_BATCH_SIZE,
    eval_batch_size=EVAL_BATCH_SIZE,
    predict_batch_size=PREDICT_BATCH_SIZE,)

train_input_fn = input_fn_builder(
    features=train_features,
    seq_length=MAX_SEQ_LENGTH,
    is_training=True,
    drop_remainder=False)

#tf.reset_default_graph()
print(f'Beginning Training!')
current_time = datetime.now()
estimator.train(input_fn=train_input_fn, max_steps=TRAIN_STEPS)
print("Training took time ", datetime.now() - current_time)

错误代码:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/tpu/tpu_sharding.py in _unshard_shape(self, shape)
    214                        (shape.as_list(), self._shard_dimension))
    215     dims = shape.as_list()
--> 216     dims[self._shard_dimension] *= self._number_of_shards
    217     return tensor_shape.as_shape(dims)
    218 

TypeError: unsupported operand type(s) for *=: 'NoneType' and 'int'

参数和其余代码在colab笔记本的此共享副本中:colab_link

python google-colaboratory tensorflow-estimator tpu google-cloud-tpu
1个回答
0
投票

为了社区的利益,在本部分中提及答案(即使在评论部分中也已回答)。>

在功能中将参数drop_remainder设置为Trueinput_fn_builder已解决该问题。

下面显示了相应的代码片段:

train_input_fn = input_fn_builder(
    features=train_features,
    seq_length=MAX_SEQ_LENGTH,
    is_training=True,
    drop_remainder=False)
© www.soinside.com 2019 - 2024. All rights reserved.