gsutil 复制失败并出现异常无法启动新线程

Question

我正在尝试从本地复制到谷歌云存储

String gsutilCommand = String.format("gsutil -m cp -r %s %s", localPath, gcsPath);
ProcessBuilder pb = new ProcessBuilder(new String[] { "bash", "-c", gsutilCommand });
pb.redirectErrorStream(true);
Process process = pb.start();

  Process Process-6:Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in 
  _bootstrap
  error: can't start new thread
  self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
  self._target(*self._args, **self._kwargs)
  File "/root/google-cloud-sdk/platform/gsutil/gslib/command.py", line 1716, in _ApplyThreads
       status_queue=status_queue)
  File "/root/google-cloud-sdk/platform/gsutil/gslib/command.py", line 2006, in __init__
       worker_thread.start()
  File "/usr/lib/python2.7/threading.py", line 745, in start
       _start_new_thread(self.__bootstrap, ())
  error: can't start new thread
       self.run()

Answer 1

什么对我有用（python3.8，CentOS 8）：

进入 .boto 设置，取消注释并将“并行”值更改为 1，如下所示：

[GSUtil]                                                                                                                               
parallel_process_count = 1                                                                                                             
parallel_thread_count = 1

这解决了 gsutil 对线程的抱怨，即使没有使用 -m 标志也是如此。

Answer 2

如果您使用较新的 gcloud 命令而不是 gsutil 您可以从 shell 运行这些命令。我在 Linux 上测试了这个。

$ gcloud config set storage/parallel_thread_count 1
$ gcloud config set storage/thread_count 1

.boto 文件已从较新的安装中删除。