如何反序列化从Python后端发送的js中的protobuf字符串?

问题描述 投票:0回答:1

我有一个 Flask 服务器,它发送序列化的协议缓冲区消息。我已在一些环境中成功反序列化此消息,包括 C++ 客户端。但是,当我尝试在 js 中反序列化我的消息时,没有任何效果。我正在使用 protobuf.js。根据文档,我认为我做得正确。

这是protobuff文件

vtkMessage.proto
:

syntax = "proto3";

package vtkMessage;

message hash_element {
  string name = 1;
  repeated float v = 2;
}

message hash {
  repeated hash_element elem = 1;
}

message vtkMsg {
  repeated int32 tris=1;
  repeated float verts=2;
  hash vals=3;
}

这是我的 Flask 服务器

test.py
:

from flask import Flask
from flask import render_template, request
import vtkMessage_pb2

app = Flask(__name__, static_folder='')

@app.route('/test',methods = ['POST'])
def test():
  data = request.get_json()
  ret = vtkMessage_pb2.vtkMsg()
  ret.tris.extend([1,2,3])
  ret.verts.extend([0.45,0.35,0.11, 0.66,0.78,0.23, 0.11,0.01,0.14])
  a = ret.vals.elem.add()
  a.name = 'test1'
  a.v.extend([0.1, 0.2])
  a = ret.vals.elem.add()
  a.name = 'test2'
  a.v.extend([0.3, 0.5])
  print(ret)
  return ret.SerializeToString() # this decodes successfully when I use C++

@app.route('/protoTest')
def protoTest():
  return render_template('protoTest.html')

这是

templates/protoTest.html

<html>
  <meta charset="UTF-8">
  <script src="https://code.jquery.com/jquery-3.7.1.min.js" integrity="sha256-/JqT3SQfawRcv/BIHPThkBvs0OEvtFFmqPF/lYI/Cxo=" crossorigin="anonymous"></script>
  <script src="//cdn.jsdelivr.net/npm/[email protected]/dist/protobuf.js"></script>

  <script type="text/javascript">
    window.onload = main;

    function main() {
      var dat = JSON.stringify({
        step: 0,
        frames: 1,
        fname: "openfoam.vtk"
      });
      protobuf.load('vtkMessage.proto', function(err,root) {
        window.vtkMsg = root.lookupType("vtkMessage.vtkMsg");
        var proto_obj = {};

        $.ajax({
          url:'/test',
          data: dat,
          type: "POST",
          contentType: "application/json;charset=utf-8",
          async: false,
          success: function(res) {
            console.log(res);
            var buffer = new TextEncoder("utf-8").encode(res);
            proto_obj = vtkMsg.decode(buffer); // WHY IS THIS FAILING???
            console.log(proto_obj);
          },
          error: function(res) {
            console.error(res);  // sometimes this also happens
          }
        });

        console.log(proto_obj);

      });
    }
  </script>

  <body>
     <h1>Protobuf test, view in console</h1>
  </body>
</html>

要获得

vtkMessage_pb2.py
,您需要运行:
protoc -I=. --python_out=. vtkMessage.proto

我怀疑

var buffer = new TextEncoder("utf-8").encode(res);
不正确,但我没有看到任何有关如何完成此操作的文档。另外,我担心 Flask 和/或 jquery 在处理
POST
请求时做了一些奇怪的事情。

javascript python protocol-buffers
1个回答
0
投票

想要使用 RESTful 机制发送 Protobuf 序列化消息是合理的,但我不清楚为什么在这种情况下不只使用纯 JSON 和 REST。

也就是说,您的代码尝试错误地混合 JSON(您

POST
JSON 到
test
方法(然后被丢弃),并且您尝试从服务器返回二进制(无效 JSON)。

我怀疑(但不在这里显示)可以将

application/grpc
(二进制)内容发布到服务器。

相反,我展示了一种将 JSON 与 Protobuf 二进制消息相结合的方法。为了发送 Protobuf 二进制消息,您必须对其进行 base64 编码。

{
  data: "{base64-encoded serialized Protobuf message}"
}

我对 Flask 和 JavaScript 都不太熟悉,因此以下代码将受益于改进,但它有效:

from flask import Flask
from flask import jsonify, render_template, request

import base64
import vtkMessage_pb2

app = Flask(__name__, static_folder='')

@app.route('/test',methods = ['POST'])
def test():
  # Discard !?
  data = request.get_json()

  msg = vtkMessage_pb2.vtkMsg()
  msg.tris.extend([1,2,3])
  msg.verts.extend([0.45,0.35,0.11, 0.66,0.78,0.23, 0.11,0.01,0.14])
  a = msg.vals.elem.add()
  a.name = 'test1'
  a.v.extend([0.1, 0.2])
  a = msg.vals.elem.add()
  a.name = 'test2'
  a.v.extend([0.3, 0.5])
  print(msg)

  # Convert Protobuf message to binary string
  b = msg.SerializeToString()

  # Base64 encode it to bundle as JSON
  data = base64.b64encode(b).decode("ascii")

  # Return JSON message
  return jsonify({"data": data})

@app.route('/protoTest')
def protoTest():
  return render_template('protoTest.html')

并且:

<html>
  <meta charset="UTF-8">
  <script src="https://code.jquery.com/jquery-3.7.1.min.js" integrity="sha256-/JqT3SQfawRcv/BIHPThkBvs0OEvtFFmqPF/lYI/Cxo=" crossorigin="anonymous"></script>
  <script src="//cdn.jsdelivr.net/npm/[email protected]/dist/protobuf.js"></script>

  <script type="text/javascript">
    window.onload = main;

    // Base64 decode and return Uint8Array
    // See: https://stackoverflow.com/a/21797381/609290
    function base64ToBytes(base64) {
      var binaryString = atob(base64);
      var bytes = new Uint8Array(binaryString.length);
      for (var i = 0; i < binaryString.length; i++) {
        bytes[i] = binaryString.charCodeAt(i);
      }
      return bytes;
    }

    function main() {
      // POSTed but ignored by the server
      var data = JSON.stringify({
        step: 0,
        frames: 1,
        fname: "openfoam.vtk"
      });
      protobuf.load('vtkMessage.proto', function(err,root) {
        window.vtkMsg = root.lookupType("vtkMessage.vtkMsg");
        var proto_obj = {};

        $.ajax({
          url:'/test',
          data: data,
          type: "POST",
          contentType: "application/json;charset=utf-8",
          async: false,
          success: function(res) {
            // JSON response {"data":...}
            console.log(res);
            // Extract the Protobuf binary data
            b = base64ToBytes(res.data);
            // Deserialize the message
            msg = vtkMsg.decode(b);
            console.log(msg);
          },
          error: function(res) {
            console.error(res);
          }
        });

        console.log(msg);

      });
    }
  </script>

  <body>
     <h1>Protobuf test, view in console</h1>
  </body>
</html>

控制台输出包括 base64 编码的消息:

{
    "data": "CgMBAgMSJGZm5j4zM7M+rkfhPcP1KD8Urkc/H4VrPq5H4T0K1yM8KVwPPhomChEKBXRlc3QxEgjNzMw9zcxMPgoRCgV0ZXN0MhIImpmZPgAAAD8="
}

您可以使用例如Bash 提取十六进制内容:

printf "CgMBAgMSJGZm5j4zM7M+rkfhPcP1KD8Urkc/H4VrPq5H4T0K1yM8KVwPPhomChEKBXRlc3QxEgjN
zMw9zcxMPgoRCgV0ZXN0MhIImpmZPgAAAD8=" \
| base64 --decode \
| xxd -c 128 -g 128
0a0301020312246666e63e3333b33eae47e13dc3f5283f14ae473f1f856b3eae47e13d0ad7233c295c0f3e1a260a110a0574657374311208cdcccc3dcdcc4c3e0a110a05746573743212089a99993e0000003f

您可以将其粘贴到例如Protobuf Decoder确认正确。

如果您浏览

localhost:5000/protoTest
,您应该会看到 JavaScript 客户端正确重新创建的
vtkMsg

© www.soinside.com 2019 - 2024. All rights reserved.