是否可以使用从相机UNITY获得的手的位置来移动3D物体？

Question

[![在此处输入图像描述][1]][1]我想创建一个统一程序，在其中我能够根据相机的输入在屏幕上提供手部的 3D 模型。我使用 HandPoseBarracuda 来实时检测和获取手的位置。 有没有办法通过Unity中的C#代码访问装配模型的骨骼点？

如何将位置映射到装配模型？ [1]：

Answer 1

我现在查看了代码。关节球和骨骼在 Animator 游戏对象的附加脚本 HandAnimator.cs 的 LateUpdate() 函数中更新。

您可以像这样简单地获取循环中所有球位置的列表。

using UnityEngine;
using UnityEngine.UI;
**using System.Collections.Generic;**

namespace MediaPipe.HandPose
{

    public sealed class HandAnimator : MonoBehaviour
    {
        #region Editable attributes

        [SerializeField] WebcamInput _webcam = null;
        [SerializeField] ResourceSet _resources = null;
        [SerializeField] bool _useAsyncReadback = true;
        [Space]
        [SerializeField] Mesh _jointMesh = null;
        [SerializeField] Mesh _boneMesh = null;
        [Space]
        [SerializeField] Material _jointMaterial = null;
        [SerializeField] Material _boneMaterial = null;
        [Space]
        [SerializeField] RawImage _monitorUI = null;

        public List<Vector3> positions;

        #endregion

        #region Private members

        HandPipeline _pipeline;

        static readonly (int, int)[] BonePairs =
        {
        (0, 1), (1, 2), (1, 2), (2, 3), (3, 4),     // Thumb
        (5, 6), (6, 7), (7, 8),                     // Index finger
        (9, 10), (10, 11), (11, 12),                // Middle finger
        (13, 14), (14, 15), (15, 16),               // Ring finger
        (17, 18), (18, 19), (19, 20),               // Pinky
        (0, 17), (2, 5), (5, 9), (9, 13), (13, 17)  // Palm
    };

        Matrix4x4 CalculateJointXform(Vector3 pos)
          => Matrix4x4.TRS(pos, Quaternion.identity, Vector3.one * 0.07f);

        Matrix4x4 CalculateBoneXform(Vector3 p1, Vector3 p2)
        {
            var length = Vector3.Distance(p1, p2) / 2;
            var radius = 0.03f;

            var center = (p1 + p2) / 2;
            var rotation = Quaternion.FromToRotation(Vector3.up, p2 - p1);
            var scale = new Vector3(radius, length, radius);

            return Matrix4x4.TRS(center, rotation, scale);
        }

        #endregion

        #region MonoBehaviour implementation

        void Start()
          => _pipeline = new HandPipeline(_resources);

        void OnDestroy()
          => _pipeline.Dispose();

        void LateUpdate()
        {
            // Feed the input image to the Hand pose pipeline.
            _pipeline.UseAsyncReadback = _useAsyncReadback;
            _pipeline.ProcessImage(_webcam.Texture);

            var layer = gameObject.layer;

            **positions = new List<Vector3>();**
            // Joint balls
        for (var i = 0; i < HandPipeline.KeyPointCount; i++)
            {
                **Vector3 pos = _pipeline.GetKeyPoint(i);
                var xform = CalculateJointXform(pos);
                Graphics.DrawMesh(_jointMesh, xform, _jointMaterial, layer);
                positions.Add(pos);**
            }

            // Bones
            foreach (var pair in BonePairs)
            {
                var p1 = _pipeline.GetKeyPoint(pair.Item1);
                var p2 = _pipeline.GetKeyPoint(pair.Item2);
                var xform = CalculateBoneXform(p1, p2);
                Graphics.DrawMesh(_boneMesh, xform, _boneMaterial, layer);
            }

            // UI update
            _monitorUI.texture = _webcam.Texture;
        }

        #endregion
    }

} // namespace MediaPipe.HandPose

最后在列表中选择您想要的每个位置，并使用所选位置更新特殊对象的位置。

Answer 2

@Mahdi_Po 抱歉这个问题，但是您是否使用插件 https://github.com/homuler/MediaPipeUnityPlugin 或者您创建了自己的插件，因为在尝试您的脚本后发现该存储库没有这样的插件Mediapipe 库命名空间中的类