AWS Neptune 建议加快响应速度

问题描述 投票：0回答：1

我最近不得不在授权服务中使用 AWS Neptune 来对实体之间具有复杂关系的数据集进行建模，其中大多数是多对多关系。这是我想出的模型：

我的主要要求包括获取特定应用程序中用户的所有角色、功能能力、属性和安全实体，因此我创建了这个 gremlin 查询来遍历图表：

%%gremlin
g
    //.with('Neptune#enableResultCache', true)
    .V()
    .has('User','ecrid','id_here')
    .project('Roles')
    .by(
        union(
            out('ASSIGNED_TO_USER_ROLE')
            .has('usage_type',within('1','3','5','7','9','11','13','15'))
            .as('UserRole')
            .out('ASSIGNED_TO_ROLE'),
            outE('BELONGS_TO')
            .has('usage_type',within('1','3','5','7','9','11','13','15'))
            .has('deny_user_access','0')
            .inV()
            .out('ASSIGNED_TO_GROUP_ROLE')
            .as('GroupRole')
            .out('ASSIGNED_TO_ROLE')
        )
        .has('system_id','id_here')
        .dedup()
        .project('Role','Functional_Abilities','Attributes')
        .by(valueMap('role_name'))
        .by(
            out('ASSIGNED_TO_ROLE_FUNCTIONAL_ABILITY')
            .as('RoleFunctionalAbility')
            .out('ASSIGNED_TO_FUNCTIONAL_ABILITY')
            .project('Functional_Ability','Secured_Entities','Attributes')
            .by(valueMap('functional_ability_name'))
            .by(out('HAS_SE').valueMap('entity_name','secured_operations').fold())
            .by(
                select('RoleFunctionalAbility')
                .out('HAS_ROLE_FUNCTIONAL_ABILITY_ATTRIBUTES')
                .project('Value', 'Name', 'Type')
                .by(coalesce(values('attribute_value'), constant('')))
                .by(coalesce(out('HAS_ATTRIBUTE_METADATA').values('attribute_name'), constant('')))
                .by(coalesce(out('HAS_ATTRIBUTE_METADATA').values('attribute_type_name'), constant('')))
                .dedup()
                .fold()
            )
            .fold()
        )
        .by(
            union(
                select('UserRole')
                .out('HAS_USER_ROLE_ATTRIBUTE')
                .has('usage_type',within('1','3','5','7','9','11','13','15'))
                .project('Value', 'Name', 'Type')
                .by(coalesce(values('attribute_value'), constant('')))
                .by(coalesce(out('HAS_ATTRIBUTE_METADATA').values('attribute_name'), constant('')))
                .by(coalesce(out('HAS_ATTRIBUTE_METADATA').values('attribute_type_name'), constant(''))),
                select('GroupRole')
                .out('HAS_GROUP_ROLE_ATTRIBUTE')
                .has('usage_type',within('1','3','5','7','9','11','13','15'))
                .project('Value', 'Name', 'Type')
                .by(coalesce(values('attribute_value'), constant('')))
                .by(coalesce(out('HAS_ATTRIBUTE_METADATA').values('attribute_name'), constant('')))
                .by(coalesce(out('HAS_ATTRIBUTE_METADATA').values('attribute_type_name'), constant('')))
            )
            .dedup()
            .fold()
        )
        .fold()
    )

我能够使用此查询获得我想要的结果。然而，对于拥有 60 个角色的用户，每个角色都有大约 50 个功能和分配的安全实体，我得到的平均响应时间为 600 毫秒，这对于要求来说很慢，所以我想问你们是否有任何建议关于我可以改进的模型/查询，以使我的模型性能更高。顺便说一句，我正在使用 r5dx8large 实例，带有缓存的时间很好，但我想改善目前丢弃缓存的响应时间 预先感谢。

gremlin graph-databases tinkerpop amazon-neptune gremlin-server
1个回答
0
投票

我刚刚通过 Neptune Gremlin 解释器 [1] 运行了这个查询，这样我就可以看到查询计划（配置文件输出总是更好，因为它包含查询执行指标）。

Optimized Traversal
===================
Neptune steps:
[
    NeptuneGraphQueryStep(Vertex) {
        JoinGroupNode {
            PatternNode[(?1, <~label>, ?2=<User>, <~>) . project ?1 .], {estimatedCardinality=0}
            PatternNode[(?1, <ecrid>, "id_here", <~>) . project ask .], {estimatedCardinality=0}
        }, {path=[Vertex(?1):GraphStep], maxVarId=3}
    },
    NeptuneTraverserConverterStep
]
+ not converted into Neptune steps: ProjectStep([Roles],[[UnionStep([[VertexStep(OUT,[ASSIGNED_TO_USER_ROLE],vertex), NeptuneHasStep([usage_type.within([1, 3, 5, 7, 9, 11, 13, 15])])@[UserRole], VertexStep(OUT,[ASSIGNED_TO_ROLE],vertex), EndStep], [VertexStep(OUT,[BELONGS_TO],edge), NeptuneHasStep([usage_type.within([1, 3, 5, 7, 9, 11, 13, 15]), deny_user_access.eq(0)]), EdgeVertexStep(IN), VertexStep(OUT,[ASSIGNED_TO_GROUP_ROLE],vertex)@[GroupRole], VertexStep(OUT,[ASSIGNED_TO_ROLE],vertex), EndStep]]), NeptuneHasStep([system_id.eq(id_here)]), NeptuneMemoryTrackerStep, DedupGlobalStep(null,null), ProjectStep([Role, Functional_Abilities, Attributes],[[PropertyMapStep([role_name],value)], [VertexStep(OUT,[ASSIGNED_TO_ROLE_FUNCTIONAL_ABILITY],vertex)@[RoleFunctionalAbility], VertexStep(OUT,[ASSIGNED_TO_FUNCTIONAL_ABILITY],vertex), ProjectStep([Functional_Ability, Secured_Entities, Attributes],[[PropertyMapStep([functional_ability_name],value)], [VertexStep(OUT,[HAS_SE],vertex), PropertyMapStep([entity_name, secured_operations],value), NeptuneMemoryTrackerStep, FoldStep], [SelectOneStep(last,RoleFunctionalAbility,null), VertexStep(OUT,[HAS_ROLE_FUNCTIONAL_ABILITY_ATTRIBUTES],vertex), ProjectStep([Value, Name, Type],[[CoalesceStep([[NeptunePropertiesStep([attribute_value],value)], [ConstantStep]])], [CoalesceStep([[VertexStep(OUT,[HAS_ATTRIBUTE_METADATA],vertex), NeptunePropertiesStep([attribute_name],value)], [ConstantStep]])], [CoalesceStep([[VertexStep(OUT,[HAS_ATTRIBUTE_METADATA],vertex), NeptunePropertiesStep([attribute_type_name],value)], [ConstantStep]])]]), NeptuneMemoryTrackerStep, DedupGlobalStep(null,null), NeptuneMemoryTrackerStep, FoldStep]]), NeptuneMemoryTrackerStep, FoldStep], [UnionStep([[SelectOneStep(last,UserRole,null), VertexStep(OUT,[HAS_USER_ROLE_ATTRIBUTE],vertex), NeptuneHasStep([usage_type.within([1, 3, 5, 7, 9, 11, 13, 15])]), ProjectStep([Value, Name, Type],[[CoalesceStep([[NeptunePropertiesStep([attribute_value],value)], [ConstantStep]])], [CoalesceStep([[VertexStep(OUT,[HAS_ATTRIBUTE_METADATA],vertex), NeptunePropertiesStep([attribute_name],value)], [ConstantStep]])], [CoalesceStep([[VertexStep(OUT,[HAS_ATTRIBUTE_METADATA],vertex), NeptunePropertiesStep([attribute_type_name],value)], [ConstantStep]])]]), EndStep], [SelectOneStep(last,GroupRole,null), VertexStep(OUT,[HAS_GROUP_ROLE_ATTRIBUTE],vertex), NeptuneHasStep([usage_type.within([1, 3, 5, 7, 9, 11, 13, 15])]), ProjectStep([Value, Name, Type],[[CoalesceStep([[NeptunePropertiesStep([attribute_value],value)], [ConstantStep]])], [CoalesceStep([[VertexStep(OUT,[HAS_ATTRIBUTE_METADATA],vertex), NeptunePropertiesStep([attribute_name],value)], [ConstantStep]])], [CoalesceStep([[VertexStep(OUT,[HAS_ATTRIBUTE_METADATA],vertex), NeptunePropertiesStep([attribute_type_name],value)], [ConstantStep]])]]), EndStep]]), NeptuneMemoryTrackerStep, DedupGlobalStep(null,null), NeptuneMemoryTrackerStep, FoldStep]]), NeptuneMemoryTrackerStep, FoldStep]]),

WARNING: >> [ProjectStep([Roles],[[UnionStep([[VertexStep(OUT,[ASSIGNED_TO_USER_ROLE],vertex), NeptuneHasStep([usage_type.within([1, 3, 5, 7, 9, 11, 13, 15])])@[UserRole], VertexStep(OUT,[ASSIGNED_TO_ROLE],vertex), EndStep], [VertexStep(OUT,[BELONGS_TO],edge), NeptuneHasStep([usage_type.within([1, 3, 5, 7, 9, 11, 13, 15]), deny_user_access.eq(0)]), EdgeVertexStep(IN), VertexStep(OUT,[ASSIGNED_TO_GROUP_ROLE],vertex)@[GroupRole], VertexStep(OUT,[ASSIGNED_TO_ROLE],vertex), EndStep]]), NeptuneHasStep([system_id.eq(id_here)]), NeptuneMemoryTrackerStep, DedupGlobalStep(null,null), ProjectStep([Role, Functional_Abilities, Attributes],[[PropertyMapStep([role_name],value)], [VertexStep(OUT,[ASSIGNED_TO_ROLE_FUNCTIONAL_ABILITY],vertex)@[RoleFunctionalAbility], VertexStep(OUT,[ASSIGNED_TO_FUNCTIONAL_ABILITY],vertex), ProjectStep([Functional_Ability, Secured_Entities, Attributes],[[PropertyMapStep([functional_ability_name],value)], [VertexStep(OUT,[HAS_SE],vertex), PropertyMapStep([entity_name, secured_operations],value), NeptuneMemoryTrackerStep, FoldStep], [SelectOneStep(last,RoleFunctionalAbility,null), VertexStep(OUT,[HAS_ROLE_FUNCTIONAL_ABILITY_ATTRIBUTES],vertex), ProjectStep([Value, Name, Type],[[CoalesceStep([[NeptunePropertiesStep([attribute_value],value)], [ConstantStep]])], [CoalesceStep([[VertexStep(OUT,[HAS_ATTRIBUTE_METADATA],vertex), NeptunePropertiesStep([attribute_name],value)], [ConstantStep]])], [CoalesceStep([[VertexStep(OUT,[HAS_ATTRIBUTE_METADATA],vertex), NeptunePropertiesStep([attribute_type_name],value)], [ConstantStep]])]]), NeptuneMemoryTrackerStep, DedupGlobalStep(null,null), NeptuneMemoryTrackerStep, FoldStep]]), NeptuneMemoryTrackerStep, FoldStep], [UnionStep([[SelectOneStep(last,UserRole,null), VertexStep(OUT,[HAS_USER_ROLE_ATTRIBUTE],vertex), NeptuneHasStep([usage_type.within([1, 3, 5, 7, 9, 11, 13, 15])]), ProjectStep([Value, Name, Type],[[CoalesceStep([[NeptunePropertiesStep([attribute_value],value)], [ConstantStep]])], [CoalesceStep([[VertexStep(OUT,[HAS_ATTRIBUTE_METADATA],vertex), NeptunePropertiesStep([attribute_name],value)], [ConstantStep]])], [CoalesceStep([[VertexStep(OUT,[HAS_ATTRIBUTE_METADATA],vertex), NeptunePropertiesStep([attribute_type_name],value)], [ConstantStep]])]]), EndStep], [SelectOneStep(last,GroupRole,null), VertexStep(OUT,[HAS_GROUP_ROLE_ATTRIBUTE],vertex), NeptuneHasStep([usage_type.within([1, 3, 5, 7, 9, 11, 13, 15])]), ProjectStep([Value, Name, Type],[[CoalesceStep([[NeptunePropertiesStep([attribute_value],value)], [ConstantStep]])], [CoalesceStep([[VertexStep(OUT,[HAS_ATTRIBUTE_METADATA],vertex), NeptunePropertiesStep([attribute_name],value)], [ConstantStep]])], [CoalesceStep([[VertexStep(OUT,[HAS_ATTRIBUTE_METADATA],vertex), NeptunePropertiesStep([attribute_type_name],value)], [ConstantStep]])]]), EndStep]]), NeptuneMemoryTrackerStep, DedupGlobalStep(null,null), NeptuneMemoryTrackerStep, FoldStep]]), NeptuneMemoryTrackerStep, FoldStep]])] << (or one of the children for each step) is not supported natively yet

由于我在此回复中提供的类似原因，查询很早就脱离了优化的查询路径：优化 Gremlin 查询性能以过滤和检索具有许多属性的数据

您需要尝试重写此查询的部分内容，以便更多查询将使用 Neptune 特定的运算符执行，而不是退回到 TinkerPop 执行。

[1] https://docs.aws.amazon.com/neptune/latest/userguide/gremlin-explain.html

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.