使用循环的Qliksense REST偏移分页

Question

我需要对来自Hubspot API的所有记录进行分页，然后我陷入偏移分页循环。根据Hubspot的API文档，响应中没有“总记录”路径，而是“has-more”字段告诉我们是否有更多记录可以从此门户获取。可用于分页的两个参数是

vidOffset & has-more

这是qliksense脚本通过rest连接器自定义分页的样子。

LIB CONNECT TO 'HubSpot ';
// Action required: Implement the logic to retrieve the total records from the REST source and assign to the 'total' local variable.
Let total = 0;

Let totalfetched = 0;
Let startAt = 0;
Let pageSize = 100;

for startAt = 0 to total step pageSize
RestConnectorMasterTable:
SQL SELECT 
    "has-more",
    "vid-offset",
    "__KEY_root",
    (SELECT 
        "addedAt",
        "vid" AS "vid_u0",
        "canonical-vid",
        "portal-id",
        "is-contact",
        "profile-token",
        "profile-url",
        "__KEY_contacts",
        "__FK_contacts",
        (SELECT 
            "@Value",
            "__FK_merged-vids"
        FROM "merged-vids" FK "__FK_merged-vids" ArrayValueAlias "@Value"),
        (SELECT 
            "__KEY_properties",
            "__FK_properties",
            (SELECT 
                "value",
                "__FK_firstname"
            FROM "firstname" FK "__FK_firstname"),
            (SELECT 
                "value" AS "value_u0",
                "__FK_lastmodifieddate"
            FROM "lastmodifieddate" FK "__FK_lastmodifieddate"),
            (SELECT 
                "value" AS "value_u1",
                "__FK_company"
            FROM "company" FK "__FK_company"),
            (SELECT 
                "value" AS "value_u2",
                "__FK_lastname"
            FROM "lastname" FK "__FK_lastname")
        FROM "properties" PK "__KEY_properties" FK "__FK_properties"),
        (SELECT 
            "@Value" AS "@Value_u0",
            "__FK_form-submissions"
        FROM "form-submissions" FK "__FK_form-submissions" ArrayValueAlias "@Value_u0"),
        (SELECT 
            "vid",
            "saved-at-timestamp",
            "deleted-changed-timestamp",
            "__KEY_identity-profiles",
            "__FK_identity-profiles",
            (SELECT 
                "type",
                "value" AS "value_u3",
                "timestamp",
                "is-primary",
                "__FK_identities"
            FROM "identities" FK "__FK_identities")
        FROM "identity-profiles" PK "__KEY_identity-profiles" FK "__FK_identity-profiles"),
        (SELECT 
            "@Value" AS "@Value_u1",
            "__FK_merge-audits"
        FROM "merge-audits" FK "__FK_merge-audits" ArrayValueAlias "@Value_u1")
    FROM "contacts" PK "__KEY_contacts" FK "__FK_contacts")
FROM JSON (wrap on) "root" PK "__KEY_root"
WITH CONNECTION(Url "https://api.hubapi.com/contacts/v1/lists/all/contacts/all");
// Action required: change URL included in 'WITH CONNECTION' as needed to support pagination for the REST source. 
// Please see the documentation for "Loading paged data."

NEXT startAt;

需要了解如何根据我的API参数设置它，即offset和hasmore属性。我如何遍历所有vidoffset值，以便我可以获取所有记录，直到has-more变为false？

这是我的json回复

Answer 1

请尝试递归调用，您是否需要将调用放在子例程中，而不是检查has_more，如果它再次等于True调用子例程。此外，每次使用新的vid-offset值时都必须更新Url参数。这是一个例子（测试它正在运行）：

SUB getOnePage(vOffset)

  LIB CONNECT TO [hubspot_api];

  RestConnectorMasterTable:
  SQL SELECT 
  (...)
  FROM JSON (wrap on) "root" PK "__KEY_root"
  WITH CONNECTION (Url "https://api.hubapi.com/contacts/v1/lists/all/contacts/all/?hapikey=YOURKEY=$(vOffset)");

  LET vHasMore = Peek('has-more',-1);
  LET vVidOffset = Peek('vid-offset',-1);

  DROP Table root;

  IF vHasMore = 'True' then

    CALL getOnePage($(vVidOffset));

  EndIf

EndSub

由于每个CALL中都有重复键，我们需要更改REST连接器中的设置，如下所示：