点燃缓存/查询花费太多时间

问题描述 投票：1回答：2

我试图通过实现内存中的数据网格来采用点燃来解决我的需求之一

正确地知道，我使用第三个持久性读/写机制从我的oracle数据库中获取数据，在拓扑结构中，我使用的是激活群集中的单个节点，该节点托管在具有8G内存和120 G硬盘的VM中。

我的节点使用的本地存储具有2G的堆上内存和50G的堆外内存，所有这些都启用逐出和交换功能

DataStorageConfiguration dataStorageCfg = new DataStorageConfiguration();

DataRegionConfiguration dataRegionCfg = new DataRegionConfiguration();



// 2G initial size (RAM).
dataRegionCfg.setInitialSize(2L * 1024 * 1024 * 1024);

// 40 GB max size (RAM).
dataRegionCfg.setMaxSize(40L * 1024 * 1024 * 1024);

// Enabling RANDOM_LRU eviction for this region.
dataRegionCfg.setPageEvictionMode(DataPageEvictionMode.RANDOM_LRU);

//dataRegionCfg.setPersistenceEnabled(true);
final String swapPath ="/opt/ignite/swap";

dataRegionCfg.setSwapPath(swapPath);

dataStorageCfg.setDefaultDataRegionConfiguration(dataRegionCfg);

cfg.setDataStorageConfiguration(dataStorageCfg);

在我的计算机上缓存此文件会花费太多时间

当缓存结束时，我的交换文件夹约为13G关于我的SQL查询没有响应

我的工具中的同一查询需要1分45秒来响应，但是使用ignite缓存查询方法不会响应，并且不会引发任何类型的错误或异常

SqlFieldsQuery sqlQuery;
FieldsQueryCursor<List<?>> queryCursor;
Iterator<List<?>> resultIt;

System.out.println(">>> All caches loaded! in : " + total + " ms");

System.out.println("---------------------------------------------- ");
System.out.println("---------------------------------------------- ");
System.out.println("---------------------------------------------- ");

System.out.println("\\n \\n \\n ");
System.out.println("---------------------------------------------- ");
System.out.println("---------------------------------------------- ");
System.out.println("---------------------------------------------- ");

System.out.println("Checking join query POC first run");

start = System.currentTimeMillis();
sqlQuery = new SqlFieldsQuery(sql);
queryCursor = ignite.cache("MInoutlineCache").query(sqlQuery);
System.out.println("query result size is : "+queryCursor.getAll().size());

end = System.currentTimeMillis() - start;
total += end;

我很好用点火器吗？以一个节点的方式使用ignite有用吗，或者我打算以分区策略构建具有很多节点的集群？

缓存的行数为1000万行还有没有其他方法可以使用第三种持久性策略在我的上下文中实现良好的内存数据网格。

抱歉，这个问题中有很多问题

NB：我正在使用网格增益控制台生成我的配置我也将缓存模式名称更新为公共，以直接执行查询]

这里是查询

 SELECT bp.name,
    CF.documentno,
    CF.MOVEMENTDATE,
    CF.m_product_id AS M_PRODUCT_ID,
    CF.product,
    CF.xx_lignegratuite,
    CF.m_attributesetinstance_id,
    CASE
      WHEN cf.isreturntrx='Y'
      THEN - CF.qtyentered
      ELSE CF.qtyentered
    END AS qtyentered,
    CF.discount,
    CF.DOCSTATUS,
    CF.ISRETURNTRX,
    CF.XX_REWARDAMT,
    CF.OperID,
    CF.clientId,
    CASE
      WHEN cf.xx_lignegratuite='N'
      THEN
        CASE
          WHEN cf.isreturntrx='Y'
          THEN             -cf.prixVente*cf.qtyentered* (1-(cf.discount/100))
          ELSE cf.prixVente*cf.qtyentered* (1-(cf.discount/100))
        END
      ELSE 0
    END AS totalline,
    CASE
      WHEN cf.XX_StartegicalProduct='Y'
      THEN (
        CASE
          WHEN cf.xx_lignegratuite='N'
          THEN
            CASE
              WHEN cf.isreturntrx='Y'
              THEN             -cf.prixVente*cf.qtyentered* (1-(cf.discount/100))
              ELSE cf.prixVente*cf.qtyentered* (1-(cf.discount/100))
            END
          ELSE 0
        END)
      ELSE 0
    END AS totallineStar,
    CASE
      WHEN cf.xx_lignegratuite='N'
      THEN
        CASE
          WHEN cf.typevente='W'
          THEN
            CASE
              WHEN cf.isreturntrx='Y'
              THEN                -(cf.XX_REWARDAMT/nb_doc)
              ELSE cf.XX_REWARDAMT/nb_doc
            END
          ELSE 0
        END
      ELSE 0
    END AS totalreward,
    CF.XX_StartegicalProduct,
    CF.SALESREP_ID,
    CF.C_DOCTYPE_ID,
    CF.AD_ORG_ID,
    CF.ad_orgtrx_id,
    CF.xx_laboratory_id,
    bp.c_bpartner_id,
    CF.nb_doc,
    CF.rate,
    CF.poste_id,
    CF.SalesRepTier_poste_id,
    CF.recSupr,
    CF.recSupr_poste_id,
    (SELECT Objectif_CA_oper
    FROM c_bpartner
    WHERE issalesrep ='Y'
    AND isemployee   ='Y'
    AND c_bpartner_id=CF.SalesRepTier
    ) AS ObjectifOp,
    (SELECT Objectif_CA_oper
    FROM c_bpartner
    WHERE issalesrep ='Y'
    AND isemployee   ='Y'
    AND c_bpartner_id=CF.SalesRepTier_poste_id
    ) AS ObjectifOp_poste_id,
    CASE
      WHEN cf.ISRETURNTRX='Y'
      THEN              -cf.QTYENTERED*prixRevient
      ELSE cf.QTYENTERED*prixRevient
    END AS consomation
  FROM
    (SELECT i.documentno,
      i.MOVEMENTDATE,
      p.m_product_id,
      p.name AS product,
      ol.xx_lignegratuite,
      il.m_attributesetinstance_id,
      il.qtyentered,
      ol.discount,
      i.DOCSTATUS,
      i.isreturntrx,
      ol.XX_REWARDAMT,
      i.C_BPartner_ID AS clientId,
      (SELECT u.C_BPARTNER_ID FROM AD_User u WHERE u.AD_User_ID = i.SALESREP_ID
      ) AS OperID,
      (SELECT ai.Valuenumber
      FROM M_AttributeInstance ai
      INNER JOIN M_Attribute a
      ON (ai.M_Attribute_ID             =a.M_Attribute_ID
      AND a.IsInstanceAttribute         ='Y')
      WHERE ai.M_AttributeSetInstance_ID=il.m_attributesetinstance_id
      AND a.Name                        ='Prix Vente'
      ) AS prixVente,
      (SELECT ai.Valuenumber
      FROM M_AttributeInstance ai
      INNER JOIN M_Attribute a
      ON (ai.M_Attribute_ID             =a.M_Attribute_ID
      AND a.IsInstanceAttribute         ='Y')
      WHERE ai.M_AttributeSetInstance_ID=il.m_attributesetinstance_id
      AND a.Name                        ='Prix Revient'
      ) AS prixRevient,
      (SELECT ai.Valuenumber
      FROM M_AttributeInstance ai
      INNER JOIN M_Attribute a
      ON (ai.M_Attribute_ID             =a.M_Attribute_ID
      AND a.IsInstanceAttribute         ='Y')
      WHERE ai.M_AttributeSetInstance_ID=il.m_attributesetinstance_id
      AND a.Name                        ='Fournisseur'
      ) AS Fournisseur,
      XX_StartegicalProduct,
      i.SALESREP_ID,
      i.C_DOCTYPE_ID,
      i.AD_ORG_ID,
      (SELECT o.AD_ORGTRX_ID
      FROM c_order o
      WHERE i.c_order_id=o.c_order_id
      ) AS ad_orgtrx_id,
      p.xx_laboratory_id,
      lt.rate,
      --COUNT(*) over (partition BY il.c_orderline_id) AS nb_doc,
      (
      SELECT COUNT(*)
      FROM m_inoutline ill
      WHERE ill.c_orderline_id=il.c_orderline_id
      )               AS nb_doc,
      ol.type         AS typevente,
      bpl.salesrep_id AS poste_id,
      (SELECT u.c_bpartner_id FROM AD_User u WHERE u.AD_User_ID = i.salesrep_id
      ) AS SalesRepTier,
      (SELECT u.c_bpartner_id FROM AD_User u WHERE u.AD_User_ID = bpl.salesrep_id
      ) AS SalesRepTier_poste_id,
      (SELECT u.XX_RecSupervisor_ID FROM AD_User u WHERE u.AD_User_ID=i.SALESREP_ID
      ) AS recSupr,
      (SELECT u.XX_RecSupervisor_ID
      FROM AD_User u
      WHERE u.AD_User_ID=bpl.salesrep_id
      ) AS recSupr_poste_id
    FROM m_inoutline il
    INNER JOIN m_inout i
    ON il.m_inout_id=i.m_inout_id
    INNER JOIN c_orderline ol
    ON ol.c_orderline_id=il.c_orderline_id
    INNER JOIN m_product p
    ON p.m_product_id=il.m_product_id
    INNER JOIN C_Bpartner bpl
    ON (bpl.c_bpartner_id=i.c_bpartner_id)
    LEFT OUTER JOIN xx_listetauxvaleur lt
    ON p.xx_listetauxvaleur_id = lt.xx_listetauxvaleur_id
    WHERE i.issotrx            ='Y'
      --AND p.m_attributeset_id   IS NOT NULL
    AND il.movementqty<>0
    ) CF
  LEFT OUTER JOIN c_bpartner bp
  ON (CF.Fournisseur=bp.c_bpartner_id)
  ORDER BY bp.name,
    documentno;
ignite gridgain
2个回答
1
投票

我在您的发言中看到很多加入。您确定在这些联接中存在的所有字段上都有索引吗？我认为在执行此操作之前，将需要进行大量优化。

我还建议您摆脱掉交换，因为这是一个过时的功能，并且其性能影响尚不清楚。

1
投票

请考虑应该回答大多数问题的一般性能提示：https://www.gridgain.com/docs/latest/perf-troubleshooting-guide/general-perf-tips

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.