任务描述

1.问题分析


目前是华为日志慢sql里的top 2,查询次数多,查询时间慢

 in 子查询元素过多,导致的查询慢


2.解决方案

 将In 字查询改为临时表inner join 查询会快很多



3.sql

SELECT `Id`,`UniqueId`,`CustomerId`,`MarketId`,`StrategyId`,`OrderSourceId`,`Type`,`ItemId`,`Sku`,`AllSkus`,`ClientSku`,`ParentSku`,`SellerSku`,`SiteCode`,`Price`,`IsAgainPublish`,`Status`,`ErrorCode`,`FailureReason`,`Date`,`Time`,`OperationType`,`Upc`,`PublishType`,`ExternalId`,`TaskStatus`,`LoadQueueTime`,`ParentOrderSourceId`,`BargainingPrice`,`IsManual`,`RuleId`,`LastModifierId`,`LastModifier`,`LastModificationTime`,`CreatorId`,`Creator`,`CreationTime` FROM `Ap_AutoPublish_Allegro_Queue`  WHERE (((( `CustomerId` = 1 ) AND ( `MarketId` = 67 )) AND  (`Sku` IN ('9920012','9918183','10029336','9912752','9911500','10028384','9907974','11074536','9987284','13040613','13040478','13028743','10075588','9984379','7462627','10072619','4280040','8234101','9836193','13018805','9829075','9800641','10500210','13018657','13244339','9799991','10468266','10162472','13341154','11340564','10453833','11329577','10161275','7199141','13283016','11310371','13265955','7120784','10453401','8796133','10160017','13262975','11307095','7179686','11557875','13262267','11296448','10453144','10452666','13177781','13177763','13177721','7679112','10802123','13168780','10767351','13168740','10755733','11746098','13155146','13154478','11707356','13153829','11694600','10299080','13149819','12303509','12373089','5941442','12239494','12231646','5751059','12238960','13015248','10705947','10728853','10710706','13089813','10709587','8634499','12922776','12317634','12910405','10708531','12901421','12865184','13564835','10707669','12901500','10707096','10703108','9793913','12863141','9789512','10702345','12863108','7414673','10450454','7407793','10344839')) ) AND NOT( `Upc` IS NULL   OR `Upc`=''));

历史记录  
  1. 2023-04-23 09:28:52, 由 陈少军 创建。
  2. 2023-04-23 09:31:32, 由 陈少军 编辑。
    修改了 任务名称,区别为:
    001- 【智能刊登慢sql】M_ShopeeV2_Listing
    001+ 【智能刊登慢sql】Ap_AutoPublish_Allegro_Queue
    001- 【智能刊登慢sql】M_ShopeeV2_Listing
    001+ 【智能刊登慢sql】Ap_AutoPublish_Allegro_Queue
    修改了 任务描述,区别为:
    003- 目前是华为日志慢sql里的top 1,查询次数多,查询时间慢
    003+ 目前是华为日志慢sql里的top 2,查询次数多,查询时间慢
    004- Limit 偏移量起始值大查询慢,可以优化为 id >256400 速度会快很多,参考解决方案优化
    004+ in 子查询元素过多,导致的查询慢
    007- FROM `M_ShopeeV2_Listing` WHERE (((( `CustomerId` = 1 ) AND ( `IsGlobal` = 1 )) AND ( 'CN' = `OrderSourceCountry` )) AND ( timestamp('2023-03-24 09:16:07.931600') > `LastModificationTime` ))
    007+ 将In 字查询改为临时表inner join 查询会快很多
    008- and Id >256400ORDER BY Id Asc LIMIT 0,200;
    008+
    009-
    009+
    010-
    010+ 3.sql
    011- 3.sql
    011+ SELECT `Id`,`UniqueId`,`CustomerId`,`MarketId`,`StrategyId`,`OrderSourceId`,`Type`,`ItemId`,`Sku`,`AllSkus`,`ClientSku`,`ParentSku`,`SellerSku`,`SiteCode`,`Price`,`IsAgainPublish`,`Status`,`ErrorCode`,`FailureReason`,`Date`,`Time`,`OperationType`,`Upc`,`PublishType`,`ExternalId`,`TaskStatus`,`LoadQueueTime`,`ParentOrderSourceId`,`BargainingPrice`,`IsManual`,`RuleId`,`LastModifierId`,`LastModifier`,`LastModificationTime`,`CreatorId`,`Creator`,`CreationTime` FROM `Ap_AutoPublish_Allegro_Queue` WHERE (((( `CustomerId` = 1 ) AND ( `MarketId` = 67 )) AND (`Sku` IN ('9920012','9918183','10029336','9912752','9911500','10028384','9907974','11074536','9987284','13040613','13040478','13028743','10075588','9984379','7462627','10072619','4280040','8234101','9836193','13018805','9829075','9800641','10500210','13018657','13244339','9799991','10468266','10162472','13341154','11340564','10453833','11329577','10161275','7199141','13283016','11310371','13265955','7120784','10453401','8796133','10160017','13262975','11307095','7179686','11557875','13262267','11296448','10453144','10452666','13177781','13177763','13177721','7679112','10802123','13168780','10767351','13168740','10755733','11746098','13155146','13154478','11707356','13153829','11694600','10299080','13149819','12303509','12373089','5941442','12239494','12231646','5751059','12238960','13015248','10705947','10728853','10710706','13089813','10709587','8634499','12922776','12317634','12910405','10708531','12901421','12865184','13564835','10707669','12901500','10707096','10703108','9793913','12863141','9789512','10702345','12863108','7414673','10450454','7407793','10344839')) ) AND NOT( `Upc` IS NULL OR `Upc`=''));
    012- SELECT `Id`,`CustomerId`,`OrderSourceId`,`OrderSourceCountry`,`IsGlobal`,`PartnerId`,`MerchantId`,`ShopId`,`SiteCode`,`Type`,`Sku`,`CustomSku`,`ProductId`,`ParentSku`,`SellerSku`,`Name`,`MainImage`,`CategoryId`,`HasVariation`,`Currency`,`Price`,`OriginalPrice`,`InflatedPrice`,`InflatedOriginalPrice`,`StockLocationId`,`StockType`,`Inventory`,`CurrentStock`,`ReservedStock`,`StockLocationId1`,`Inventory1`,`CurrentStock1`,`ReservedStock1`,`Weight`,`Length`,`Width`,`Height`,`IsPreOrder`,`ShippingTime`,`Condition`,`SizeChart`,`CreateTime`,`UpdateTime`,`Status`,`Sales`,`Views`,`Likes`,`RatingStar`,`Comments`,`PromotionId`,`PublishTime`,`UnPublishTime`,`AutoPublishStrategyId`,`AutoPublishTime`,`IsAutoPublish`,`RePublishStrategyId`,`RePublishTime`,`AutoUnPublishStrategyId`,`AutoUnPublishTime`,`AutoAdjustPriceStrategyId`,`AutoAdjustPriceTime`,`AutoPromotionStrategyId`,`AutoPromotionTime`,`LastPhoenixPlanStrategyType`,`EditStatus`,`LastModifierId`,`LastModifier`,`LastModificationTime`,`CreatorId`,`Creator`,`CreationTime`FROM `M_ShopeeV2_Listing` WHERE (((( `CustomerId` = 1 ) AND ( `IsGlobal` = 1 )) AND ( 'CN' = `OrderSourceCountry` )) AND ( timestamp('2023-03-24 09:16:07.931600') > `LastModificationTime` ))ORDER BY Id Asc LIMIT 256400,200;
    003- <p>目前是华为日志慢sql里的top 1,查询次数多,查询时间慢</p>
    003+ <p>目前是华为日志慢sql里的top 2,查询次数多,查询时间慢</p>
    004- <p>Limit 偏移量起始值大查询慢,可以优化为 id &gt;<span>256400 速度会快很多,参考解决方案优化</span></p>
    004+ <p>in 子查询元素过多,导致的查询慢</p>
    007- <p>FROM `M_ShopeeV2_Listing` <br />WHERE (((( `CustomerId` = 1 ) AND ( `IsGlobal` = 1 )) AND ( 'CN' = `OrderSourceCountry` )) AND ( timestamp('2023-03-24 09:16:07.931600') &gt; `LastModificationTime` ))</p>
    007+ <p>将In 字查询改为临时表inner join 查询会快很多</p>
    008- <p>and Id &gt;<span>256400</span><br />ORDER BY Id Asc LIMIT 0,200;<br /></p><div><br /></div>
    008+ <div><br /></div>
    009-
    009+ <p><br /></p>
    010- <p><br /></p>
    010+ <p>3.sql</p>
    011- <p>3.sql</p>
    011+ <p>SELECT `Id`,`UniqueId`,`CustomerId`,`MarketId`,`StrategyId`,`OrderSourceId`,`Type`,`ItemId`,`Sku`,`AllSkus`,`ClientSku`,`ParentSku`,`SellerSku`,`SiteCode`,`Price`,`IsAgainPublish`,`Status`,`ErrorCode`,`FailureReason`,`Date`,`Time`,`OperationType`,`Upc`,`PublishType`,`ExternalId`,`TaskStatus`,`LoadQueueTime`,`ParentOrderSourceId`,`BargainingPrice`,`IsManual`,`RuleId`,`LastModifierId`,`LastModifier`,`LastModificationTime`,`CreatorId`,`Creator`,`CreationTime` FROM `Ap_AutoPublish_Allegro_Queue` WHERE (((( `CustomerId` = 1 ) AND ( `MarketId` = 67 )) AND (`Sku` IN ('9920012','9918183','10029336','9912752','9911500','10028384','9907974','11074536','9987284','13040613','13040478','13028743','10075588','9984379','7462627','10072619','4280040','8234101','9836193','13018805','9829075','9800641','10500210','13018657','13244339','9799991','10468266','10162472','13341154','11340564','10453833','11329577','10161275','7199141','13283016','11310371','13265955','7120784','10453401','8796133','10160017','13262975','11307095','7179686','11557875','13262267','11296448','10453144','10452666','13177781','13177763','13177721','7679112','10802123','13168780','10767351','13168740','10755733','11746098','13155146','13154478','11707356','13153829','11694600','10299080','13149819','12303509','12373089','5941442','12239494','12231646','5751059','12238960','13015248','10705947','10728853','10710706','13089813','10709587','8634499','12922776','12317634','12910405','10708531','12901421','12865184','13564835','10707669','12901500','10707096','10703108','9793913','12863141','9789512','10702345','12863108','7414673','10450454','7407793','10344839')) ) AND NOT( `Upc` IS NULL OR `Upc`=''));</p>
    012- <p>SELECT `Id`,`CustomerId`,`OrderSourceId`,`OrderSourceCountry`,`IsGlobal`,`PartnerId`,`MerchantId`,`ShopId`,`SiteCode`,`Type`,`Sku`,`CustomSku`,`ProductId`,`ParentSku`,`SellerSku`,`Name`,`MainImage`,`CategoryId`,`HasVariation`,`Currency`,`Price`,`OriginalPrice`,`InflatedPrice`,`InflatedOriginalPrice`,`StockLocationId`,`StockType`,`Inventory`,`CurrentStock`,`ReservedStock`,`StockLocationId1`,`Inventory1`,`CurrentStock1`,`ReservedStock1`,`Weight`,`Length`,`Width`,`Height`,`IsPreOrder`,`ShippingTime`,`Condition`,`SizeChart`,`CreateTime`,`UpdateTime`,`Status`,`Sales`,`Views`,`Likes`,`RatingStar`,`Comments`,`PromotionId`,`PublishTime`,`UnPublishTime`,`AutoPublishStrategyId`,`AutoPublishTime`,`IsAutoPublish`,`RePublishStrategyId`,`RePublishTime`,`AutoUnPublishStrategyId`,`AutoUnPublishTime`,`AutoAdjustPriceStrategyId`,`AutoAdjustPriceTime`,`AutoPromotionStrategyId`,`AutoPromotionTime`,`LastPhoenixPlanStrategyType`,`EditStatus`,`LastModifierId`,`LastModifier`,`LastModificationTime`,`CreatorId`,`Creator`,`CreationTime`<br />FROM `M_ShopeeV2_Listing` <br />WHERE (((( `CustomerId` = 1 ) AND ( `IsGlobal` = 1 )) AND ( 'CN' = `OrderSourceCountry` )) AND ( timestamp('2023-03-24 09:16:07.931600') &gt; `LastModificationTime` ))<br />ORDER BY Id Asc LIMIT 256400,200;<br /></p>
  3. 2023-04-23 09:53:08, 由 程伟 指派给 韦思成。
    修改了 指派给,旧值为 "chengwei",新值为 "weisicheng"。
  4. 2023-04-24 10:24:35, 由 陈少军 添加备注。

    这个表数据很多,开发找产品确认是否可以归档2023年前的数据,统计如下:可以归档约35000万


    select count(0) from Ap_AutoPublish_Allegro_Queue where Date >20230101   -- 22656892

    select count(0) from Ap_AutoPublish_Allegro_Queue -- 5700000

  5. 2023-04-24 10:26:10, 由 陈少军 添加备注。

    统计更正

    select count(0) from Ap_AutoPublish_Allegro_Queue where Date >20230101   -- 22656892  


    select count(0) from Ap_AutoPublish_Allegro_Queue -- 57000000
  6. 2023-04-24 11:30:25, 由 韦思成 指派给 韦思成。

    先清除2023-03-01以前 失败的记录,约860万+

  7. 2023-04-24 14:57:32, 由 韦思成 完成。
    修改了 总计消耗,旧值为 "0",新值为 "1"。
    修改了 指派给 ,旧值为 "weisicheng",新值为 "chenshaojun"。
    修改了 实际完成,旧值为 "",新值为 "2023-04-24 14:57:04"。
    修改了 任务状态,旧值为 "wait",新值为 "done"。
    修改了 实际开始,旧值为 "0000-00-00 00:00:00",新值为 "2023-04-24 14:57:22"。
    修改了 预计剩余,旧值为 "1",新值为 "0"。
    修改了 由谁完成,旧值为 "",新值为 "weisicheng"。

    已清理完成

  8. 2023-10-31 14:53:16, 由 周杰 关闭。
    修改了 任务状态,旧值为 "done",新值为 "closed"。
    修改了 指派给 ,旧值为 "chenshaojun",新值为 "closed"。
    修改了 由谁关闭,旧值为 "",新值为 "zhoujie"。
    修改了 关闭时间,旧值为 "0000-00-00 00:00:00",新值为 "2023-10-31 14:53:16"。
    修改了 关闭原因,旧值为 "",新值为 "done"。
所属迭代 系统管理部 - 系统服务优化
所属模块 /
相关需求 暂无
指派给 Closed 于 2023-10-31 14:53:16
任务类型 后端
任务标签 初始
开发组 有品开发组
任务状态 已关闭
进度 100%
优先级 P2
关键词
抄送给 许瑞丰
由谁创建 陈少军 于 2023-04-23 09:28:52
由谁完成 韦思成 于 2023-04-24 14:57:04
由谁取消 暂无
由谁关闭 周杰 于 2023-10-31 14:53:16
关闭原因 已完成
最后编辑 周杰 于 2023-10-31 14:53:16
团队 最初预计 总计消耗 预计剩余
工时信息
最初预计 1工时
总计消耗 1工时
预计剩余 0工时
预计开始 2023-04-23
实际开始 2023-04-24
截止日期 2023-04-23