1.问题分析
智能刊登慢sql top 10, 大量的这种慢sql
2.解决方案
2.1分析发现sql有使用索引,导致慢的原因是并发高和表本身数据比较多,索引不是最优的,从业务上考虑数据逻辑闭环,将同步成功status=1的数据,删除或归档处理
select count(0) from M_Mercadolibre_Listing_ProductId -- 82292489
select count(0) from M_Mercadolibre_Listing_ProductId where CustomerID=1 and `Status`=1 -- 20928532 确认业务:删除或归档处理
2.2 看这个业务应该是每次批量获取50条数据,进行同步:
优化:可以去掉排序和Limit 0,50 改为ID查询(`Id` > 557424057 and `Id` < 557424057 + 50)也是批量获取数据,默认是ID升序的,优化后速度会非常快,从100秒+到 3秒,因ID连续但这个查询条件查询出的ID是递增但是不连续,有可能=50查不出数据,建议+ 100000(`Id` > 557424057 and `Id` < 557424057 + 100000),这个查询条件一共只有14632条数据,也可以一次查询出来,具体业务根据实际情况定,这个方案让sql走了最优索引查询的快,做一个指引 如:
SELECT
`CustomerId`,
`OrderSourceId`,
`ProductId`,
`Status`,
`FailureLog`,
`Date`,
`Time`,
`SiteStatus`,
`SiteProductId`,
`GlobalStatus`,
`LastGlobalFinishedTime`,
`MarketStatus`,
`LastMarketFinishedTime`,
`Remark`,
`LastModifierId`,
`LastModifier`,
`LastModificationTime`,
`CreatorId`,
`Creator`,
`CreationTime`,
`Id`
FROM
`M_Mercadolibre_Listing_ProductId`
WHERE
((((((
`Id` > 557424057 and `Id` < 557424057 + 50
)
AND ( `CustomerId` = 1 ))
AND ( `OrderSourceId` = 11744 ))
AND ( `SiteStatus` = 'success' ))
AND ( `GlobalStatus` = 0 ))
AND ( `Time` < TIMESTAMP ( '2023-04-07 08:39:32.546924' ) ))
3.sql
SELECT
`CustomerId`,
`OrderSourceId`,
`ProductId`,
`Status`,
`FailureLog`,
`Date`,
`Time`,
`SiteStatus`,
`SiteProductId`,
`GlobalStatus`,
`LastGlobalFinishedTime`,
`MarketStatus`,
`LastMarketFinishedTime`,
`Remark`,
`LastModifierId`,
`LastModifier`,
`LastModificationTime`,
`CreatorId`,
`Creator`,
`CreationTime`,
`Id`
FROM
`M_Mercadolibre_Listing_ProductId`
WHERE
((((((
`Id` > 557424057
)
AND ( `CustomerId` = 1 ))
AND ( `OrderSourceId` = 11744 ))
AND ( `SiteStatus` = 'success' ))
AND ( `GlobalStatus` = 0 ))
AND ( `Time` < TIMESTAMP ( '2023-04-07 08:39:32.546924' ) ))
ORDER BY
Id ASC
LIMIT 0,
50