PHP - 从SQL Server和MySQL同步两个相同的表并仅插入差异，但代码运行速度非常慢

Question

我当前代码的工作方式，它从第一个数据库（SQL Server）中选择所有数据，然后从MySQL中选择所有数据，然后只插入差异。问题是表中有超过70k +行和照片文件，它运行速度非常慢并且消耗大量内存。以下是其中一个较小的表的示例代码：

$mssql_array = array();
$ms_sql = "SELECT * FROM SALARY";
$mssql_query = sqlsrv_query($conn_mssql , $ms_sql);

while($row = sqlsrv_fetch_array($mssql_query))
{

  $mssql_array[] = array('SGRADE' => $row['SGRADE'], 'STEP' => $row['STEP'], 'AMOUNT' => $row['AMOUNT'], 'REMARK' => $row['REMARK'], 'unique' => $row['unique']);

}

foreach($mssql_array as $key => $value)
{

   //SELECT FROM MySQL DB
   $my_sql = " SELECT * FROM SALARY WHERE SGRADE ='".$value['SGRADE']."' AND STEP = '".$value['STEP']."' AND AMOUNT = '".$value['AMOUNT']."' AND REMARK = '".$value['REMARK']."' AND `unique` = '".$value['unique']."' ";
   $mysql_query = mysqli_query($conn_mysql , $my_sql);
   $num_rows = mysqli_num_rows($mysql_query);

   $sql = $conn_mysql->prepare("INSERT INTO salary VALUES (?, ?, ?, ?, ?)"); 
   $sql->bind_param("sidsi", $SGRADE, $STEP, $AMOUNT, $REMARK, $unique);
   $SGRADE = $value['SGRADE'];
   $STEP = $value['STEP'];
   $AMOUNT = $value['AMOUNT'];
   $REMARK = $value['REMARK'];
   $unique = $value['unique'];

   if ($num_rows == 0)
   {
     //INSERT INTO MySQL DB
     $sql->execute();
   }
}

虽然它对小型表的工作速度非常快（小于5秒），但对于大型表，它需要大约2-3个小时（300k +行，60列）。我有一些想法可以使这段代码更快，这是在SQL Server表的主键列上执行select max（*）查询（上表中的“unique”），只取最后一行的“唯一”号并将其与MySQL的最后一行“独特”进行比较。如果SQL Server的“唯一”列值高于MySQL，则会在两个表之间插​​入所有不匹配的记录。事实是，我不知道如何将其转换为代码，或者甚至是否可能。与我上面使用的相比，我可以使用的任何想法或替代品？

Answer 1

所以我终于找到了答案，这是我制作的新代码：

$mssql_result = sqlsrv_query($conn_mssql, "SELECT MAX([unique]) FROM salary");
$mssql_row = sqlsrv_fetch_array($mssql_result);
echo $mssql_row[0];

$mysql_result = mysqli_query($conn_mysql, "SELECT MAX(`unique`) FROM salary");
$mysql_row = mysqli_fetch_array($mysql_result);
echo $mysql_row[0];

if($mssql_row[0] != $mysql_row[0])
{
  //SELECT FROM SQL SERVER DB
  $mssql_array = array();
  $ms_sql = "SELECT * FROM SALARY where [unique] > $mysql_row[0]";
  $mssql_query = sqlsrv_query($conn_mssql , $ms_sql);

  while($row = sqlsrv_fetch_array($mssql_query))
  {
    $mssql_array[] = array('SGRADE' => $row['SGRADE'], 'STEP' => $row['STEP'], 'AMOUNT' => $row['AMOUNT'], 'REMARK' => $row['REMARK'], 'unique' => $row['unique']);
  }

  foreach($mssql_array as $key => $value)
  {
    $sql = $conn_mysql->prepare("INSERT INTO salary VALUES (?, ?, ?, ?, ?)"); 
    $sql->bind_param("sidsi", $SGRADE, $STEP, $AMOUNT, $REMARK, $unique);
    $SGRADE = $value['SGRADE'];
    $STEP = $value['STEP'];
    $AMOUNT = $value['AMOUNT'];
    $REMARK = $value['REMARK'];
    $unique = $value['unique'];
    $sql->execute();
  }
}
else
{
  echo "No new records to add!";
}

现在它不再仅仅选择所有内容来检查并仅根据“唯一”列比较最大行值。跑得比以前快得多。

PS：不知道为什么我会被投票