在我们的项目中,我们使用bcp命令导出大约数百万行并将输出记录到输出文件中。对于bcp导入,我可以使用指定no的bcp
开关来控制-b
命令的输出。批量导入的行数。输出是这样的:
Starting copy...
1000 rows sent to SQL Server. Total sent: 1000
1000 rows sent to SQL Server. Total sent: 2000
1000 rows sent to SQL Server. Total sent: 3000
1000 rows sent to SQL Server. Total sent: 4000
1000 rows sent to SQL Server. Total sent: 5000
1000 rows sent to SQL Server. Total sent: 6000
1000 rows sent to SQL Server. Total sent: 7000
1000 rows sent to SQL Server. Total sent: 8000
1000 rows sent to SQL Server. Total sent: 9000
1000 rows sent to SQL Server. Total sent: 10000
1000 rows sent to SQL Server. Total sent: 11000
1000 rows sent to SQL Server. Total sent: 12000
SQLState = 22001, NativeError = 0
可以通过增加-b
开关发送的数量轻松减少:
Starting copy...
10000 rows sent to SQL Server. Total sent: 10000
SQLState = 22001, NativeError = 0
12406 rows copied.
Network packet size (bytes): 4096
Clock Time (ms.) Total : 75 Average : (165413.3 rows per sec.)
但是对于bcp导出,我无法控制输出,对于一百万行,日志变得太大,例如。以下命令
bcp Temp.dbo.TestTable out outdata.txt -t , -f file.fmt -S Server -U user-P password -m 10
输出:
Starting copy...
1000 rows successfully bulk-copied to host-file. Total received: 1000
SQLState = S1000, NativeError = 0
Error = [Microsoft][ODBC Driver 11 for SQL Server]Warning: BCP import with a format file will convert empty strings in delimited columns to NULL.
1000 rows successfully bulk-copied to host-file. Total received: 2000
1000 rows successfully bulk-copied to host-file. Total received: 3000
1000 rows successfully bulk-copied to host-file. Total received: 4000
1000 rows successfully bulk-copied to host-file. Total received: 5000
1000 rows successfully bulk-copied to host-file. Total received: 6000
1000 rows successfully bulk-copied to host-file. Total received: 7000
1000 rows successfully bulk-copied to host-file. Total received: 8000
1000 rows successfully bulk-copied to host-file. Total received: 9000
1000 rows successfully bulk-copied to host-file. Total received: 10000
1000 rows successfully bulk-copied to host-file. Total received: 11000
1000 rows successfully bulk-copied to host-file. Total received: 12000
1000 rows successfully bulk-copied to host-file. Total received: 13000
1000 rows successfully bulk-copied to host-file. Total received: 14000
1000 rows successfully bulk-copied to host-file. Total received: 15000
1000 rows successfully bulk-copied to host-file. Total received: 16000
1000 rows successfully bulk-copied to host-file. Total received: 17000
1000 rows successfully bulk-copied to host-file. Total received: 18000
1000 rows successfully bulk-copied to host-file. Total received: 19000
1000 rows successfully bulk-copied to host-file. Total received: 20000
1000 rows successfully bulk-copied to host-file. Total received: 21000
1000 rows successfully bulk-copied to host-file. Total received: 22000
我试过用-b
传递bcp out
开关,但它总是以1000的批量导出它们并通过greping
或seding
过滤它们将花费太多时间。谢谢你的帮助。
在bcp中似乎没有解决方案。但是,有一种解决方法;将bcp命令行打包到xp_cmdshell语句中并指定no_output选项:
EXEC xp_cmdshell "bcp Temp.dbo.TestTable out outdata.txt -t , -f file.fmt -S Server -U user-P password -m 10", no_output
资料来源:click here
bcp sometable out outfile -S Server -U user -P password > export.log
注意最后的> export.log位。这将使用log填充export.log。因此,如果您的命令失败,您可以检查。有关此方法的更多详细信息,请参阅here。bcp sometable out outfile -S Server -U user -P password -o export.log
这次注意到-o export.log到底了。阻止命令行输出:
bcp Temp.dbo.TestTable out outdata.txt -t , -f file.fmt -S Server -U user-P password -m 10>nul
我知道这是旧的,但我偶然发现它寻找相同的答案,并看到这里没有好的答案。我想出了以下内容,它让我得到了我想要的东西,并且认为我会在这里发布,以防它帮助其他人:
set @BCP_CMD = 'bcp ...etc...'; -- declare and set this appropriately
DROP TABLE IF EXISTS #bcpOutput;
CREATE TABLE #bcpOutput ([BCP_Output] varchar(2048));
Insert into #bcpOutput ([BCP_Output]) EXECUTE master..xp_cmdshell @BCP_CMD;
SELECT BCP_Output FROM #bcpOutput where
-- Here I filter out Blank lines, all those "rows successfully..." lines, etc.
BCP_Output not like ''
AND BCP_Output not like '%rows successfully%'
AND BCP_Output not like '%S1000%'
AND BCP_Output not like '%empty strings in delimited%'
AND BCP_Output not like '%Starting copy%'
AND BCP_Output not like '%Network packet size%' ;
这会产生:
BCP_Output
69673 rows copied.
Clock Time (ms.) Total : 406 Average : (171608.38 rows per sec.)
另一种方法在技术上“工作”就好了:追加
| grep -v -e "rows successfully\|Starting copy\|...and so on..."
到你的“bcp ...”命令。不过,对我来说,它将<5秒的bcp操作变成了近30秒,这是不可接受的。也许在您的环境中,grep的管道效果更好。也许值得尝试一下。