使用shell保留CSV格式时删除不必要的字符

问题描述 投票:-1回答:1

我有一个以下格式的CSV文件

902610747280285697, possible future hurricaneirma analog 1995\xC2\xA02003\xC2\xA02004\xC2\xA02008 2010 til east leeward doubtful afterward invest93l
902611695239094277, midlevel ridge push invest93l future hurricaneirma wsw ridge east leave som
902642953373577216, midlevel ridge push invest93l future hurricaneirma wsw ridge east leave som
902711459561525248, midlevel ridge push invest93l future hurricaneirma wsw ridge east leave som
902755305158782976, 12z ecmwf setup support major strike east coast high east deep uul west hurricaneirma
902772740507275265, possible future hurricaneirma analog 1995\xC2\xA02003\xC2\xA02004\xC2\xA02008 2010 til east leeward doubtful
902777486186086400, future hurricaneirma satellite look impressive tropicaldepression10 24 hour
903355611810852867, hurricaneirma think f ***
903355689455804416, hurricaneirma tropics weather
903411347337162752, hurricaneirma shiiiiiiitty t *** im possibly wrong
903411365607591936, hurricaneirma 3000 mile cat 3 hurricane watch closely
903989185845088257, 

我如何删除像*,\ xC2 \ xA02003 \ xC2 \ xA02004 \ xC2 \ xA0这样的字符和空行,就像以后可能在Scala处理中抛出错误的空行一样。我需要保持CSV结构相同以前的方式,但要求删除这些。

请帮我在shell脚本中实现这个目的?再次感谢你,因为我是shell脚本的新手

编辑:

你能否告诉我如何纠正损坏的行(没有',')之类的

902755305158782976, 12z ecmwf setup support major strike east coast high east deep uul west hurricaneirma
902777486186086400, future hurricaneirma satellite look impressive tropicaldepression10 24 hour
903355611810852867 hurricaneirma think
903355611810852868 hurricagggneirma think
shell csv awk sed tr
1个回答
2
投票

您可以使用sed,但我很确定,您可能无法获得100%的结果。您应该使用您正在处理的文件的本机工具来获得所需格式的结果。无论如何以下是我的尝试:

$ sed -E '/^[^,]*$/d;/^[0-9]+, *$/d;s/ \*+ */ /;s/\\[xX][^\ ,]*//g' case_file_48246326

产量

902610747280285697, possible future hurricaneirma analog 1995 2010 til east leeward doubtful afterward invest93l
902611695239094277, midlevel ridge push invest93l future hurricaneirma wsw ridge east leave som
902642953373577216, midlevel ridge push invest93l future hurricaneirma wsw ridge east leave som
902711459561525248, midlevel ridge push invest93l future hurricaneirma wsw ridge east leave som
902755305158782976, 12z ecmwf setup support major strike east coast high east deep uul west hurricaneirma
902772740507275265, possible future hurricaneirma analog 1995 2010 til east leeward doubtful
902777486186086400, future hurricaneirma satellite look impressive tropicaldepression10 24 hour
903355611810852867, hurricaneirma think f 
903355689455804416, hurricaneirma tropics weather
903411347337162752, hurricaneirma shiiiiiiitty t im possibly wrong
903411365607591936, hurricaneirma 3000 mile cat 3 hurricane watch closely
© www.soinside.com 2019 - 2024. All rights reserved.