我有一个包含超过400K文本文件的文件夹。
名字就像
deID.RESUL_12433287659.txt_234323456.txt
deID.RESUL_34534563649.txt_345353567.txt
deID.RESUL_44235345636.txt_537967875.txt
deID.RESUL_35234663456.txt_423452545.txt
每个文件都有不同的内容
我想获取文件名和文件内容并放入CSV。
就像是:
file_name,file_content
deID.RESUL_12433287659.txt_234323456.txt,Content 1
deID.RESUL_34534563649.txt_345353567.txt,Content 2
deID.RESUL_44235345636.txt_537967875.txt,Content 3
deID.RESUL_35234663456.txt_423452545.txt,Content 4
我知道如何使用以下方法获取CSV目录中的所有文件:
find * > files.csv
我怎样才能抓取文件的内容?
csv
顾名思义是逗号分隔文件。你提出了一张桌子。find *
有些奇怪,find
已经递归扫描。 find .
足以包括所有find *
(好吧,除非你考虑到有些奇怪的shell glob规则)。# create file for a MCVE
while IFS=' ' read -r file content; do echo "$content" > "$file"; done <<EOF
deID.RESUL_12433287659.txt_234323456.txt Content 1
deID.RESUL_34534563649.txt_345353567.txt Content 2
deID.RESUL_44235345636.txt_537967875.txt Content 3
deID.RESUL_35234663456.txt_423452545.txt Content 4
EOF
{
# I'm using `|` as the separator for columns
# output header names
echo 'file_name|file_content';
# this is the hearth of the script
# find the files
# for each file execute `sh -c 'printf "%s|%s\n" "$1" "$(<"$1")"' -- <filename>`
# printf - nice printing
# "$(<"$1")" - gets file content and also removes trailing empty newlines. Neat.
find . -type f -name 'deID.*' -exec sh -c 'printf "%s|%s\n" "$1" "$(<"$1")"' -- {} \;
} |
# nice formatting:
column -t -s'|' -o ' '
将输出:
file_name file_content
./deID.RESUL_44235345636.txt_537967875.txt Content 3
./deID.RESUL_35234663456.txt_423452545.txt Content 4
./deID.RESUL_34534563649.txt_345353567.txt Content 2
./deID.RESUL_12433287659.txt_234323456.txt Content 1