我有一个文本文件,由FORTRAN程序产生的,用比较奇怪的(且确实烦人)格式:
3.4502 1.5959 0.2160 0.9423 0.1098 1.2463 -2.8673 0.8803
3.5724 1.8022 0.3423 1.0801 2.4177 -0.2012 -0.1142 -0.2061
2.6028 2.6395 0.2959 0.8280 2.0526 -0.0721 -1.1345 0.0110
2.5628 0.0000 0.0539 0.0000 -0.4520 1.3030 -3.0792 1.0428
1.1823 1.4084 0.2315 1.1359 1.5945 3.2098 1.6739 0.0713
0.0296 1.3689 0.0000 1.0425 -0.4525 1.3043 -2.9785 1.0428
2.4825 1.6460 0.2573 2.4801 3.4533 1.5960 0.3609 0.9574
2.2358 0.8858 0.1344 0.5376 3.1102 -0.8025 0.1282 -0.8398
0.0000 1.4078 1.5464 1.0526 3.9754 3.7823 0.3376 0.1303
3.3068 2.5148 0.2390 -0.3816
-0.4672 1.3604 2.0157 1.0405
4.4009 2.9969 0.8777 3.6270
3.0271 4.1610 0.2094 3.0105
-0.4889 1.3888 3.1442 1.0423
6.0767 1.7731 0.6439 2.3744
5.9313 1.3423 0.2204 1.0397
4.4335 2.9075 -0.0328 -0.4526
4.8670 2.6906 0.1088 0.0275
2.5303 3.3157 -0.2649 0.9895
4.3957 3.4142 0.3900 0.4282
3.3185 1.4058 0.2024 3.3997
0.9097 1.3423 0.2388 1.1809
1.3302 1.6167 0.2009 1.0491
2.4382 -0.1739 0.4722 3.5331
1.8617 1.4082 0.2140 0.6741
我想分别读取前四个和后四列,它们存储到NumPy的阵列。使用numpy.genfromtxt,我很容易地得到了前四列中的数据:
object_scores = numpy.genfromtxt("results.out", usecols=(0,1,2,3), max_rows=9)
但是,试图为其他四列做同样的时
descriptor_scores = numpy.genfromtxt("results.out", usecols=(4,5,6,7), max_rows=25)
我得到错误信息,似乎在第四列可能与失踪细胞的一个长长的清单。
ValueError: Some errors were detected !
Line #10 (got 4 columns instead of 1)
Line #11 (got 4 columns instead of 1)
Line #12 (got 4 columns instead of 1)
Line #13 (got 4 columns instead of 1)
Line #14 (got 4 columns instead of 1)
Line #15 (got 4 columns instead of 1)
Line #16 (got 4 columns instead of 1)
Line #17 (got 4 columns instead of 1)
Line #18 (got 4 columns instead of 1)
Line #19 (got 4 columns instead of 1)
Line #20 (got 4 columns instead of 1)
Line #21 (got 4 columns instead of 1)
Line #22 (got 4 columns instead of 1)
Line #23 (got 4 columns instead of 1)
Line #24 (got 4 columns instead of 1)
Line #25 (got 4 columns instead of 1)
如何解决这个问题的任何提示或建议?
不幸的列似乎并不具有相同的宽度(10的前四个字段,然后11)。如果是的话,delimiter=
的numpy.genfromtxt
选项可以帮助你。
下面是一个替代的解决方案来读取4个字段开始,在37柱:
descriptor_scores = numpy.genfromtxt([s[37:] for s in open("results.out")], usecols=(0,1,2,3), max_rows=25)
如果该文件的格式是始终不变的,这会做:
import numpy as np
def squash(obj):
return [[float(element) for element in column if element.strip() != ''] for column in obj]
with open('results.out') as f:
data = f.read()
lines = data.split('\n')
number_width = 6
number_spacing = 4
result = squash(zip(*[[line[i:i + number_width] for i in range(0, len(line), number_width + number_spacing)]
for line in lines]))
first_four_cols = np.array(result[0:4]).T
last_four_cols = np.array(result[4:]).T
与复制和粘贴到文件
In [85]: data = np.genfromtxt('stack54544789.py', delimiter=[10]*8)
In [86]: data
Out[86]:
array([[3.4502, 1.5959, 0.216 , 0.9423, 0.1098, nan, 2.8673, 0.8803],
[3.5724, 1.8022, 0.3423, 1.0801, nan, nan, nan, 0.2061],
[2.6028, 2.6395, 0.2959, 0.828 , nan, nan, 1.1345, 0.011 ],
[2.5628, 0. , 0.0539, nan, 0.452 , nan, 3.0792, 1.0428],
[1.1823, 1.4084, 0.2315, 1.1359, 1.5945, 3.2098, 1.6739, 0.0713],
...
[ nan, nan, nan, nan, 1.3302, 1.6167, 0.2009, 1.0491],
[ nan, nan, nan, nan, nan, 0.1739, 0.4722, 3.5331],
[ nan, nan, nan, nan, 1.8617, 1.4082, 0.214 , 0.6741],
[ nan, nan, nan, nan, nan, nan, nan, nan]])
这看起来几乎权利;我认为额外的nan
来自负号是错误的。
In [87]: data = np.genfromtxt('stack54544789.py', delimiter=[9]+[10]*7)
In [88]: data
Out[88]:
array([[ 3.4502, 1.5959, 0.216 , 0.9423, 0.1098, 1.2463, -2.8673,
0.8803],
[ 3.5724, 1.8022, 0.3423, 1.0801, 2.4177, -0.2012, -0.1142,
-0.2061],
[ 2.6028, 2.6395, 0.2959, 0.828 , 2.0526, -0.0721, -1.1345,
0.011 ],
[ 2.5628, 0. , 0.0539, 0. , -0.452 , 1.303 , -3.0792,
1.0428],
...
[ nan, nan, nan, nan, 2.4382, -0.1739, 0.4722,
3.5331],
[ nan, nan, nan, nan, 1.8617, 1.4082, 0.214 ,
0.6741],
[ nan, nan, nan, nan, nan, nan, nan,
nan]])
虽然这是从分隔格式当然不同像.csv
(并因此可能是恼人的一些),Fortran和类似语言常常使用固定宽度的格式,如本实施例中。这是因为他们表现非常好,对于较大的文件,它们往往直接匹配的数据是如何在内存中表示的,这使得它更容易在这些语言的代码。
我不知道你的例子包含了完整的数据(StackOverflow上可以摆脱一些空白的你)。但我想到的是,当你直接读取文件,每列将宽恰好10个字符,你可以这样阅读:
def convert(s):
try:
return float(s)
except ValueError:
return None
data = []
size = 10
with open('input.data', 'r') as f:
for line in f:
# process line, minus the EOL (len(line)-1)
data.append([convert(line[0+i:size+i]) for i in range(0, len(line)-1, size)])
其他人已经注意到,列的宽度似乎有所不同,但我认为这仅仅是你将数据复制到你的问题的神器 - 这似乎很有可能是场实际上是源数据文件中的所有相同的宽度。