如何计算数据帧的协方差矩阵

问题描述 投票:0回答:3

我已经使用pandas read_fwf函数读取了传感器数据的数据帧。我需要找到读取928991 x 8矩阵的协方差矩阵。最后,我想找到特征向量和特征值,使用该协方差矩阵的主成分分析算法。

python pca covariance eigenvalue
3个回答
1
投票

为什么不使用pd.DataFrame.cov function


1
投票

首先,您需要使用df.values将pandas数据帧放到numpy数组中。例如:

A = df.values

将数据放入numpy数组后,计算协方差矩阵或PCA会很容易。更多:

# import functions you need to compute covariance matrix from numpy
from numpy import array
from numpy import mean
from numpy import cov
from numpy.linalg import eig

# assume you load your data using pd.read_fwf to variable *df*
df = pd.read_fwf(filepath, widths=col_widths, names=col_names)
#put dataframe values to a numpy array
A = df.values
#check matrix A's shape, it should be (928991, 8)
print(A.shape)
# calculate the mean of each column
M = mean(A.T, axis=1)
print(M)
# center columns by subtracting column means
C = A - M
print(C)
# calculate covariance matrix of centered matrix
V = cov(C.T)
print(V)
# eigendecomposition of covariance matrix
values, vectors = eig(V)
print(vectors)
print(values)
# project data
P = vectors.T.dot(C.T)
print(P.T)

首先运行该示例打印原始矩阵,然后是居中协方差矩阵的特征向量和特征值,最后是原始矩阵的投影。以下是您可能对PCA task有用的链接。


0
投票

这个问题的答案如下

import pandas as pd
import numpy as np
from numpy.linalg import eig

df_sensor_data = pd.read_csv('HT_Sensor_dataset.dat', delim_whitespace=True)
del df_sensor_data['id']
del df_sensor_data['time']
del df_sensor_data['Temp.']
del df_sensor_data['Humidity']
df = df_sensor_data.notna().astype('float64')
covariance_matrix = df_sensor_data.cov()
print(covariance_matrix)

values, vectors = eig(covariance_matrix)
print(values)
print(vectors)
© www.soinside.com 2019 - 2024. All rights reserved.