首先,准备一个数据库来存储数据:
psql -U postgres
create database vn;
通过web api获取数据:
import requests,json,time
import pandas as pd
from pandas import json_normalize
ticker = 'aaa'
start_date='2020-01-01'
end_date='2023-05-10'
fd = int(time.mktime(time.strptime(start_date, "%Y-%m-%d")))
td = int(time.mktime(time.strptime(end_date, "%Y-%m-%d")))
data = requests.get('https://apipubaws.tcbs.com.vn/stock-insight/v1/stock/bars-long-term?ticker={}&type=stock&resolution=D&from={}&to={}'.format(ticker, fd, td)).json()
df = json_normalize(data['data'])
df['tradingDate'] = pd.to_datetime(df.tradingDate.str.split("T", expand=True)[0])
df.insert(0,'ticker',ticker)
df.columns = ['ticker','open', 'high', 'low', 'close', 'volume', 'date']
我们可以看到
date
列是升序排列的。准备写入数据的引擎:
import sqlalchemy
from sqlalchemy import create_engine
db_user = 'postgres'
db_ip = '127.0.0.1'
db_pass = 'pass_of_your_local_db'
db_name = 'vn'
engine = create_engine('postgresql://{}:{}@{}/{}'.format(db_user,db_pass,db_ip,db_name))
然后dtype:
dtype={
'ticker':sqlalchemy.types.Text(),
'open':sqlalchemy.types.DECIMAL(precision=15,scale=2,asdecimal=True),
'high':sqlalchemy.types.DECIMAL(precision=15,scale=2,asdecimal=True),
'low':sqlalchemy.types.DECIMAL(precision=15,scale=2,asdecimal=True),
'close':sqlalchemy.types.DECIMAL(precision=15,scale=2,asdecimal=True),
'volume':sqlalchemy.types.DECIMAL(precision=15,scale=2,asdecimal=True),
'date':sqlalchemy.types.Date()
}
将数据写入数据库表
quote
:
df.to_sql('quote',con=engine,if_exists='append',index=False,dtype=dtype)
在postgres控制台查看数据:
select * from quote where ticker='aaa';
为什么
date
字段的数据处于乱序状态,既不升序也不降序?