我在Java类下面,尝试从CSV
文件导入数据并插入到oracle数据库表中。
[当我尝试运行此Java类时,出现错误,如'java.lang.IndexOutOfBoundsException'。我只是尝试使用log.info("Insert values to table" +sql);
查看插入语句,我可以看到生成了一些插入语句,然后引发了错误。
我是Java的新手,所以不确定如何处理此异常。
下面是我的代码:
@Service
public class OBSCSVImporter {
private final static Logger log = LoggerFactory.getLogger(tur.class);
//Tabellenname auf Spalten in der Tabelle
Map<String, List<ColumnData>> columnDataCache = new HashMap<>();
private final JdbcTemplate jdbcTemplate;
public OBSCSVImporter(JdbcTemplate jdbcTemplate) {
this.jdbcTemplate = jdbcTemplate;
}
/**
* Be careful: Deletes all the data from the given table.
* @param tableName the name of the table the data of which should be deleted.
*/
public void deleteTableData(String tableName){
jdbcTemplate.update("delete from " + tableName);
}
public void importFile(BufferedReader bufferedReader, String tableName, String fileName,
Map<Integer, Function<String, String>> COLUMN_TYPE_MAPPINGS,
Map<String, Function<String, String>> COLUMN_NAME_MAPPINGS) throws IOException {
importFile(bufferedReader, tableName, fileName, COLUMN_TYPE_MAPPINGS, COLUMN_NAME_MAPPINGS, new ArrayList<>());
}
public void importFile(BufferedReader bufferedReader, String tableName, String fileName,
Map<Integer, Function<String, String>> COLUMN_TYPE_MAPPINGS,
Map<String, Function<String, String>> COLUMN_NAME_MAPPINGS,
List<FixedValue> fixedValues) throws IOException {
String firstLine = bufferedReader.readLine();
List<String> headings = Splitter
.on(";")
.trimResults()
.splitToList(firstLine)
.stream()
.map(str -> str.replace("XXX-", ""))
.map(str -> str.replace("-", ""))
.map(str -> str.replaceAll("\\.", ""))
.map(str -> str.equals("ORDER") ? "ORDERID": str) // to work for ORDER columns in CSV files
.map(String::toUpperCase)
.filter(str -> str.length() > 0) // to suppress empty columns at the end of the line
.collect(toList());
List<ColumnData> columnData = getColumns(tableName)
.stream()
.filter(cd -> !cd.getName().equals("SYSTEM_INSERTED_AT") && !cd.getName().equals("DATEINAME"))
.collect(toList());
List<String> columnNames = columnData.stream().map(ColumnData::getName).collect(toList());
//Nur Spalten, die sowohl in der Datenbank als auch in der CSV-Datei enthalten sind, werden verarbeitet
List<String> matchingColumns = intersection(headings, columnNames);
List<String> columnsInDbButNotInCsv = subtract(columnNames, headings);
columnsInDbButNotInCsv = subtract(columnsInDbButNotInCsv, fixedValues.stream().map(f -> f.getColumnName()).collect(toList()));
columnsInDbButNotInCsv.forEach(s -> log.warn("Spalte '" + s + "' in Datenbank vorhanden, wird jedoch nicht in der CSV bereitgestellt"));
//Index to columnData
Map<Integer, ColumnData> columnsToInsert = new HashMap<>();
for (String column : matchingColumns) {
columnsToInsert.put(headings.indexOf(column), columnData.stream().filter(cd -> cd.getName().equals(column)).findAny().get());
}
int counter = 0;
while (true) {
String line = bufferedReader.readLine();
if (line == null) {
break;
}
counter++;
List<String> columns = Splitter
.on(";")
.splitToList(line);
StringBuilder columnSqlFragment = new StringBuilder();
StringBuilder valuesSqlFragment = new StringBuilder();
for (int i = 0; i < headings.size(); i++) {
String columnName = headings.get(i);
String value = columns.get(i);
if (!columnsToInsert.containsKey(i)) {
continue;
}
if (i != 0) {
columnSqlFragment.append(", ");
valuesSqlFragment.append(", ");
}
int columnType = columnsToInsert.get(i).getType();
java.util.function.Function<String, String> mapping = COLUMN_NAME_MAPPINGS.containsKey(columnName) ? COLUMN_NAME_MAPPINGS.get(columnName) : COLUMN_TYPE_MAPPINGS.get(columnType);
if(mapping == null) {
throw new IllegalStateException("Kein Mapping für Spalte mit dem Namen " + columnName + " aus der Tabelle " + tableName + " mit Typ " + columnType + " gefunden");
}
columnSqlFragment.append("\"" + columnName + "\"");
valuesSqlFragment.append(mapping.apply(value));
}
for(FixedValue fv : fixedValues) {
columnSqlFragment.append(", ");
columnSqlFragment.append(fv.getColumnName());
valuesSqlFragment.append(", ");
valuesSqlFragment.append(fv.getValueSQLFragment());
}
String sql = "insert into " + tableName + " (SYSTEM_INSERTED_AT, DATEINAME, " + columnSqlFragment + ") VALUES (sysdate, '" + fileName + "', " + valuesSqlFragment + ")";
log.info("Insert values to table" +sql);
jdbcTemplate.update(sql);
}
log.info(counter + " Zeilen erfolgreich eingelesen.");
}
private List<ColumnData> getColumns(String tableName) {
if(columnDataCache.containsKey(tableName)) {
return columnDataCache.get(tableName);
}
List<ColumnData> result = new ArrayList<>();
try (Connection connection = jdbcTemplate.getDataSource().getConnection()) {
ResultSet rs = connection.getMetaData().getColumns(null, null, tableName, null);
boolean anyResults = false;
while (rs.next()) {
anyResults = true;
String columnName = rs.getString(4).toUpperCase();
int type = rs.getInt(5);
result.add(new ColumnData(columnName, type));
}
columnDataCache.putIfAbsent(tableName, result);
if(!anyResults) {
throw new RuntimeException("Could not retrieve columns for table " + tableName);
}
return result;
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
}
CSV文件仅包含1000行。以下是CSV文件中的示例数据:
1;KK;130560;Morgan Stanley & Co. International,;RW 2018/0465;20190805;-1179,92;CHF;20191128;20191110;J;20191110;C;
1;KK;130561;Arkem Ltd, London;RSUK 2019/0145;20191127;-4500,00;EUR;20191128;20200326;N;0;6;
1;KK;130562;Morgan Stanley & Co. International,;ABW 2019-01292;20190807;-102,40;USD;20191128;20191112;J;20191112;C;
1;KK;130562;Morgan Stanley & Co. International,;ABW 2019-01547;20191008;-4,80;USD;20191128;20200113;N;0;6;
1;KK;130563;Vogt Consulting & Management AG, Va;RSCH 2019/0126;20191127;-833,33;EUR;20191128;20200326;N;0;6;
1;KK;130563;Vogt Consulting & Management AG, Va;RSCH 2019/0127;20191127;-2500,00;EUR;20191128;20200326;N;0;6;
1;KK;130565;Garraway Capital Management LLP, Lo;RSUK 2019/0095;20190822;-4000,00;GBP;20191128;20191220;N;0;6;
1;KK;130566;Acoro Asset Management AG, Küsnacht;RSCH 2019/0107;20191118;-1077,00;CHF;20191128;20200223;N;0;6;
1;KK;130569;Marathon Asset Management LLP, Lond;RSUK 2019/0138;20191111;-6000,00;USD;20191128;20200216;N;0;6;
1;KK;130572;LeanVal Invest GmbH, Leverkusen;122019;20191125;-240,00;EUR;20191128;20200226;N;0;6;
1;KK;130576;Mako Global Derivatives Partnership;RSUK 2019/0149;20191127;-2625,00;GBP;20191128;20200326;N;0;6;
1;KK;130578;Gladstone Capital Management LLP, L;RSUK 2019/0129;20190930;-2040,00;EUR;20191128;20200128;N;0;6;
1;KK;130579;WA Capital GmbH, Siegsdorf;RSDE 2019/0169;20190923;-892,50;EUR;20191128;20200121;N;0;6;
1;KK;130584;Mandamus Capital Limited, Dublin 2
;RSUK 2019/0156;20191127;-1200,00;EUR;20191128;20200326;N;0;6;
1;KK;130588;Morgan Stanley Europe SE, Frankfurt;ABW 2019-01796;20191106;-97,50;USD;20191128;20200211;N;0;6;
我不确定是哪一行导致了错误-您可以输入行号或堆栈跟踪信息吗?但是,这让我担心:
for (int i = 0; i < headings.size(); i++) {
String columnName = headings.get(i);
String value = columns.get(i); // <- this
您正在从i
到headings.size()
进行迭代,但是如果headings.size()
大于columns.size()
怎么办?然后,您的程序将尝试访问不存在的索引columns
,这将为您提供IndexOutOfBounds异常。