加快在Java中读取CSV的时间

Question

我的CSVReader代码相对无效，请参见下文。读取30000多行需要30秒钟以上。如何尽快加快阅读速度？

public class DataReader {

    private String csvFile;
    private List<String> sub = new ArrayList<String>();
    private List<List> master = new ArrayList<List>();


    public void ReadFromCSV(String csvFile) {

        String line = "";
        String cvsSplitBy = ",";

        try (BufferedReader br = new BufferedReader(new FileReader(csvFile))) {
            System.out.println("Header " + br.readLine());
            while ((line = br.readLine()) != null) {

                // use comma as separator
                String[] list = line.split(cvsSplitBy);
//                System.out.println("the size is " + country[1]);
                for (int i = 0; i < list.length; i++) {
                    sub.add(list[i]);
                }
                List<String> temp = (List<String>) ((ArrayList<String>) sub).clone();
//                master.add(new ArrayList<String>(sub));
                master.add(temp);
                sub.removeAll(sub);
            }

        } catch (IOException e) {
            e.printStackTrace();
        }

        System.out.println(master);
    }

    public List<List> getMaster() {
        return master;
    }

}

Answer 1

我没有那么大的CSV，但是您可以尝试以下操作：

public static void main(String[] args) throws IOException {
    Path csvPath = Paths.get("path/to/file.csv");
    List<List<String>> master = Files.lines(csvPath)
            .skip(1)
            .map(line -> Arrays.asList(line.split(",")))
            .collect(Collectors.toList());
}

编辑：我用一个具有50k条目的CSV sample进行了尝试，并且代码在不到一秒钟的时间内运行。