I am not able to connect my spark Code which is written on eclipse. Below is the code please guid me how I can do the same. Anything would be helpful import java.util.Arrays; import org.apache.spark.SparkConf; import org.apache.spark.api.java.JavaPairRDD; import org.apache.spark.api.java.JavaRDD; import org.apache.spark.api.java.JavaSparkContext; public class SparkTest {
public static void main(String [] args){
SparkConf conf = new SparkConf().setAppName("JD Word
Counter“)。setMaster(” local“);
JavaSparkContext sc = new JavaSparkContext(conf); //hdfs://localhost:8020/user/root/textfile/test.txt JavaRDD<String> inputFile =
sc.textFile(“ hdfs:// localhost:8020 / user / root / textfile / test.txt”);System.out.println(“ Hello start”);System.out.println(inputFile.collect()); JavaRDDwordsFromFile = inputFile.flatMap(content->Arrays.asList(content.split(“”))。iterator());System.out.println(“ hello end”);
//JavaPairRDD countData = wordsFromFile.mapToPair(t -> new Tuple2(t, 1)).reduceByKey((x, y) -> (int) x + (int) y); //wordsFromFile.saveAsTextFile("hdfs://localhost:8020/user/root/fileTest/"); System.out.println(" This java program is complete"); } } Error I/O error constructing remote block reader. org.apache.hadoop.net.ConnectTimeoutException: 60000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=/172.18.0.2:50010] at org.apache.hadoop.net.NetUtils.c
将localhost更改为hdp沙箱的ip地址,或将hdfs-site.xml文件放入类路径中。