Twitter spark streaming:登录尝试次数过多

问题描述 投票:0回答:1

我正在运行由apache提供的this示例来传输推文。但是,我无法获取任何数据,因为似乎流API不断尝试连接到Twitter,从而导致此跟踪:

2816 [Twitter Stream consumer /  [1][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
2927 [Twitter Stream consumer /  [2][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
2989 [Twitter Stream consumer /  [3][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3063 [Twitter Stream consumer /  [4][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3113 [Twitter Stream consumer /  [5][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3153 [Twitter Stream consumer /  [6][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3197 [Twitter Stream consumer /  [7][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3245 [Twitter Stream consumer /  [8][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3288 [Twitter Stream consumer /  [9][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3333 [Twitter Stream consumer /  [10][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3374 [Twitter Stream consumer /  [11][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.

Popular topics in last 60 seconds (0 total):
3428 [Twitter Stream consumer /  [12][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3510 [Twitter Stream consumer /  [13][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3563 [Twitter Stream consumer /  [14][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.

Popular topics in last 10 seconds (0 total):
3595 [Twitter Stream consumer /  [15][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3626 [Twitter Stream consumer /  [16][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3672 [Twitter Stream consumer /  [17][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3723 [Twitter Stream consumer /  [18][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3762 [Twitter Stream consumer /  [19][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3806 [Twitter Stream consumer /  [20][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3849 [Twitter Stream consumer /  [21][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3865 [Twitter Stream consumer /  [7][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Exceeded connection limit for user

3868 [Twitter Stream consumer /  [7][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
3878 [Twitter Stream consumer /  [1][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Exceeded connection limit for user

3879 [Twitter Stream consumer /  [1][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - Waiting for 10000 milliseconds
3880 [Twitter Stream consumer /  [5][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Exceeded connection limit for user

3880 [Twitter Stream consumer /  [5][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
3898 [Twitter Stream consumer /  [22][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3911 [Twitter Stream consumer /  [3][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Exceeded connection limit for user

3911 [Twitter Stream consumer /  [3][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
3944 [Twitter Stream consumer /  [23][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3985 [Twitter Stream consumer /  [24][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4036 [Twitter Stream consumer /  [25][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4086 [Twitter Stream consumer /  [26][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4127 [Twitter Stream consumer /  [27][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4168 [Twitter Stream consumer /  [28][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4208 [Twitter Stream consumer /  [29][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4256 [Twitter Stream consumer /  [30][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4347 [Twitter Stream consumer /  [31][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4377 [Twitter Stream consumer /  [4][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Easy there, Turbo. Too many requests recently. Enhance your calm.

4377 [Twitter Stream consumer /  [4][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
4402 [Twitter Stream consumer /  [32][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4455 [Twitter Stream consumer /  [33][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4504 [Twitter Stream consumer /  [34][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4551 [Twitter Stream consumer /  [35][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4584 [Twitter Stream consumer /  [36][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.

Popular topics in last 60 seconds (0 total):
4621 [Twitter Stream consumer /  [37][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4664 [Twitter Stream consumer /  [38][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.

Popular topics in last 10 seconds (0 total):
4697 [Twitter Stream consumer /  [39][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4728 [Twitter Stream consumer /  [40][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4755 [Twitter Stream consumer /  [41][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4786 [Twitter Stream consumer /  [42][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4818 [Twitter Stream consumer /  [43][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4854 [Twitter Stream consumer /  [44][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4877 [Twitter Stream consumer /  [8][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Easy there, Turbo. Too many requests recently. Enhance your calm.

4877 [Twitter Stream consumer /  [8][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
4891 [Twitter Stream consumer /  [45][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4924 [Twitter Stream consumer /  [46][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4955 [Twitter Stream consumer /  [47][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4990 [Twitter Stream consumer /  [48][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5025 [Twitter Stream consumer /  [49][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5067 [Twitter Stream consumer /  [50][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5103 [Twitter Stream consumer /  [51][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5137 [Twitter Stream consumer /  [52][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5171 [Twitter Stream consumer /  [53][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5203 [Twitter Stream consumer /  [54][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5235 [Twitter Stream consumer /  [55][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5271 [Twitter Stream consumer /  [56][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5302 [Twitter Stream consumer /  [57][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5334 [Twitter Stream consumer /  [58][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5364 [Twitter Stream consumer /  [59][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5397 [Twitter Stream consumer /  [60][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5411 [Twitter Stream consumer /  [9][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Easy there, Turbo. Too many requests recently. Enhance your calm.

5411 [Twitter Stream consumer /  [9][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
5429 [Twitter Stream consumer /  [61][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5461 [Twitter Stream consumer /  [62][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5492 [Twitter Stream consumer /  [63][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5521 [Twitter Stream consumer /  [64][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5553 [Twitter Stream consumer /  [65][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5582 [Twitter Stream consumer /  [66][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5616 [Twitter Stream consumer /  [67][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5650 [Twitter Stream consumer /  [68][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
^CException in thread "dispatcher-event-loop-6" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stop(ReceiverSupervisor.scala:137)
        at org.apache.spark.streaming.receiver.ReceiverSupervisorImpl$$anon$1$$anonfun$receive$1.applyOrElse(ReceiverSupervisorImpl.scala:82)
        at org.apache.spark.rpc.netty.Inbox$$anonfun$process$1.apply$mcV$sp(Inbox.scala:117)
        at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:205)
        at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:101)
        at org.apache.spark.rpc.netty.Dispatcher$MessageLoop.run(Dispatcher.scala:221)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
5907 [Twitter Stream consumer /  [10][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Easy there, Turbo. Too many requests recently. Enhance your calm.

5907 [Twitter Stream consumer /  [10][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

有谁知道它为什么要建立多个连接?这似乎是问题的根本原因。

scala apache-spark twitter spark-streaming
1个回答
0
投票

异常是由软件依赖项之间的不兼容性引起的。自从上次提交2017年以来,Apache Bahir似乎已经停止了开发。因此,似乎让twitter流入火花的唯一方法是将所有内容降级为兼容版本,或者使用twitter4j来创建流而不是火花流构造。

© www.soinside.com 2019 - 2024. All rights reserved.