[2023/04/18 13:27:55 WARN YarnSchedulerBackend$YarnSchedulerEndpoint:
Requesting driver to remove executor 4 for reason Container from a bad
node: container_1681790542995_0003_01_000008 on host: node1. Exit
status: 1. Diagnostics: [2023-04-18 13:27:53.964]Exception from
container-launch. Container id: container_1681790542995_0003_01_000008
Exit code: 1 [2023-04-18 13:27:53.966]Container exited with a non-zero
exit code 1. Error file: prelaunch.err. Last 4096 bytes of
prelaunch.err : Last 4096 bytes of stderr :
ty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748) 23/04/18 13:27:55 WARN YarnSchedulerBackend$YarnSchedulerEndpoint: Requesting driver to
remove executor 5 for reason Container from a bad node:
container_1681790542995_0003_01_000009 on host: node3. Exit status: 1.
Diagnostics: [2023-04-18 13:27:53.964]Exception from container-launch.
Container id: container_1681790542995_0003_01_000009 Exit code: 1
spark-submit \
--master yarn \
--deploy-mode client \
--driver-memory 1G \
--num-executors 3 \
--executor-cores 2 \
--executor-memory 4G \
--class com.game.logs.statistic.ItemInfoSql \ /data/scripts/dw/xx_xxx_log_statistic-1.0-SNAPSHOT.jar
xx_xxx_log_xxx_xxx 2023-04-13