Hadoop 2.8 NodeManager在Windows上关闭

问题描述 投票:1回答:1

我已经按照本网站上的所有步骤进行操作。在我的PC上,所有设备都运行正常,但是在我的笔记本电脑上,nodeManager正在关闭。

https://muhammadbilalyar.github.io/blogs/How-to-install-Hadoop-on-Window-10/?fbclid=IwAR3u_-xwIulzm9KJ7ivsPD_JyQWI0jPvV9s_Fhp1RpPFupY3LWvrulzRr_4

以下是我运行start-all.cmd命令后的nodeManager状态。

r class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.Res
ourceLocalizationService$LocalizerTracker
19/11/27 12:42:00 INFO containermanager.AuxServices: Adding auxiliary service ma
preduce_shuffle, "mapreduce_shuffle"
19/11/27 12:42:00 INFO monitor.ContainersMonitorImpl:  Using ResourceCalculatorP
lugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@2b72cb8a
19/11/27 12:42:00 INFO monitor.ContainersMonitorImpl:  Using ResourceCalculatorP
rocessTree : null
19/11/27 12:42:00 INFO monitor.ContainersMonitorImpl: Physical memory check enab
led: true
19/11/27 12:42:00 INFO monitor.ContainersMonitorImpl: Virtual memory check enabl
ed: true
19/11/27 12:42:00 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree curre
ntly is supported only on Linux.
19/11/27 12:42:00 INFO monitor.ContainersMonitorImpl: ContainersMonitor enabled:
 true
19/11/27 12:42:01 WARN util.SysInfoWindows: Expected split length of sysInfo to
be 11. Got 7
19/11/27 12:42:01 WARN monitor.ContainersMonitorImpl: NodeManager's totalPmem co
uld not be calculated. Setting it to -1
19/11/27 12:42:01 INFO nodemanager.NodeStatusUpdaterImpl: Nodemanager resources:
 memory set to 8192MB.
19/11/27 12:42:01 INFO nodemanager.NodeStatusUpdaterImpl: Nodemanager resources:
 vcores set to 8.
19/11/27 12:42:01 INFO nodemanager.NodeStatusUpdaterImpl: Initialized nodemanage
r with : physical-memory=8192 virtual-memory=17204 virtual-cores=8
19/11/27 12:42:01 INFO util.JvmPauseMonitor: Starting JVM pause monitor
19/11/27 12:42:01 INFO ipc.CallQueueManager: Using callQueue: class java.util.co
ncurrent.LinkedBlockingQueue queueCapacity: 2000 scheduler: class org.apache.had
oop.ipc.DefaultRpcScheduler
19/11/27 12:42:01 INFO ipc.Server: Starting Socket Reader #1 for port 51483
19/11/27 12:42:01 WARN util.SysInfoWindows: Expected split length of sysInfo to
be 11. Got 7
19/11/27 12:42:01 INFO pb.RpcServerFactoryPBImpl: Adding protocol org.apache.had
oop.yarn.api.ContainerManagementProtocolPB to the server
19/11/27 12:42:01 INFO containermanager.ContainerManagerImpl: Blocking new conta
iner-requests as container manager rpc server is still starting.
19/11/27 12:42:01 INFO ipc.Server: IPC Server Responder: starting
19/11/27 12:42:01 INFO ipc.Server: IPC Server listener on 51483: starting
19/11/27 12:42:01 INFO security.NMContainerTokenSecretManager: Updating node add
ress : annupc:51483
19/11/27 12:42:01 INFO ipc.CallQueueManager: Using callQueue: class java.util.co
ncurrent.LinkedBlockingQueue queueCapacity: 500 scheduler: class org.apache.hado
op.ipc.DefaultRpcScheduler
19/11/27 12:42:01 INFO pb.RpcServerFactoryPBImpl: Adding protocol org.apache.had
oop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server
19/11/27 12:42:01 INFO localizer.ResourceLocalizationService: Localizer started
on port 8040
19/11/27 12:42:01 INFO ipc.Server: Starting Socket Reader #1 for port 8040
19/11/27 12:42:01 INFO ipc.Server: IPC Server listener on 8040: starting
19/11/27 12:42:01 INFO ipc.Server: IPC Server Responder: starting
19/11/27 12:42:01 WARN localizer.ResourceLocalizationService: Permissions incorr
ectly set for dir /tmp/hadoop-anwar/nm-local-dir/usercache, should be rwxr-xr-x,
 actual value = rwxrwxr-x
19/11/27 12:42:01 INFO localizer.ResourceLocalizationService: Attempting to init
ialize /tmp/hadoop-anwar/nm-local-dir
19/11/27 12:42:02 WARN localizer.ResourceLocalizationService: Permissions incorr
ectly set for dir /tmp/hadoop-anwar/nm-local-dir/usercache, should be rwxr-xr-x,
 actual value = rwxrwxr-x
19/11/27 12:42:02 WARN localizer.ResourceLocalizationService: Failed to setup lo
cal dir /tmp/hadoop-anwar/nm-local-dir, which was marked as good.
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Permissions incorrectly
set for dir /tmp/hadoop-anwar/nm-local-dir/usercache, should be rwxr-xr-x, actua
l value = rwxrwxr-x
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkLocalDir(ResourceLocalizationService.java:1560)

        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1528)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService$1.onDirsChanged(ResourceLocalizationService.java:271
)
        at org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection.registe
rDirsChangeListener(DirectoryCollection.java:197)
        at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.reg
isterLocalDirsChangeListener(LocalDirsHandlerService.java:242)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.serviceStart(ResourceLocalizationService.java:371)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
        at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerM
anagerImpl.serviceStart(ContainerManagerImpl.java:490)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
        at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(No
deManager.java:369)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNod
eManager(NodeManager.java:637)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManage
r.java:684)
19/11/27 12:42:02 INFO service.AbstractService: Service org.apache.hadoop.yarn.s
erver.nodemanager.containermanager.localizer.ResourceLocalizationService failed
in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
 Failed to setup local dir /tmp/hadoop-anwar/nm-local-dir, which was marked as g
ood.
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to setup local di
r /tmp/hadoop-anwar/nm-local-dir, which was marked as good.
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1533)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService$1.onDirsChanged(ResourceLocalizationService.java:271
)
        at org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection.registe
rDirsChangeListener(DirectoryCollection.java:197)
        at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.reg
isterLocalDirsChangeListener(LocalDirsHandlerService.java:242)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.serviceStart(ResourceLocalizationService.java:371)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
        at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerM
anagerImpl.serviceStart(ContainerManagerImpl.java:490)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
        at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(No
deManager.java:369)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNod
eManager(NodeManager.java:637)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManage
r.java:684)
Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Permissions i
ncorrectly set for dir /tmp/hadoop-anwar/nm-local-dir/usercache, should be rwxr-
xr-x, actual value = rwxrwxr-x
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkLocalDir(ResourceLocalizationService.java:1560)

        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1528)
        ... 13 more
19/11/27 12:42:02 INFO ipc.Server: Stopping server on 8040
19/11/27 12:42:02 INFO ipc.Server: Stopping IPC Server listener on 8040
19/11/27 12:42:02 INFO ipc.Server: Stopping IPC Server Responder
19/11/27 12:42:02 INFO localizer.ResourceLocalizationService: Public cache exiti
ng
19/11/27 12:42:02 INFO service.AbstractService: Service org.apache.hadoop.yarn.s
erver.nodemanager.containermanager.ContainerManagerImpl failed in state STARTED;
 cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to setup
local dir /tmp/hadoop-anwar/nm-local-dir, which was marked as good.
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to setup local di
r /tmp/hadoop-anwar/nm-local-dir, which was marked as good.
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1533)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService$1.onDirsChanged(ResourceLocalizationService.java:271
)
        at org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection.registe
rDirsChangeListener(DirectoryCollection.java:197)
        at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.reg
isterLocalDirsChangeListener(LocalDirsHandlerService.java:242)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.serviceStart(ResourceLocalizationService.java:371)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
        at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerM
anagerImpl.serviceStart(ContainerManagerImpl.java:490)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
        at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(No
deManager.java:369)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNod
eManager(NodeManager.java:637)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManage
r.java:684)
Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Permissions i
ncorrectly set for dir /tmp/hadoop-anwar/nm-local-dir/usercache, should be rwxr-
xr-x, actual value = rwxrwxr-x
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkLocalDir(ResourceLocalizationService.java:1560)

        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1528)
        ... 13 more
19/11/27 12:42:02 INFO ipc.Server: Stopping server on 51483
19/11/27 12:42:02 INFO ipc.Server: Stopping IPC Server listener on 51483
19/11/27 12:42:02 INFO ipc.Server: Stopping IPC Server Responder
19/11/27 12:42:02 INFO service.AbstractService: Service NodeManager failed in st
ate STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Fail
ed to setup local dir /tmp/hadoop-anwar/nm-local-dir, which was marked as good.
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to setup local di
r /tmp/hadoop-anwar/nm-local-dir, which was marked as good.
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1533)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService$1.onDirsChanged(ResourceLocalizationService.java:271
)
        at org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection.registe
rDirsChangeListener(DirectoryCollection.java:197)
        at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.reg
isterLocalDirsChangeListener(LocalDirsHandlerService.java:242)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.serviceStart(ResourceLocalizationService.java:371)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
        at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerM
anagerImpl.serviceStart(ContainerManagerImpl.java:490)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
        at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(No
deManager.java:369)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNod
eManager(NodeManager.java:637)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManage
r.java:684)
Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Permissions i
ncorrectly set for dir /tmp/hadoop-anwar/nm-local-dir/usercache, should be rwxr-
xr-x, actual value = rwxrwxr-x
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkLocalDir(ResourceLocalizationService.java:1560)

        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1528)
        ... 13 more
19/11/27 12:42:02 WARN nodemanager.NodeResourceMonitorImpl: org.apache.hadoop.ya
rn.server.nodemanager.NodeResourceMonitorImpl is interrupted. Exiting.
19/11/27 12:42:02 INFO impl.MetricsSystemImpl: Stopping NodeManager metrics syst
em...
19/11/27 12:42:02 INFO impl.MetricsSystemImpl: NodeManager metrics system stoppe
d.
19/11/27 12:42:02 INFO impl.MetricsSystemImpl: NodeManager metrics system shutdo
wn complete.
19/11/27 12:42:02 FATAL nodemanager.NodeManager: Error starting NodeManager
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to setup local di
r /tmp/hadoop-anwar/nm-local-dir, which was marked as good.
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1533)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService$1.onDirsChanged(ResourceLocalizationService.java:271
)
        at org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection.registe
rDirsChangeListener(DirectoryCollection.java:197)
        at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.reg
isterLocalDirsChangeListener(LocalDirsHandlerService.java:242)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.serviceStart(ResourceLocalizationService.java:371)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
        at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerM
anagerImpl.serviceStart(ContainerManagerImpl.java:490)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
        at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(No
deManager.java:369)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNod
eManager(NodeManager.java:637)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManage
r.java:684)
Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Permissions i
ncorrectly set for dir /tmp/hadoop-anwar/nm-local-dir/usercache, should be rwxr-
xr-x, actual value = rwxrwxr-x
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkLocalDir(ResourceLocalizationService.java:1560)

        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1528)
        ... 13 more
19/11/27 12:42:02 INFO nodemanager.NodeManager: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NodeManager at annupc/192.168.1.103
************************************************************/
hadoop hadoop2 nodemanager
1个回答
0
投票

您是否在HADOOP目录中获得以下bin文件

https://github.com/s911415/apache-hadoop-3.1.0-winutils

也,尝试清除datanode文件并从C:\中删除临时文件>

© www.soinside.com 2019 - 2024. All rights reserved.