为什么我成功运行的mapreduce作业没有作为条目显示在资源管理器Web界面（0.0.0.0:8088）中？

问题描述 投票：0回答：0

您好，我已经完成了 hadoop 集群安装/配置。我已经运行了几个 MapReduce 测试，它们成功地返回了结果。但是，当我尝试在资源管理器 Web 界面（0.0.0.0:8088）上跟踪它们时，什么也没有出现。

我检查了每条日志，一切似乎都很好。

这是下面运行的 hadoop 作业，以及 Web 界面的图像。

hadoop@rai-lab-hdwk-01:~$ yarn jar /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.4.jar pi 16 1000
Number of Maps  = 16
Samples per Map = 1000
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Wrote input for Map #10
Wrote input for Map #11
Wrote input for Map #12
Wrote input for Map #13
Wrote input for Map #14
Wrote input for Map #15
Starting Job
2023-07-24 16:05:38,030 INFO impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties
2023-07-24 16:05:38,112 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
2023-07-24 16:05:38,112 INFO impl.MetricsSystemImpl: JobTracker metrics system started
2023-07-24 16:05:38,245 INFO input.FileInputFormat: Total input files to process : 16
2023-07-24 16:05:38,255 INFO mapreduce.JobSubmitter: number of splits:16
2023-07-24 16:05:38,367 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1819974892_0001
2023-07-24 16:05:38,367 INFO mapreduce.JobSubmitter: Executing with tokens: []
2023-07-24 16:05:38,476 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
2023-07-24 16:05:38,477 INFO mapreduce.Job: Running job: job_local1819974892_0001
2023-07-24 16:05:38,479 INFO mapred.LocalJobRunner: OutputCommitter set in config null
2023-07-24 16:05:38,488 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2023-07-24 16:05:38,488 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2023-07-24 16:05:38,488 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2023-07-24 16:05:38,531 INFO mapred.LocalJobRunner: Waiting for map tasks
2023-07-24 16:05:38,532 INFO mapred.LocalJobRunner: Starting task: attempt_local1819974892_0001_m_000000_0
2023-07-24 16:05:38,559 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2023-07-24 16:05:38,560 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2023-07-24 16:05:38,580 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2023-07-24 16:05:38,583 INFO mapred.MapTask: Processing split: hdfs://10.4.5.242:9000/user/hadoop/QuasiMonteCarlo_1690214736371_1624327287/in/part0:0+118
2023-07-24 16:05:38,711 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2023-07-24 16:05:38,711 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
2023-07-24 16:05:38,712 INFO mapred.MapTask: soft limit at 83886080
2023-07-24 16:05:38,712 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
2023-07-24 16:05:38,712 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
2023-07-24 16:05:38,717 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2023-07-24 16:05:38,770 INFO mapred.LocalJobRunner:
2023-07-24 16:05:38,772 INFO mapred.MapTask: Starting flush of map output
2023-07-24 16:05:38,772 INFO mapred.MapTask: Spilling map output
2023-07-24 16:05:38,772 INFO mapred.MapTask: bufstart = 0; bufend = 18; bufvoid = 104857600
2023-07-24 16:05:38,772 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600
2023-07-24 16:05:38,780 INFO mapred.MapTask: Finished spill 0
2023-07-24 16:05:38,791 INFO mapred.Task: Task:attempt_local1819974892_0001_m_000000_0 is done. And is in the process of committing
2023-07-24 16:05:38,794 INFO mapred.LocalJobRunner: Generated 1000 samples.
2023-07-24 16:05:38,794 INFO mapred.Task: Task 'attempt_local1819974892_0001_m_000000_0' done.
2023-07-24 16:05:38,803 INFO mapred.Task: Final Counters for attempt_local1819974892_0001_m_000000_0: Counters: 23
        File System Counters
                FILE: Number of bytes read=319766
                FILE: Number of bytes written=876110
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=118
                HDFS: Number of bytes written=1888
                HDFS: Number of read operations=7
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=18
                HDFS: Number of bytes read erasure-coded=0
        Map-Reduce Framework
                Map input records=1
                Map output records=2
                Map output bytes=18
                Map output materialized bytes=28
                Input split bytes=149
                Combine input records=0
                Spilled Records=2
                Failed Shuffles=0
                Merged Map outputs=0
                GC time elapsed (ms)=0
                Total committed heap usage (bytes)=348127232
        File Input Format Counters
                Bytes Read=118
2023-07-24 16:05:38,803 INFO mapred.LocalJobRunner: Finishing task: attempt_local1819974892_0001_m_000000_0
2023-07-24 16:05:38,804 INFO mapred.LocalJobRunner: Starting task: attempt_local1819974892_0001_m_000001_0
2023-07-24 16:05:38,805 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2023-07-24 16:05:38,805 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2023-07-24 16:05:38,805 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2023-07-24 16:05:38,807 INFO mapred.MapTask: Processing split: hdfs://10.4.5.242:9000/user/hadoop/QuasiMonteCarlo_1690214736371_1624327287/in/part1:0+118
2023-07-24 16:05:38,915 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2023-07-24 16:05:38,916 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
2023-07-24 16:05:38,916 INFO mapred.MapTask: soft limit at 83886080
2023-07-24 16:05:38,916 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
2023-07-24 16:05:38,916 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
2023-07-24 16:05:38,916 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2023-07-24 16:05:38,923 INFO mapred.LocalJobRunner:
2023-07-24 16:05:38,923 INFO mapred.MapTask: Starting flush of map output
2023-07-24 16:05:38,923 INFO mapred.MapTask: Spilling map output
2023-07-24 16:05:38,923 INFO mapred.MapTask: bufstart = 0; bufend = 18; bufvoid = 104857600
2023-07-24 16:05:38,923 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600
2023-07-24 16:05:38,925 INFO mapred.MapTask: Finished spill 0
2023-07-24 16:05:38,927 INFO mapred.Task: Task:attempt_local1819974892_0001_m_000001_0 is done. And is in the process of committing
2023-07-24 16:05:38,930 INFO mapred.LocalJobRunner: Generated 1000 samples.
2023-07-24 16:05:38,930 INFO mapred.Task: Task 'attempt_local1819974892_0001_m_000001_0' done.
2023-07-24 16:05:38,930 INFO mapred.Task: Final Counters for attempt_local1819974892_0001_m_000001_0: Counters: 23
 


..... 


Too long to post it all


.....
  
    
        File System Counters
                FILE: Number of bytes read=341805
                FILE: Number of bytes written=877010
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=1888
                HDFS: Number of bytes written=1888
                HDFS: Number of read operations=52
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=18
                HDFS: Number of bytes read erasure-coded=0
        Map-Reduce Framework
                Map input records=1
                Map output records=2
                Map output bytes=18
                Map output materialized bytes=28
                Input split bytes=149
                Combine input records=0
                Spilled Records=2
                Failed Shuffles=0
                Merged Map outputs=0
                GC time elapsed (ms)=0
                Total committed heap usage (bytes)=1898971136
        File Input Format Counters
                Bytes Read=118
2023-07-24 16:05:40,647 INFO mapred.LocalJobRunner: Finishing task: attempt_local1819974892_0001_m_000015_0
2023-07-24 16:05:40,647 INFO mapred.LocalJobRunner: map task executor complete.
2023-07-24 16:05:40,650 INFO mapred.LocalJobRunner: Waiting for reduce tasks
2023-07-24 16:05:40,650 INFO mapred.LocalJobRunner: Starting task: attempt_local1819974892_0001_r_000000_0
2023-07-24 16:05:40,659 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2023-07-24 16:05:40,659 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2023-07-24 16:05:40,664 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2023-07-24 16:05:40,668 INFO mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5c25a9e6
2023-07-24 16:05:40,670 WARN impl.MetricsSystemImpl: JobTracker metrics system already initialized!
2023-07-24 16:05:40,690 INFO reduce.MergeManagerImpl: MergerManager: memoryLimit=2610115328, maxSingleShuffleLimit=652528832, mergeThreshold=1722676224, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2023-07-24 16:05:40,692 INFO reduce.EventFetcher: attempt_local1819974892_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2023-07-24 16:05:40,724 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1819974892_0001_m_000005_0 decomp: 24 len: 28 to MEMORY
2023-07-24 16:05:40,727 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local1819974892_0001_m_000005_0
2023-07-24 16:05:40,727 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->24
2023-07-24 16:05:40,729 WARN io.ReadaheadPool: Failed readahead on ifile
EBADF: Bad file descriptor
        at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)
        at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:419)
        at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:296)
        at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:220)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)
2023-07-24 16:05:40,730 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1819974892_0001_m_000012_0 decomp: 24 len: 28 to MEMORY
2023-07-24 16:05:40,732 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local1819974892_0001_m_000012_0
2023-07-24 16:05:40,732 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 2, commitMemory -> 24, usedMemory ->48
2023-07-24 16:05:40,733 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1819974892_0001_m_000006_0 decomp: 24 len: 28 to MEMORY
2023-07-24 16:05:40,734 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local1819974892_0001_m_000006_0
2023-07-24 16:05:40,734 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 3, commitMemory -> 48, usedMemory ->72
2023-07-24 16:05:40,736 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1819974892_0001_m_000000_0 decomp: 24 len: 28 to MEMORY
2023-07-24 16:05:40,737 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local1819974892_0001_m_000000_0
2023-07-24 16:05:40,737 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 4, commitMemory -> 72, usedMemory ->96
2023-07-24 16:05:40,739 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1819974892_0001_m_000013_0 decomp: 24 len: 28 to MEMORY
2023-07-24 16:05:40,740 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local1819974892_0001_m_000013_0
2023-07-24 16:05:40,740 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 5, commitMemory -> 96, usedMemory ->120
2023-07-24 16:05:40,741 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1819974892_0001_m_000007_0 decomp: 24 len: 28 to MEMORY
2023-07-24 16:05:40,742 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local1819974892_0001_m_000007_0
2023-07-24 16:05:40,742 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 6, commitMemory -> 120, usedMemory ->144
2023-07-24 16:05:40,744 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1819974892_0001_m_000001_0 decomp: 24 len: 28 to MEMORY
2023-07-24 16:05:40,744 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local1819974892_0001_m_000001_0
2023-07-24 16:05:40,745 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 7, commitMemory -> 144, usedMemory ->168
2023-07-24 16:05:40,746 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1819974892_0001_m_000014_0 decomp: 24 len: 28 to MEMORY
2023-07-24 16:05:40,747 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local1819974892_0001_m_000014_0
2023-07-24 16:05:40,747 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 8, commitMemory -> 168, usedMemory ->192
2023-07-24 16:05:40,749 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1819974892_0001_m_000008_0 decomp: 24 len: 28 to MEMORY
2023-07-24 16:05:40,750 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local1819974892_0001_m_000008_0
2023-07-24 16:05:40,750 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 9, commitMemory -> 192, usedMemory ->216
2023-07-24 16:05:40,752 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1819974892_0001_m_000002_0 decomp: 24 len: 28 to MEMORY
2023-07-24 16:05:40,752 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local1819974892_0001_m_000002_0
2023-07-24 16:05:40,753 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 10, commitMemory -> 216, usedMemory ->240
2023-07-24 16:05:40,754 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1819974892_0001_m_000015_0 decomp: 24 len: 28 to MEMORY
2023-07-24 16:05:40,755 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local1819974892_0001_m_000015_0
2023-07-24 16:05:40,755 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 11, commitMemory -> 240, usedMemory ->264
2023-07-24 16:05:40,756 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1819974892_0001_m_000009_0 decomp: 24 len: 28 to MEMORY
2023-07-24 16:05:40,757 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local1819974892_0001_m_000009_0
2023-07-24 16:05:40,757 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 12, commitMemory -> 264, usedMemory ->288
2023-07-24 16:05:40,762 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1819974892_0001_m_000003_0 decomp: 24 len: 28 to MEMORY
2023-07-24 16:05:40,762 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local1819974892_0001_m_000003_0
2023-07-24 16:05:40,762 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 13, commitMemory -> 288, usedMemory ->312
2023-07-24 16:05:40,764 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1819974892_0001_m_000010_0 decomp: 24 len: 28 to MEMORY
2023-07-24 16:05:40,764 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local1819974892_0001_m_000010_0
2023-07-24 16:05:40,764 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 14, commitMemory -> 312, usedMemory ->336
2023-07-24 16:05:40,765 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1819974892_0001_m_000004_0 decomp: 24 len: 28 to MEMORY
2023-07-24 16:05:40,766 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local1819974892_0001_m_000004_0
2023-07-24 16:05:40,766 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 15, commitMemory -> 336, usedMemory ->360
2023-07-24 16:05:40,767 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1819974892_0001_m_000011_0 decomp: 24 len: 28 to MEMORY
2023-07-24 16:05:40,768 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local1819974892_0001_m_000011_0
2023-07-24 16:05:40,768 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 16, commitMemory -> 360, usedMemory ->384
2023-07-24 16:05:40,768 INFO reduce.EventFetcher: EventFetcher is interrupted.. Returning
2023-07-24 16:05:40,769 INFO mapred.LocalJobRunner: 16 / 16 copied.
2023-07-24 16:05:40,770 INFO reduce.MergeManagerImpl: finalMerge called with 16 in-memory map-outputs and 0 on-disk map-outputs
2023-07-24 16:05:40,777 INFO mapred.Merger: Merging 16 sorted segments
2023-07-24 16:05:40,777 INFO mapred.Merger: Down to the last merge-pass, with 16 segments left of total size: 336 bytes
2023-07-24 16:05:40,779 INFO reduce.MergeManagerImpl: Merged 16 segments, 384 bytes to disk to satisfy reduce memory limit
2023-07-24 16:05:40,779 INFO reduce.MergeManagerImpl: Merging 1 files, 358 bytes from disk
2023-07-24 16:05:40,780 INFO reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2023-07-24 16:05:40,780 INFO mapred.Merger: Merging 1 sorted segments
2023-07-24 16:05:40,780 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 351 bytes
2023-07-24 16:05:40,780 INFO mapred.LocalJobRunner: 16 / 16 copied.
2023-07-24 16:05:40,786 INFO Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
2023-07-24 16:05:40,839 INFO mapred.Task: Task:attempt_local1819974892_0001_r_000000_0 is done. And is in the process of committing
2023-07-24 16:05:40,841 INFO mapred.LocalJobRunner: 16 / 16 copied.
2023-07-24 16:05:40,841 INFO mapred.Task: Task attempt_local1819974892_0001_r_000000_0 is allowed to commit now
2023-07-24 16:05:40,853 INFO output.FileOutputCommitter: Saved output of task 'attempt_local1819974892_0001_r_000000_0' to hdfs://10.4.5.242:9000/user/hadoop/QuasiMonteCarlo_1690214736371_1624327287/out
2023-07-24 16:05:40,854 INFO mapred.LocalJobRunner: reduce > reduce
2023-07-24 16:05:40,854 INFO mapred.Task: Task 'attempt_local1819974892_0001_r_000000_0' done.
2023-07-24 16:05:40,854 INFO mapred.Task: Final Counters for attempt_local1819974892_0001_r_000000_0: Counters: 30
        File System Counters
                FILE: Number of bytes read=343123
                FILE: Number of bytes written=877368
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=1888
                HDFS: Number of bytes written=2103
                HDFS: Number of read operations=57
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=21
                HDFS: Number of bytes read erasure-coded=0
        Map-Reduce Framework
                Combine input records=0
                Combine output records=0
                Reduce input groups=2
                Reduce shuffle bytes=448
                Reduce input records=32
                Reduce output records=0
                Spilled Records=32
                Shuffled Maps =16
                Failed Shuffles=0
                Merged Map outputs=16
                GC time elapsed (ms)=0
                Total committed heap usage (bytes)=1898971136
        Shuffle Errors
                BAD_ID=0
                CONNECTION=0
                IO_ERROR=0
                WRONG_LENGTH=0
                WRONG_MAP=0
                WRONG_REDUCE=0
        File Output Format Counters
                Bytes Written=97
2023-07-24 16:05:40,860 INFO mapred.LocalJobRunner: Finishing task: attempt_local1819974892_0001_r_000000_0
2023-07-24 16:05:40,860 INFO mapred.LocalJobRunner: reduce task executor complete.
2023-07-24 16:05:41,483 INFO mapreduce.Job:  map 100% reduce 100%
2023-07-24 16:05:41,484 INFO mapreduce.Job: Job job_local1819974892_0001 completed successfully
2023-07-24 16:05:41,501 INFO mapreduce.Job: Counters: 36
        File System Counters
                FILE: Number of bytes read=5678187
                FILE: Number of bytes written=14902328
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=17936
                HDFS: Number of bytes written=32311
                HDFS: Number of read operations=529
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=309
                HDFS: Number of bytes read erasure-coded=0
        Map-Reduce Framework
                Map input records=16
                Map output records=32
                Map output bytes=288
                Map output materialized bytes=448
                Input split bytes=2390
                Combine input records=0
                Combine output records=0
                Reduce input groups=2
                Reduce shuffle bytes=448
                Reduce input records=32
                Reduce output records=0
                Spilled Records=64
                Shuffled Maps =16
                Failed Shuffles=0
                Merged Map outputs=16
                GC time elapsed (ms)=12
                Total committed heap usage (bytes)=19696451584
        Shuffle Errors
                BAD_ID=0
                CONNECTION=0
                IO_ERROR=0
                WRONG_LENGTH=0
                WRONG_MAP=0
                WRONG_REDUCE=0
        File Input Format Counters
                Bytes Read=1888
        File Output Format Counters
                Bytes Written=97
Job Finished in 3.559 seconds
Estimated value of Pi is 3.14250000000000000000

这是resource manager web interface 8088

的照片

About - resource manager web interface 8088

我尝试检查日志，但它们很好。我希望 MapReduce 作业在资源管理器 Web 界面上显示为一个条目。

hadoop mapreduce hadoop-yarn
最新问题
© www.soinside.com 2019 - 2024. All rights reserved.