水槽-无法配置水槽-没有为水槽配置通道

Question

我已经配置了水槽以读取日志文件并写入HDFS。启动水槽时，会读取日志文件，但不会将其写入HDFS。 flume.log发出警告消息-could not configure sink - no channel configured for sink，但我已经分配了一个要下入conf文件的通道。

以下是配置文件和错误消息：

File: spool-to-hdfs.properties
# List all components.
agent1.sources = source1
agent1.sinks = sink1
agent1.channels = channel1

# Describe source.
agent1.sources.source1.type = spooldir
agent1.sources.source1.spoolDir =/Suriya/flume/input_files

# Describe channel
#agent1.channels.channel1.type = file
#agent1.channels.channel1.checkpointDir = /Suriya/flume/checkpointDir
#agent1.channels.channel1.dataDirs =/Suriya/flume/dataDirs
agent1.channels.channel1.type = memory

# Describe sink
agent1.sinks.sink1.type = hdfs
#agent1.sinks.sink1.hdfs.path = hdfs://sandbox.hortonworks.com:8020/hdfs/Suriya/flume
agent1.sinks.sink1.hdfs.path = hdfs://localhost/hdfs/Suriya/flume
agent1.sinks.sink1.hdfs.fileType= DataStream
agent1.sinks.sink1.hdfs.writeFormat = Text

# Bind source and sink to channel
agent1.sources.source1.channels = channel1
agent1.sinks.sink1.channels = channel1

**-- starting the agent**
flume-ng agent --conf-file spool-to-hdfs.properties --conf /etc/flume/conf --name agent1;

flume.log

03 Aug 2015 23:37:16,699 WARN  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks:697)  - Could not configure sink  sink1 due to: No channel configured for sink: sink1
org.apache.flume.conf.ConfigurationException: No channel configured for sink: sink1 at org.apache.flume.conf.sink.SinkConfiguration.configure(SinkConfiguration.java:51)

Answer 1

Replace the bind part with.

# Bind source and sink to channel
agent1.sources.source1.channels = channel1
agent1.sinks.sink1.channels = channel1

This bind config
# Bind source and sink to channel
agent1.sources.source1.channels = channel1
agent1.sinks.sink1.channel = channel1


agent1.sources.source1.channels = channel1 
Looks okay

BUT
agent1.sinks.sink1.channels = channel1
Should be 
agent1.sinks.sink1.channel = channel1

让我们知道进展如何。

Answer 2

我对此conf文件也有相同的问题。

# Naming the components on the current agent. 
TwitterAgent.sources = Twitter 
TwitterAgent.channels = MemChannel 
TwitterAgent.sinks = 

List item


# Describing/Configuring the source 
TwitterAgent.sources.Twitter.type = org.apache.flume.source.twitter.TwitterSource
TwitterAgent.sources.Twitter.consumerKey = 
TwitterAgent.sources.Twitter.consumerSecret = 
TwitterAgent.sources.Twitter.accessToken = 
TwitterAgent.sources.Twitter.accessTokenSecret = 
TwitterAgent.sources.Twitter.keywords = bigdata

# Describing/Configuring the sink 

TwitterAgent.sinks.HDFS.type = hdfs 
TwitterAgent.sinks.HDFS.hdfs.path = hdfs://localhost:9000/Tweets
TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream 
TwitterAgent.sinks.HDFS.hdfs.writeFormat = Text 
TwitterAgent.sinks.HDFS.hdfs.batchSize = 1000
TwitterAgent.sinks.HDFS.hdfs.rollSize = 0 
TwitterAgent.sinks.HDFS.hdfs.rollCount = 10000 

# Describing/Configuring the channel 
TwitterAgent.channels.MemChannel.type = memory 
TwitterAgent.channels.MemChannel.capacity = 10000 
TwitterAgent.channels.MemChannel.transactionCapacity = 100


# Binding the source and sink to the channel 
     TwitterAgent.sources.Twitter.channels = MemChannel
     TwitterAgent.sinks.HDFS.channel = MemChannel