如何使用Apache星火ML库进行随机森林网格搜索

Question

我想在我的随机森林模型的Apache星火进行网格搜索。但我没能找到一个例子这样做。是否有样本数据，我可以利用网格搜索做超参数整定任何的例子吗？

Answer 1

from pyspark.ml import Pipeline
from pyspark.ml.classification import RandomForestClassifier
from pyspark.ml.evaluation import BinaryClassificationEvaluator
from pyspark.ml.tuning import CrossValidator, ParamGridBuilder


rf = RandomForestClassifier(labelCol="indexedLabel", featuresCol="indexedFeatures", numTrees=10)
pipeline = Pipeline(stages=[rf])
paramGrid = ParamGridBuilder().addGrid(rf.numTrees, [10, 30]).build()

crossval = CrossValidator(estimator=pipeline,
                          estimatorParamMaps=paramGrid,
                          evaluator=BinaryClassificationEvaluator(),
                          numFolds=2) 

cvModel = crossval.fit(training_df)

超参数和网格在addGrid方法定义