由于我公司的安全义务，他们拒绝使用预构建的 hadoop.dll 和 winutils，以便将它们添加到路径中并能够在本地开发。 所以我分叉了https://github.com/apache/hadoop/tree/release-3.2.2-RC5并基于本教程使用本机二进制文件在Windows上构建和安装Hadoop-DZone
我写这个yaml脚本是为了准备教程中提到的步骤。因为我无法将 protoc 添加到系统环境变量中，所以我只是下载它并将它放在项目目录中，以便它能够找到它。我还使用了 maven protoc.exe，因为版本 2.5.0 不再存在于 protobuf 版本中。
另一件事是解决 MsBuild 版本 100 的错误和错误我不得不通过下载它并在本地重新定位它来将项目重新定位到 MSBuild v142，然后在我的远程仓库上手动修改 winutils.pojxcv 文件。
这是 yaml：
trigger:
- master
pool:
vmImage: 'windows-2019'
steps:
- task: PowerShell@2
displayName: 'Install Cygwin'
inputs:
targetType: 'inline'
script: |
choco install cygwin -y
errorActionPreference: 'stop'
- task: PowerShell@2
displayName: 'Test cygwin'
inputs:
targetType: 'inline'
script: |
uname -a
- task: PowerShell@2
displayName: 'Install Windows SDK'
inputs:
targetType: 'inline'
script: |
choco install windows-sdk-10.0 -y
errorActionPreference: 'stop'
- task : Bash@3
displayName: 'Install Protoc'
inputs:
targetType: 'inline'
script: |
#!/bin/bash
downloadUrl=https://repo1.maven.org/maven2/com/google/protobuf/protoc/2.5.0/protoc-2.5.0-windows-x86_64.exe
outputDir="$(System.DefaultWorkingDirectory)"
mkdir -p $outputDir
curl -L -o "$outputDir/protoc.exe" $downloadUrl
ls -al "$(System.DefaultWorkingDirectory)"
protocPath="$(System.DefaultWorkingDirectory)/protoc.exe"
output=$("$protocPath" --version)
if [[ "$output" =~ "libprotoc 2.5.0" ]]; then
echo "Protoc version 2.5.0 installed successfully"
else
echo "Error: Protoc version mismatch or command failed"
exit 1
fi
- task: PowerShell@2
displayName: 'List environment variables'
inputs:
targetType: 'inline'
script: |
Get-ChildItem Env: | Sort-Object Name | Format-Table Name, Value -AutoSize
- task: PowerShell@2
displayName: 'Test cMake'
inputs:
targetType: 'inline'
script: |
cmake --version
- task: Maven@3
displayName: 'Build Parent Project'
inputs:
mavenPomFile: '$(System.DefaultWorkingDirectory)\pom.xml'
options: '-Pdist,native-win -DskipTests -Dtar -X'
goals: 'clean package'
我遇到的问题是当我尝试打包解决方案时出现此错误：
INFO] Executing tasks
Build sequence for target(s) `main' is [main]
Complete build sequence is [main, ]
main:
Property "BUILD_DIR" has not been set
Property "BUILD_DIR" has not been set
Property "snappy.lib" has not been set
Property "openssl.lib" has not been set
Property "BUILD_DIR" has not been set
Property "BUILD_DIR" has not been set
Property "snappy.lib" has not been set
Property "openssl.lib" has not been set
[exec] Current OS is Windows Server 2019
[exec] Executing 'bash' with arguments:
[exec] './dist-copynativelibs.sh'
[exec]
[exec] The ' characters around the executable and arguments are
[exec] not part of the command.
Execute:Java13CommandLauncher: Executing 'bash' with arguments:
'./dist-copynativelibs.sh'
The ' characters around the executable and arguments are
not part of the command.
[exec] Windows Subsystem for Linux has no installed distributions.
[exec] Distributions can be installed by visiting the Microsoft Store:
[exec] https://aka.ms/wslstore
[exec]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Apache Hadoop Main 2.8.0-SNAPSHOT:
[INFO]
[INFO] Apache Hadoop Main ................................. SUCCESS [ 15.697 s]
[INFO] Apache Hadoop Build Tools .......................... SUCCESS [ 4.240 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [ 4.108 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [ 13.099 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [ 0.125 s]
[INFO] Apache Hadoop Project Dist POM ..................... FAILURE [ 0.954 s]
[INFO] Apache Hadoop Maven Plugins ........................ SKIPPED
[INFO] Apache Hadoop MiniKDC .............................. SKIPPED
[INFO] Apache Hadoop Auth ................................. SKIPPED
[INFO] Apache Hadoop Auth Examples ........................ SKIPPED
[INFO] Apache Hadoop Common ............................... SKIPPED
.....
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (pre-dist) on project hadoop-project-dist: An Ant BuildException has occured: exec returned: -1
[ERROR] around Ant part ...<exec failonerror="true" dir="D:\a\1\s\hadoop-project-dist\target" executable="bash">... @ 41:88 in D:\a\1\s\hadoop-project-dist\target\antrun\build-main.xml
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (pre-dist) on project hadoop-project-dist: An Ant BuildException has occured: exec returned: -1
around Ant part ...<exec failonerror="true" dir="D:\a\1\s\hadoop-project-dist\target" executable="bash">... @ 41:88 in D:\a\1\s\hadoop-project-dist\target\antrun\build-main.xml
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:375)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:298)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:960)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: exec returned: -1
around Ant part ...<exec failonerror="true" dir="D:\a\1\s\hadoop-project-dist\target" executable="bash">... @ 41:88 in D:\a\1\s\hadoop-project-dist\target\antrun\build-main.xml
at org.apache.maven.plugin.antrun.AntRunMojo.execute (AntRunMojo.java:355)
非常感谢您的帮助。我可以看到它无法在 antrun-plugin 下的 pom.xml 中执行 bash 脚本，但为什么？这是一个cygwin问题吗？还是我错过了什么？ 谢谢