如何重试失败的 Android 仪器测试？

Question

我正在寻找一种方法来重试运行失败的 Android 仪器测试或称为集成测试/ui 测试。我认为可以有 3 种方法来做到这一点。

• 使用 Jenkins 阶段脚本重试 -> 检索所有失败的测试并使用 run gradle 命令再次执行它们

  ./gradlew clean app:connectedVariantNameAndroidTest

  -> 我还没有尝试过这种方法。

• 插件：我尝试过Gordon，但是，它的最低 Java 支持版本是 17，与我们的 CI 机器（Java 11）不符

• 在代码中实现为一些辅助类：尝试了这个post->当仪器测试失败时，重试没有发生。

我认为重试仪器化失败的测试通常在测试自动化中使用，因为仪器化测试不稳定且容易失败。如果您有任何方法可以实现此目的，请帮忙。

Answer 1

您可以从任何阶段重新启动声明式管道。如果您保留构建工作区，并且错误出现在管道执行之外，而不是在代码、测试或依赖项中，那么它将起作用。在这种情况下，您需要更新您的来源。 在某些情况下，您可能需要使用

gradle(dotnet mvn etc ...) clean

如果您的代码必须重新编译，您可以将其放在

post{always {}}

步骤中。

---

示例

詹金斯这边

1. 简单的例子：

pipeline {
    agent {label 'linux'}

    stages {
        stage('Random exit code') {
            steps {
                script{

// random result implementation based on exit code
                    def status = sh(
                                    returnStdout: true,
                                    script:'''
                                            bash -c 'echo $(( $RANDOM % 2 ))'
                                            '''
                                    ).toInteger()
                    print("script status: $status")
                    if (status != 0) {
                      currentBuild.result = 'FAILED'
                    }
                }
            }
        }
    }
    post{
        success{
            echo 'ok'
        }
        failure{
            echo 'fail'
// rebuild same pipeline
            build  wait: false, job: '/dev-env/simple_example'
        }
            
    }
}

詹金斯邮报

1. 声明式管道阶段重启的高级示例：

管道：

pipeline {
    agent {label 'linux'}
    stages {
        stage('build') {
            steps {
                sh 'touch testfile'
            }
        }
        stage('parralel tests') {
            parallel{
                stage('unit test'){
                    steps{
                        sh "echo '${STAGE_NAME} stage' >> testfile"
                        sh 'cat testfile'
                    }
                }
                stage('smoke test'){
                    steps{
                        sh "echo '${STAGE_NAME} stage' >> testfile"
                        sh 'cat testfile'
                    }
                }
                stage('postman api test'){
                    steps{
                        sh "echo '${STAGE_NAME} stage' >> testfile"
                        sh 'cat testfile'
                    }
                }
            }
        }
        stage('integration tests') {
            steps {
                sh "echo '${STAGE_NAME} stage' >> testfile"
                sh 'cat testfile'
            }
        }
    }
    
    post{
        success{
            // cleanWs() 
            // cleanWs()  commented becouse this build always success.
            // in real world you need to cleanup your workspace if run is OK  
            sh 'cat testfile'
        }
        failure{
            sh 'cat testfile'
            echo 'something is failed'
        }
            
    }
}

输出：

11:58:49  + echo integration tests stage
11:58:49  [Pipeline] sh
11:58:49  + cat testfile
11:58:49  unit test stage
11:58:49  smoke test stage
11:58:49  postman api test stage
11:58:49  integration tests stage
11:58:49  [Pipeline] }
11:58:49  [Pipeline] // stage
11:58:49  [Pipeline] stage
11:58:49  [Pipeline] { (Declarative: Post Actions)
11:58:49  [Pipeline] sh
11:58:49  + cat testfile
11:58:49  unit test stage
11:58:49  smoke test stage
11:58:49  postman api test stage
11:58:49  integration tests stage <<<<
11:58:49  [Pipeline] }
11:58:49  [Pipeline] // stage
11:58:49  [Pipeline] }
11:58:49  [Pipeline] // node
11:58:49  [Pipeline] End of Pipeline
11:58:50  Notified JIRA that a build has completed.
11:58:50  Finished: SUCCESS

重播阶段： 重播输出：

12:02:32  + echo integration tests stage
12:02:32  [Pipeline] sh
12:02:32  + cat testfile
12:02:32  unit test stage
12:02:32  smoke test stage
12:02:32  postman api test stage
12:02:32  integration tests stage
12:02:32  integration tests stage
12:02:32  [Pipeline] }
12:02:32  [Pipeline] // stage
12:02:32  [Pipeline] stage
12:02:32  [Pipeline] { (Declarative: Post Actions)
12:02:32  [Pipeline] sh
12:02:33  + cat testfile
12:02:33  unit test stage
12:02:33  smoke test stage
12:02:33  postman api test stage
12:02:33  integration tests stage
12:02:33  integration tests stage
12:02:33  [Pipeline] }
12:02:33  [Pipeline] // stage
12:02:33  [Pipeline] }
12:02:33  [Pipeline] // node
12:02:33  [Pipeline] End of Pipeline
12:02:33  Notified JIRA that a build has completed.
12:02:33  Finished: SUCCESS

说明：

查看后期评论 您可以看到集成测试阶段在同一环境中运行。这意味着您不需要重新启动二进制编译的构建阶段

12:02:33  + cat testfile
12:02:33  unit test stage
12:02:33  smoke test stage
12:02:33  postman api test stage
12:02:33  integration tests stage
12:02:33  integration tests stage <<<<<<<

如果您愿意，您可以编译简单和高级的示例。但我觉得这有点过分了

摇篮侧

您可以在前后包装器中添加一些逻辑来检查测试状态并在某些条件下重播案例。不幸的是，gradle 测试重试插件不支持Android Instrumentation 测试。