Spock与DbUnit测试用例缓慢

问题描述 投票:3回答:3

我们已经实施了Spock + Db单元框架作为自动化单元测试的一部分。

我们现在有2000个测试用例(功能),用于150个规格和DbUnit。

在这里,我们在DB中添加所需的条目,然后测试每个方法的行为。

我们观察到,执行这些测试用例需要大约2小时和30分钟。

我有时间戳setup夹具并在特征方法中添加时间戳。以下是我的观察:

allergy.dao.AllergyFormDAOSpec > Get Allergy Form STANDARD_OUT
    setup method execution started at : Fri Jan 12 19:00:42 IST 2018

allergy.dao.AllergyFormDAOSpec > API to get Allergy Form STANDARD_OUT
Feature method execution started at : Fri Jan 12 19:00:44 IST 2018
Feature method execution ended at: Fri Jan 12 19:00:45 IST 2018
Total time taken to run the one test case: 242

cleanup method execution started at : Fri Jan 12 19:00:45 IST 2018
Total time taken to run a feature method : 2531

在这里,我观察到在安装后加载一个特征方法平均需要2-4秒。但是,原始测试用例执行时间不到一秒。

我想知道我是否能指出这里有什么延迟?因为,2000个测试用例的3秒意味着Spock所花费的时间几乎是1小时和30分钟,而不是真正的功能执行。

总结一下,我们希望减少Spock测试用例在每天运行时所花费的总时间。

规格

package allergy.dao

import java.util.Date

import org.dbunit.IDatabaseTester;
import org.dbunit.ext.mssql.InsertIdentityOperation;

import allergy.AllergyForm;
import be.janbols.spock.extension.dbunit.DbUnit;
import spock.lang.Shared
import util.MasterSpec

class AllergyFormDAOSpec extends MasterSpec {
    def dao = new AllergyFormDAO();
    @Shared Date timeStart1
    @Shared Date timeEnd1

@DbUnit(configure={ IDatabaseTester it ->
    it.setUpOperation = InsertIdentityOperation.REFRESH
    it.tearDownOperation = InsertIdentityOperation.DELETE
})
def content =  {
    allergy_form(formId:99999,formName:'DummySpockForm',displayIndex:1,deleteFlag:0,is_biological:1)
    allergy_form_facilities(id:99999,formId:99999,facilityid:2)
    form_concentration(id:99999,formId:99999,name:'1:100',deleteflag:0,displayindex:1)
}

def setup(){
    timeStart1 = new Date()
    println "setup method execution started at : " +  timeStart1
}

def "API to test delete Form facility"(){
    def startTime = new Date()
    println "Feature method execution started at : " +  startTime
    given:"form Id is given"
        def formId = 99999
    when:"delete form facilities"
        def result =dao.deleteFormFacilities(null, formId)
    then:"validate result"
        (result>0)==true
        def endTime = new Date()
        println "Feature method execution ended at: " +  endTime
        println 'Total time taken to run the one test case: '+ (endTime.getTime() - startTime.getTime())
}

def cleanup() {
    timeEnd1 = new Date()
    println "cleanup method execution started at : " +  timeEnd1

    def difference = timeEnd1.time - timeStart1.time
    println "Total time taken to run a fixture method : " + difference
}
}

MASTERSPEC

package util

import com.ecw.dao.SqlTranslator
import catalog.Root
import spock.lang.Shared
import spock.lang.Specification

import javax.sql.DataSource

/**

 */
class MasterSpec extends Specification {

@Shared
Properties properties = new Properties()
@Shared
public DataSource dataSource
@Shared
protected xmlDataSource = [:]

static int timeCntr = 0;

//setup is to read xml file's content in xmlDataSource Hashmap
def setup(){

    //Get Running Class name without its package
    def className = this.class.name.substring(this.class.name.lastIndexOf('.') + 1)
    def resourceAnno = specificationContext.currentFeature.featureMethod.getAnnotation(FileResource)

    if(resourceAnno != null){
        def files = resourceAnno.xmlFiles()
        def packageName = (this.class.package.name).replaceAll('\\.','/')

        for(int i=0;i< files.length;i++){
            def f = new File("src/test/resources/"+packageName+"/"+className+"/"+files[i])
            def engine = new groovy.text.GStringTemplateEngine()
            def template = engine.createTemplate(f).make(null)
            def xmlString = template.toString()

            //load the hashmap with file name as Key and its content in form of string as Value
            xmlDataSource.put(files[i].split("\\.")[0],xmlString)
        }
    }
}

def setupSpec() {
    Date timeStart = new Date()

    File propertiesFile = new File('src/test/webapps/myApp/conf/connection.properties').withInputStream {
        properties.load it
    }

    String strDBName = getPropertyValue("myApp.DBName")
    if(strDBName.indexOf('?') > -1){
        strDBName = strDBName.substring(0, strDBName.indexOf('?'))
    }
    String strServerName = getPropertyValue("myApp.DBHost");
    if(strServerName.indexOf(':') > -1){
        strServerName = strServerName.substring(0, strServerName.indexOf(':'))
    }
    String strUrl = getPropertyValue("myApp.DBUrl")
    String strPort = strUrl.substring(strUrl.lastIndexOf(':') + 1)

    //FOR MSSQL
    System.setProperty("myApp.SkipJndi", "yes")
    //dataSource = new JtdsDataSource()
    Object newObject = null;
    if(SqlTranslator.isDbSqlServer()){
        newObject = Class.forName("net.sourceforge.jtds.jdbcx.JtdsDataSource").newInstance()
    } else if(SqlTranslator.isDbMySql()){
        newObject = Class.forName("com.mysql.jdbc.jdbc2.optional.MysqlDataSource").newInstance()
    }

    dataSource = (DataSource)newObject
    dataSource.setDatabaseName(strDBName)
    dataSource.setUser(getPropertyValue("myApp.DBUser"))
    dataSource.setPassword(getPropertyValue("myApp.DBPassword"))
    dataSource.setServerName(strServerName)
    dataSource.setPortNumber(Integer.parseInt(strPort))

}
}
unit-testing spock dbunit
3个回答
2
投票

这个问题确实过于宽泛,无法提供足够的信息来提供合格的答案。但是,我可以说Spock本身应该非常快(不像'原始'JUnit那么快,毕竟它仍然很常规,但真的足够快,无法进行测试)。

从您的问题来看,您似乎怀疑Spock是一个瓶颈,所以,例如,您可以放置​​一个空的Spock测试并在DbUnit启用的测试旁边测量它的执行,我可以向您保证,您将获得的时间将是微不足道的。

所以我认为原因是在安装/清理期间,DbUnit调用了一些与数据库相关的代码(可能是模式生成/表格填充和/或删除),这需要花费很多时间。因此,我的第二次尝试只是打印在测试期间运行的SQL查询,可能您会发现其中许多在安装方法期间运行。

另一个可能的原因是,在进行测试时,在测试之前插入的数据太多了。

而另一个可能的原因是你正在运行测试的数据库太慢(网络速度慢,数据库本身太忙了)。

现在有什么解决方案呢? :)您可能想看看Spring的方法来测试数据访问层+如何进行初始设置。由于它远远超出了问题的范围,我不会在这里谈论春天太多,但只是作为一个想法:

  • 生成一次数据
  • 在所有测试运行之前插入它
  • 随着测试的开始,开始一个交易
  • 在测试完成时回滚事务(即使测试成功),这样数据也不会被保留。如果您使用隔离并且数据库服务器支持它,您可以并行运行测试,没有问题。

如果缓慢的原因是数据库服务器,那么(除了明显的建议,如“更改您的RDBMS”),您可以尝试在同一台机器中的docker中运行数据库/甚至在使用TestContainers在本地测试开始之前启动数据库。


1
投票

答案实际上非常简单:您忘记测量主规格的设置时间。也许你假设儿童规范中的setup()方法会覆盖其父规范的setup()方法。但在Spock中却没有!所有的

  • setupSpec()
  • setup()
  • cleanup()
  • cleanupSpec()

类层次结构中的所有规范将按以下顺序执行:首先是基类,然后是子类。

让我用一个原始的例子告诉你我在说什么:

主规格:

为什么是线程局部静态变量?好吧,也许你正在同时运行你的测试。对于这个简单的例子,没有必要。

package de.scrum_master.stackoverflow

import spock.lang.Specification
import static System.currentTimeMillis

class MasterSpec extends Specification {
  static ThreadLocal<Long> startMillis = new ThreadLocal<>()

  def setupSpec() {
    startMillis.set(currentTimeMillis())
    sleep 50
    println "BaseSpec.setupSpec: " + (currentTimeMillis() - startMillis.get())
  }

  def cleanupSpec() {
    sleep 50
    println "BaseSpec.cleanupSpec: " + (currentTimeMillis() - startMillis.get())
  }

  def setup() {
    sleep 50
    println "BaseSpec.setup: " + (currentTimeMillis() - startMillis.get())
  }

  def cleanup() {
    sleep 50
    println "BaseSpec.cleanup: " + (currentTimeMillis() - startMillis.get())
  }
}

衍生规格:

为什么叫DerivedTest这个名字?只是因为我的Maven构建配置的方式是基于默认名称*Test(Surefire单元测试)或*IT(故障安全集成测试)查找测试。

package de.scrum_master.stackoverflow

import spock.lang.Unroll

import static java.lang.System.currentTimeMillis

class DerivedTest extends MasterSpec {
  def setupSpec() {
    sleep 50
    println "DerivedTest.setupSpec: " + (currentTimeMillis() - startMillis.get())
  }

  def cleanupSpec() {
    sleep 50
    println "DerivedTest.cleanupSpec: " + (currentTimeMillis() - startMillis.get())
  }

  def setup() {
    sleep 50
    println "DerivedTest.setup: " + (currentTimeMillis() - startMillis.get())
  }

  def cleanup() {
    sleep 50
    println "DerivedTest.cleanup: " + (currentTimeMillis() - startMillis.get())
  }

  @Unroll
  def "feature #id"() {
    given:
    long featureStartMillis = currentTimeMillis()
    sleep 50
    println "DerivedTest.feature $id: " + (currentTimeMillis() - startMillis.get())

    expect:
    true

    cleanup:
    println "DerivedTest.feature $id ONLY: " + (currentTimeMillis() - featureStartMillis)

    where:
    id << ["A", "B"]
  }
}

控制台日志:

BaseSpec.setupSpec: 105
DerivedTest.setupSpec: 193
BaseSpec.setup: 286
DerivedTest.setup: 336
DerivedTest.feature A: 396
DerivedTest.feature A ONLY: 55
DerivedTest.cleanup: 453
BaseSpec.cleanup: 504
BaseSpec.setup: 556
DerivedTest.setup: 606
DerivedTest.feature B: 656
DerivedTest.feature B ONLY: 50
DerivedTest.cleanup: 706
BaseSpec.cleanup: 757
DerivedTest.cleanupSpec: 808
BaseSpec.cleanupSpec: 858

你能看到执行的顺序以及每一步消耗的时间吗?

我会说,你在主规范中的复杂操作(读取配置文件,初始化和填充数据库等)


1
投票

为了分析这个问题的根本原因,我们确实在下面的场景中运行了一个差事:

1)没有任何Db或模拟依赖项的1000个Spock测试用例(PowerMock)

示例代码解释方案:

package mathOperations;

import groovy.lang.Closure
import mathOperations.Math
import spock.lang.Specification

class MathSpec extends Specification {
    def objMath =new Math()

    def "API to test addition of two numbers"() {
        given :"a and b"
            def a=10
            def b=5
        when: "Math.AddNumber is called with given values"
            def result =objMath.addNumber(a,b)
        then: "Result should be 15"
            result==15
    }

    def "API to test subrtaction of two numbers"() {
        given :"a and b"
            def a=10
            def b=5
        when: "Math.subtractNumber is called with given values"
            def result =objMath.subtractNumber(a, b)
        then: "Result should be 5"
            result==5
    }

    def "API to test multiplication of two numbers"() {
        given :"a and b"
            def a=10
            def b=5
        when: "Math.multiplyNumber is called with given values"
            def result =objMath.multiplyNumber(a,b)
        then: "Result should be 50"
            result==50
    }
    def "API to test division two numbers"() {
        given :"a and b"
            def a=10
            def b=5
        when: "Math.divisionNumber is called with given values"
            def result =objMath.divisionNumber(a,b)
        then: "Result should be 2"
            result==2
    }

    def "API to test whether given both numbers are equal - Affirmative"() {
        given :"a and b"
            def a=10
            def b=10
        when: "Math.equalNumber is called with given values"
            def result =objMath.equalNumber(a,b)
        then: "It should return true"
            result==true
    }

    def "API to test whether given both numbers are equal - Negative"() {
        given :"a and b"
            def a=10
            def b=11
        when: "Math.equalNumber is called with given values"
            def result =objMath.equalNumber(a,b)
        then: "It should return false"
            result==false
    }
}

- >花费了25.153秒,包括构建时间,下面是下面的报告

Spock test cases without any Db or mocking dependencies

2)1000个带有模拟的Spock测试用例(PowerMock)

示例代码解释方案:

package mathOperations;

import groovy.lang.Closure
import mathOperations.Math
import spock.lang.Specification
import org.powermock.core.classloader.annotations.PrepareForTest
import org.powermock.modules.junit4.rule.PowerMockRule
import org.junit.Rule
import utils.QRCodeUtils
import org.powermock.api.mockito.PowerMockito
import static org.powermock.api.mockito.PowerMockito.mockStatic
import static org.mockito.BDDMockito.*

@PrepareForTest([QRCodeUtils.class])
class MathSpec extends Specification {
    def objMath =new Math()

    @Rule PowerMockRule powerMockRule = new PowerMockRule()

    def "API to test add two numbers"() {
        given :"a and b"
        def a=10
        def b=5
        when: "Math.AddNumber is call with given values"
        mockGetOTPAttemptStatus(5)
        def result =objMath.addNumber(a,b)
        then: "result should be 15"
        result==15
    }

    def "API to test subract two numbers"() {
        given :"a and b"
        def a=10
        def b=5
        when: "Math.subtractNumber is call with given values"
        mockGetOTPAttemptStatus(5)
        def result =objMath.subtractNumber(a, b)
        then: "result should be 5"
        result==5
    }
    def "API to test multiple two numbers"() {
        given :"a and b"
        def a=10
        def b=5
        when: "Math.multiplyNumber is call with given values"
        mockGetOTPAttemptStatus(5)
        def result =objMath.multiplyNumber(a,b)
        then: "result should be 50"
        result==50
    }
    def "API to test divide two numbers"() {
        given :"a and b"
        def a=10
        def b=5
        when: "Math.divisionNumber is call with given values"
        mockGetOTPAttemptStatus(5)
        def result =objMath.divisionNumber(a,b)
        then: "result should be 2"
        result==2
    }
    def "API to test modulo of a number"() {
        given :"a and b"
        def a=10
        def b=5
        when: "Math.moduloNumber is call with given values"
        mockGetOTPAttemptStatus(5)
        def result =objMath.moduloNumber(a,b)
        then: "result should be 0"
        result==0
    }

    def "API to test power of a number"() {
        given :"a and b"
        def a=10
        def b=2
        when: "Math.powerofNumber is call with given values"
        mockGetOTPAttemptStatus(5)
        def result =objMath.powerofNumber(a,b)
        then: "result should be 0"
        result==8
    }

    def "API to test numbers are equal -affirmative"() {
        given :"a and b"
        def a=10
        def b=10
        when: "Math.equalNumber is call with given values"
        mockGetOTPAttemptStatus(5)
        def result =objMath.equalNumber(a,b)
        then: "It should return true"
        result==true
    }

    def "API to test numbers are equal -negative"() {
        given :"a and b"
        def a=10
        def b=11
        when: "Math.equalNumber is call with given values"
        mockGetOTPAttemptStatus(5)
        def result =objMath.equalNumber(a,b)
        then: "It should return false"
        result==false
    }

    void mockGetOTPAttemptStatus(int status) {
        mockStatic(QRCodeUtils.class)
        when(QRCodeUtils.getOTPAttemptStatus(anyInt())).thenReturn(status)
    }
}

- >花了9分钟14.222秒,包括构建时间,下面是报告

Spock test cases with mocking

3)仅有Dbunit的1000个Spock测试用例。 (通常,我们在测试用例中插入平均15-20个表项。这里,我们添加了相同的)

示例代码解释方案:

package mathOperations;

import groovy.lang.Closure
import java.sql.Statement
import mathOperations.Math
import spock.lang.Shared
import util.BaseSpec
import catalog.Root
import spock.lang.Ignore
import org.dbunit.ext.mssql.InsertIdentityOperation
import be.janbols.spock.extension.dbunit.DbUnit
import org.dbunit.IDatabaseTester

class MathSpec extends BaseSpec {

    @Shared root
    def objMath =new Math()

    @DbUnit(configure={
        IDatabaseTester it ->
        it.setUpOperation = InsertIdentityOperation.REFRESH
        it.tearDownOperation = InsertIdentityOperation.DELETE
    })
    def content =  {
        table1(id:99,MasterFile:'UnitTestFile',DataElementName:'test',DataElementDBColName:'TestDbCol',DataElementTableName:'TestTable')
        table2(id:99,MasterFile:'UnitTestFile',DataElementName:'test',DataElementDBColName:'TestDbCol',DataElementTableName:'TestTable')
        table3(id:99,Code:'T00.0',Status:'A',LongDesc:'Unit Testing')
        table4(id:99,Code:'T00.0',Status:'A',LongDesc:'Unit Testing')
        table5(id:99,Code:'T00.0',Status:'A',LongDesc:'Unit Testing')
        table6(id:99,Code:'T00.0',Status:'A',LongDesc:'Unit Testing')
        table7(id:99,Code:'T00.0',Status:'A',LongDesc:'Unit Testing')
        table8(id:99,Code:'T00.0',Status:'A',LongDesc:'Unit Testing')
        table9(id:99,Code:'T00.0',Status:'A',LongDesc:'Unit Testing')
        table10(id:99,Code:'T00.0',Status:'A',LongDesc:'Unit Testing')
    }

    def "API to test addition of two numbers"() {
        given :"a and b"
            def a=10
            def b=5
        when: "Math.AddNumber is called with given values"
            def result =objMath.addNumber(a,b)
        then: "Result should be 15"
            result==15
    }

    def "API to test subrtaction of two numbers"() {
        given :"a and b"
            def a=10
            def b=5
        when: "Math.subtractNumber is called with given values"
            def result =objMath.subtractNumber(a, b)
        then: "Result should be 5"
            result==5
    }

    def "API to test multiplication of two numbers"() {
        given :"a and b"
            def a=10
            def b=5
        when: "Math.multiplyNumber is called with given values"
            def result =objMath.multiplyNumber(a,b)
        then: "Result should be 50"
            result==50
    }
    def "API to test division two numbers"() {
        given :"a and b"
            def a=10
            def b=5
        when: "Math.divisionNumber is called with given values"
            def result =objMath.divisionNumber(a,b)
        then: "Result should be 2"
            result==2
    }

    def "API to test whether given both numbers are equal - Affirmative"() {
        given :"a and b"
            def a=10
            def b=10
        when: "Math.equalNumber is called with given values"
            def result =objMath.equalNumber(a,b)
        then: "It should return true"
            result==true
    }

    def "API to test whether given both numbers are equal - Negative"() {
        given :"a and b"
            def a=10
            def b=11
        when: "Math.equalNumber is called with given values"
            def result =objMath.equalNumber(a,b)
        then: "It should return false"
            result==false
    }
}

- >花了57分钟18.136秒,包括建造时间,下面是报告Spock test cases with only Dbunit

我们得出一个结论,Spock没有花时间运行测试用例,但Power Mock做了仪器qazxsw poi和DbUnit(使用反射加载一切)是瓶颈。

解决方案:我们将使用内部框架来插入/删除数据库数据而不是DbUnit。此外,我们用SO Link取代了Power Mock

最终结果:正如我在问题中发布的总时间从2小时和30分钟开始,现在已经减少到这1000个测试用例的6分钟。 :)

© www.soinside.com 2019 - 2024. All rights reserved.