从 Nose2 插件跳过单元测试

问题描述 投票:0回答:2

实际上我无法从 Nose2 插件中跳过单元测试。我能够将测试标记为已跳过并在最终结果中查看原因,但测试仍在运行。只要插件处于活动状态,这个示例代码基本上应该跳过任何测试。

from nose2.events import Plugin

class SkipAllTests(Plugin):
    def startTest(self, event):
        event.result.addSkip(event.test, 'skip it')
        event.handled = True

如果我调用

event.test.skipTest('reason')
它实际上会像它应该的那样引发
SkipTest
异常,只是异常没有被测试运行器捕获,它只是在我的
startTest
钩子方法内部引发。有什么想法吗?

python unit-testing nose nose2
2个回答
2
投票

我不认为你真的可以用

startTest
钩子阻止测试运行。
nose2
文档
建议使用
matchPath
getTestCaseNames
来执行此操作。这是一个使用
matchPath
的工作示例:

from nose2.events import Plugin

class SkipAllTests(Plugin):
    configSection = "skipper"
    commandLineSwitch = (None, 'skipper', "Skip all tests")

    def matchPath(self, event):
        event.handled = True
        return False

matchPath
文档实际上明确解释了如何使用它来停止测试运行:

插件可以使用这个钩子来防止 python 模块被加载 由测试加载器或强制它们由测试加载器加载。放 event.handled 为 True 并返回 False 使加载程序跳过 模块。

使用此方法将阻止加载测试用例。如果您希望测试以跳过的形式实际显示在列表中,而不是根本不显示在测试列表中,您可以使用

StartTestEvent
:

做一些 hackery
def dummy(*args, **kwargs):
    pass

class SkipAllTests(Plugin):
    configSection = "skipper"
    commandLineSwitch = (None, 'skipper', "Skip all tests")
    def startTest(self, event):
        event.test._testFunc = dummy
        event.result.addSkip(event.test, 'skip it')
        event.handled = True

在这里,我们用一个什么都不做的虚拟函数替换了测试将要运行的实际函数。这样,当测试执行时,它不会执行任何操作,然后报告它已被跳过。


0
投票

我遇到了同样的问题......我的解决方案是:

class SkipMe :
"""
Use this module together with nose2 and preferably with the 'with such.A() ...' construct.

Synopsis          :
    import nose2.tools.such as such
    import inspect
    import functools
    import unittest

    with such.A( 'thingy') as it :                  # Create a SkipMe object.
        skipme = SkipMe( 'my_id' )                  # You can give this section/chapter a name
        skipme = SkipMe()                           # or just leave it as 'Default'.

        @it.has_setup                               # Integrate to test setup to skip all tests if you like.
        def setup() :
            skipme.skip_all()                       # To skip every test in here.
            skipme.skip_all( 'for some reason' )    # Same but give a reason in verbose mode.
            ...

        @it.has_test_setup                          # Recommended to integrate into setup for every test.
        def test_setup() :
            skipme.skip_if_reason()                 # Skip test if an overall skip reason was set.
            ...

        @it.should( "just be skipped" )
        @skipme.skip_reg( reason='I want it that way' ) # Intentionally skip a single test
        def test_skipping():
            ...

        @it.should( "do something basic")
        @skipme.skip_reg( skip_all_on_failed=True ) # Register this test. If it fails, skip all consecutive tests.
        def test_basic_things() :
            ...

        @it.should( "test something")
        @skipme.skip_reg()                          # Register this test. It will be marked as passed or failed.
        def test_one_thing() :
            ...

        @it.should( "test another thing")           # Skip this test if previous 'test_one_thing' test failed.
        @skipme.skip_reg( func_list=['test_one_thing'] )
        def test_another_thing() :                  # 'skipme.helper' is a unittest.TestCase object.
            skipme.helper.assertIs( onething, otherthing, 'MESSAGE' )
            ...

        it.createTests( globals() )

Purpose :   Have a convenient way of skipping tests in a nose2 'whith such.A() ...' construct.
            As a bonus you have with the 'object.helper' method a 'unittest.TestCase' object.
            You can use it to have all the fine assert functions at hand, like assertRaisesRegexp.

Initialize object with:
    :param  chapter :   A string as identifier for a chapter/section.

Defaults:
    chapter :   'Default'

Prerequisites:  import nose2.tools.such as such, unittest, inspect, functools

Description:
    Basically this class has an internal dict that sets and browses for values.
    The dict looks like:
        {
            'IDENTIFYER_1' : {                          # This is the value of 'chapter'.
                '__skip_all__'  :   BOOLEAN|FUNC_NAME,  # Skip all consecutive tests if this is True or a string.
                'func1' :   BOOLEAN,                    # Register functions as True (passed) or False (failed).
                'func2' :   BOOLEAN,                    # Also skipped tests are marked as failed.
                ...
            },
            'IDENTIFYER_1' : { ... },                   # This is the domain of another SkipMe object with
            ...                                         # a different value for 'chapter'
        }

    It provides a decorator 'object.skip_reg' to decorate test functions to register them to the class,
    meaning updating the internal dict accordingly.
    Skipped tests are marked as failed in this context.

    Skipping all tests of a 'with such.A()' construct:
        Integrate it into the setup and the test setup like so:

            with such.A( 'thingy') as it :
                skipme = SkipMe()

                @it.has_setup
                def setup() :
                    skipme.skip_all()

                @it.has_test_setup
                def test_setup() :
                    skipme.skip_if_reason()

    If you intend to skip all tests or all consecutive tests after a special test failed,
    you need only the '@it.has_test_setup' part.

    Register tests with the 'skip_reg' method:

        Decorate the test functions with the 'object.skip_reg' method under the @it.should decorator.
            Example:

            with such.A( 'thingy') as it :
                skipme = SkipMe()
                # Same setup as above
                ...

                @it.should( "Do something")
                @skipme.skip_reg()                                  # Just register this function.
                @skipme.skip_reg( reason='SOME REASON' )            # Skip this test.
                @skipme.skip_reg( func_list=[TEST_FUNCTION_NAMES] ) # Skip test if one function in the list failed.
                @skipme.skip_reg( skip_all_on_failed=True )         # Skip all consecutive tests if this fails.
                @skipme.skip_reg( func_list=[LIST_OF_TEST_FUNCTIONS], skip_all_on_failed=True ) # Or both.

Example:
    import nose2.tools.such as such
    import inspect
    import functools
    import unittest

    with such.A( 'thingy' ) as it :
        skipme = SkipMe()

        @it.has_test_setup
        def test_setup() :
            skipme.skip_if_reason()

        @it.should( "Do something" )
        @skipme.skip_reg()
        def test_one():
            raise

        @it.should( "Do another thing" )
        @skipme.skip_reg( func_list=[ 'test_one' ] )
        def test_two():
            pass

        @it.should( "just skip" )
        @skipme.skip_reg( reason='I want it that way' )
        def test_three():
            pass

        it.createTests( globals() )

    # Then run:
    nose2 --layer-reporter --plugin=nose2.plugins.layers -v
    # Prints:
    A thingy
      should Do something ... ERROR
      should Do another thing ... skipped because of failed: 'test_one'
      should just skip ... skipped intentionally because: 'I want it that way'
      ...
"""
chapter_of = {}

def __init__( self, chapter=None ) :
    """
    Initialize a SkipMe object.
    :param chapter: If set, must be a string, else it's 'Default'.
    """
    func_name = inspect.stack()[ 0 ][ 3 ]  # This function Name
    if chapter is None :
        chapter = 'Default'                                 # Set default chapter for convenience

    if not isinstance( chapter, str ) :
        wrong_type = type( chapter )
        raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'\n"
                          .format( "ERROR", 'SkipMe', func_name, str( chapter ) )
                          + "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )

    self.chapter = chapter
    self.helper = self.SkipMeHelper()                       # Set unittest.TestCase object as helper

@classmethod
def set_chapter( cls, chapter=None, func=None, value=None ):
    """
    Mark a function of a chapter as passed (True) or failed (False) in class variable 'chapter_of'.
    Expands 'chapter_of' by chapter name, function and passed/failed value.
    :param chapter:     Chapter the function belongs to
    :param func:        Function name
    :param value:       Boolean
    :return:            None
    """
    func_name = inspect.stack()[ 0 ][ 3 ]  # This function Name
    if chapter is None :
        chapter = 'Default'                                 # Set default chapter for convenience

    if not isinstance( chapter, str ) :
        wrong_type = type( chapter )
        raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'\n"
                          .format( "ERROR", 'SkipMe', func_name, str( chapter ) )
                          + "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )

    if func is None :
        raise ValueError( "{0} {1}.{2}: No input for 'func'".format( "ERROR", 'SkipMe', func_name ) )

    if not isinstance( func, str ) :
        wrong_type = type( func )
        raise ValueError( "{0} {1}.{2}: Invalid input for 'func': '{3}'\n"
                          .format( "ERROR", 'SkipMe', func_name, str( func ) )
                          + "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )

    if not isinstance( value, bool ) :
        raise ValueError( "{0} {1}.{2}: No or invalid input for 'value'".format( "ERROR", 'SkipMe', func_name ) )

    if chapter not in cls.chapter_of :                      # If we have this chapter not yet,
        cls.chapter_of[ chapter ] = {}                      # add it and set skip all to false.
        cls.chapter_of[ chapter ][ '__skip_all__' ] = False

    if func not in cls.chapter_of[ chapter ] :              # If we don't have the function yet, add it with value.
        cls.chapter_of[ chapter ][ func ] = value

@classmethod
def get_func_state( cls, chapter=None, func_list=None ):
    """
    Return function names  out of function list that previously failed
    :param chapter:     The chapter to search for functions
    :param func_list:   Browse for these function names
    :return:            List with failed functions. If none found, an empty list.
    """
    func_name = inspect.stack()[ 0 ][ 3 ]  # This function Name
    if chapter is None :
        chapter = 'Default'                                 # Set default chapter for convenience

    if not isinstance( chapter, str ) :
        wrong_type = type( chapter )
        raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'\n"
                          .format( "ERROR", 'SkipMe', func_name, str( chapter ) )
                          + "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )

    if func_list is None :
        raise ValueError( "{0} {1}.{2}: No input for 'func_list'".format( "ERROR", 'SkipMe', func_name ) )

    #-------------------------
    # Function candidates to check.
    # Collect those candidates, that previously returned as failed or skipped.
    # Otherwise, return empty list.
    if isinstance( func_list, list ) :
        func_candidates = func_list
    elif isinstance( func_list, str ) :
        func_candidates = [ x.strip() for x in func_list.split( ',' ) ]
    else:
        wrong_type = type( func_list )
        raise ValueError( "{0} {1}: Invalid input for 'func_list': '{2}'\n"
                          .format( "ERROR", func_name, str( func_list ) )
                          + "{0} Must be list or comma separated string, but was: '{1}'"
                          .format( "INFO", wrong_type.__name__ ) )

    to_return = []                                          # List of failed functions
    if chapter not in cls.chapter_of :                      # If chapter not found, just return empty list
        return to_return

    for func in func_candidates :                           # Otherwise look for each candidate
        if func not in cls.chapter_of[ chapter ] :          # if it's in the chapter, skip if not.
            continue

        if not cls.chapter_of[ chapter ][ func ] :          # If it's value is False, append it.
            to_return.append( func )

    return to_return

@classmethod
def mark_chapter_as_skipped( cls, chapter=None, func=None ):
    """
    Mark chapter as skipped. Maybe because of a failed function
    :param chapter:     Which chapter to mark as skipped
    :param func:        Maybe the failed function that causes this decision
    :return:            None
    """
    func_name = inspect.stack()[ 0 ][ 3 ]                   # This function Name
    if chapter is None :
        chapter = 'Default'                                 # Set default chapter for convenience

    if not isinstance( chapter, str ) :
        wrong_type = type( chapter )
        raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'\n"
                          .format( "ERROR", 'SkipMe', func_name, str( chapter ) )
                          + "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )

    # Either func is a name or True.
    if func :
        if not isinstance( func, str ) :
            wrong_type = type( chapter )
            raise ValueError( "{0} {1}.{2}: Invalid input for 'func': '{3}'\n"
                              .format( "ERROR", 'SkipMe', func_name, str( func ) )
                              + "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )
    else :
        func = True

    if chapter not in cls.chapter_of :                      # If we have this chapter not yet,
        cls.chapter_of[ chapter ] = {}                      # add it and set skip all to false.

    cls.chapter_of[ chapter ][ '__skip_all__' ] = func

@classmethod
def chapter_marked_skipped( cls, chapter=None ):
    """
    Check if a chapter is marked to skip.
    :param chapter:     The chapter to check
    :return:    False   :   Chapter is not marked to be skipped
                True    :   Chapter was intentionally skipped
                String  :   This function was marked with 'skip_all_on_failed=True' and failed.
    """
    func_name = inspect.stack()[ 0 ][ 3 ]                   # This function Name
    if chapter is None :
        chapter = 'Default'                                 # Set default chapter for convenience

    if not isinstance( chapter, str ) :
        wrong_type = type( chapter )
        raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'\n"
                          .format( "ERROR", 'SkipMe', func_name, str( chapter ) )
                          + "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )

    to_return = False
    if chapter not in cls.chapter_of :
        return to_return

    to_return = cls.chapter_of[ chapter ].get( '__skip_all__', False )
    return to_return

def skip_reg( self, func_list=None, skip_all_on_failed=None, reason=None ) :
    """
    Synopsis          :
        skipme = SkipMe( 'my_id' )
        @skipme.skip_reg()
        def some_test_func() :
            ...

        @skipme.skip_reg( func_list=[ LIST_OF_FUNCTIONS_TO_SKIP_IF_FAILED ], skip_all_on_failed=BOOLEAN,
                        reason=REASON )
        def some_other_test_func():
            ...

    Purpose           : Decorator to register functions in a SkipMe object and skip tests if necessary

    Incoming values :
        :param func_list    :   List or comma separated string with function names.
                                Skip this test if one of these functions in the list failed or were skipped.
        :param chapter      :   Identifier string that can be used to control skipping more generally.
                                Default name for chapter is 'Default'.
        :param reason       :   Skip this test in any case and set a string for the reason.
        :param skip_all_on_failed   :   Boolean. If this test fails, mark the current chapter
                                        to skip the rest of tests.
    Outgoing results  :
        :return     :   Updated class attribute 'chapter_of', maybe skipped test.
    Defaults          :
            chapter :   'Default'
    Prerequisites     :
        inspect, mars.colors.mcp
    Description:
        Register functions by decorating the functions.
        It returns a dict with the function name as key and the function
        reference as value.
    """
    func_name = inspect.stack()[ 0 ][ 3 ]  # This function Name
    chapter = self.chapter

    if isinstance( func_list, list ) :
        func_candidates = func_list
    elif isinstance( func_list, str ) :
        func_candidates = [ x.strip() for x in func_list.split( ',' ) ]
    elif func_list is None:
        func_candidates = []
    else :
        wrong_type = type( func_list )
        raise ValueError( "{0} {1}: Invalid input for 'func_list': '{2}'\n"
                          .format( "ERROR", func_name, str( func_list ) )
                          + "{0} Must be list or comma separated string, but was: '{1}'"
                          .format( "INFO", wrong_type.__name__ ) )

    if reason and not isinstance( reason, str ) :
        wrong_type = type( func_list )
        raise ValueError( "{0} {1}: Invalid input for 'reason': '{2}'\n"
                          .format( "ERROR", func_name, str( reason ) )
                          + "{0} Must be string, but was: '{1}'"
                          .format( "INFO", wrong_type.__name__ ) )
    def inner_skip_reg( func ) :
        @functools.wraps( func )
        def skip_reg_wrapper( *args, **kwargs ) :
            #-------------------------
            # First check if the whole chapter was marked as skipped.
            # The function either returns:
            #   True            :   Means 'skip_all' was set in the beginning (e.g. with @has_setup)
            #   False           :   No, chapter is not marked as to be skipped
            #   Function name   :   This function was marked with 'skip_all_on_failed' and failed.
            skip_reason = self.get_skip_reason()
            if skip_reason :
                if isinstance( skip_reason, bool ) :
                    self.helper.skipTest( "chapter '{0}' because it's marked to be skipped".format( chapter ) )
                else :
                    self.helper.skipTest( "chapter '{0}' because of: '{1}'"
                                   .format( chapter, skip_reason ) )

            #-------------------------
            # Then check if we are just intended to skip by a reason. If so, mark and skip
            if reason :
                self.__class__.set_chapter( chapter=chapter, func=func.__name__, value=False )
                self.helper.skipTest( "intentionally because: '{0}'".format( reason ) )

            #-------------------------
            # Now see if one of our functions we depend on failed.
            # If so, mark our func as failed and skip it.
            if func_candidates :
                found_failed = self.__class__.get_func_state( chapter=chapter, func_list=func_candidates )
                if found_failed :
                    self.__class__.set_chapter( chapter=chapter, func=func.__name__, value=False )
                    self.helper.skipTest( "because of failed: '{0}'".format( ', '.join( found_failed ) ) )

            #-------------------------
            # Now run the test.
            # If it fails (assertion error), mark as failed (False), else mark as passed (True)
            # If it fails and was marked as 'skip_all_on_failed', mark chapter as to skip all further tests.
            try :
                result = func( *args, **kwargs )
                self.__class__.set_chapter( chapter=chapter, func=func.__name__, value=True )
            except Exception as error :
                self.__class__.set_chapter( chapter=chapter, func=func.__name__, value=False )
                if skip_all_on_failed :
                    self.__class__.mark_chapter_as_skipped( chapter=chapter, func=func.__name__ )
                if error :
                    raise error
                else :
                    raise

            return result
        return skip_reg_wrapper
    return inner_skip_reg

def get_skip_reason( self ):
    chapter = self.chapter
    skip_reason = self.__class__.chapter_marked_skipped( chapter=chapter )
    return skip_reason

def skip_all( self, reason=None ):
    func_name = inspect.stack()[ 0 ][ 3 ]  # This function Name
    if reason is not None :
        if not isinstance( reason, str ) :
            wrong_type = type( reason )
            raise ValueError( "{0} {1}: Invalid input for 'reason': '{2}'\n"
                              .format( "ERROR", func_name, str( reason ) )
                              + "{0} Must be string, but was: '{1}'".format( "INFO", wrong_type.__name__ ) )

    self.__class__.mark_chapter_as_skipped(chapter=self.chapter, func=reason )

def skip_if_reason( self ):
    skip_reason = self.get_skip_reason()
    if skip_reason:
        if skip_reason is True :
            reason = "it's marked to be skipped."
        else :      # Either function or other text.
            if skip_reason in self.__class__.chapter_of[ self.chapter ].keys() :
                reason = "'{0}' failed.".format( skip_reason )
            else :
                reason = "'{0}'.".format( skip_reason )
        self.helper.skipTest( "chapter '{0}' because: {1}".format( self.chapter, reason ) )

class SkipMeHelper( unittest.TestCase ):
    def runTest(self):
        pass

SkipMeHelper.maxDiff = None         # No limitation in depth while comparing
© www.soinside.com 2019 - 2024. All rights reserved.