通过模拟 Java 反射调用来对 Hive 自定义 UDF 进行单元测试用例

Question

我有一个需求，需要开发使用 Java 反射 API 来调用外部类的 Hive 自定义 UDF。

由于我是 Java Reflection 的新手，所以我花了一些时间学习它并能够进行基本的实现。

但是我在为此实现编写单元测试用例时遇到问题，因为我在模拟反射 API 方面面临一些挑战。

以下是 Hive 自定义 UDF 的示例。

ReverseString.java

public class ReverseString extends GenericUDF {

    private StringObjectInspector input;
    Class<?> c = null;
    Object ob = null;


    @Override
    public ObjectInspector initialize(ObjectInspector[] arg0) throws UDFArgumentException {

        // create an ObjectInspector for the input
        ObjectInspector input = arg0[0];

        // check to make sure the input is a string
        if (!(input instanceof StringObjectInspector)) {
            throw new UDFArgumentException("input must be a string");
        }


        this.input = (StringObjectInspector) input;
        System.out.println("Success. Input formatted correctly");

        return init(arg0);
    }

    public ObjectInspector init(ObjectInspector[] arg0) throws UDFArgumentException {

        try {
            c = Class.forName("com.hive.inherit.DummyUdf");
            ob = c.getConstructor().newInstance();
            Method method = c.getMethod("print",String.class);
            String res = (String) method.invoke(ob,"TEst");
            System.out.println("RES: "+res);

        } catch (NoSuchMethodException | ClassNotFoundException | IllegalAccessException | InvocationTargetException | InstantiationException e) {
            e.printStackTrace();
        }

        return PrimitiveObjectInspectorFactory.javaStringObjectInspector;
    }


    @Override
    public Object evaluate(DeferredObject[] arg0) throws HiveException {

        if (input == null || arg0.length != 1 || arg0[0].get() == null) {
            return null;
        }
        String forwards = input.getPrimitiveJavaObject(arg0[0].get());
        System.out.println("forwards:"+forwards);

        return reverse(forwards);
    }


    public  Object reverse(String in) {
        Object res = null ;
        try {
            if (this.c != null && this.ob != null) {
                Method method = this.c.getMethod("reverse", String.class);
                res = (String) method.invoke(this.ob, in);
            }else{
                c = Class.forName("com.hive.inherit.DummyUdf");
                ob = c.getConstructor().newInstance();
                Method method = c.getMethod("reverse", String.class);
                res = method.invoke(ob, in);
            }
        } catch (NoSuchMethodException | IllegalAccessException | InvocationTargetException | ClassNotFoundException | InstantiationException e) {
            e.printStackTrace();
        }
        return res;
    }

    @Override
    public String getDisplayString(String[] strings) {
        return null;
    }
}

下面是 Reflection 调用的类。

DummyUdf.java

package com.hive.inherit;

public class DummyUdf {
    

    public DummyUdf(){
        System.out.println("DummyUdf");
    }

    public String print(String str){
        System.out.println("DummyUdf-str:"+str);
        return str;
    }

    public String reverse(String in) {

        int l = in.length();
        StringBuilder sb = new StringBuilder();

        for (int i = 0; i < l; i++) {
            sb.append(in.charAt(l - i - 1));
        }
        return sb.toString();
    }
}

我正在尝试实现的单元测试用例，

ReverseStringTest.class

@RunWith(MockitoJUnitRunner.class)
public class ReverseStringTest {

    @Test
    public void testSimpleString() throws HiveException {

        //ReverseString r = Mockito.spy(new ReverseString());
        ReverseString r = mock(ReverseString.class);
        ObjectInspector input = PrimitiveObjectInspectorFactory.javaStringObjectInspector;
        when(r.init(Mockito.any())).thenReturn(input);
        JavaStringObjectInspector resultInspector = (JavaStringObjectInspector) r.initialize(
                new ObjectInspector[] { input });
        Text forwards = new Text("hello");
        when(r.reverse(Mockito.any())).thenReturn("olleh");
        Object result = r.evaluate(new GenericUDF.DeferredObject[] { new GenericUDF.DeferredJavaObject(forwards) });
        System.out.println(result);
        assertEquals("olleh", resultInspector.getPrimitiveJavaObject(result));
    }
}

测试用例因 NullPointerException 失败。

java.lang.NullPointerException at com.hive.udf.ReverseStringTest.testSimpleString(ReverseStringTest.java:34) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)

有人可以建议如何适当地嘲笑这个吗？

提前致谢。

Answer 1

经过一番研究，我找到了上述问题的可行解决方案。

以下是工作解决方案：

public class ReverseStringTest {

    @Test
    public void testSimpleString() throws Exception {

        ReverseString reverseString = org.mockito.Mockito.spy(ReverseString.class);

        ObjectInspector[] arg0 = new ObjectInspector[] {PrimitiveObjectInspectorFactory.javaStringObjectInspector,
                PrimitiveObjectInspectorFactory.javaStringObjectInspector} ;

        when(reverseString.init(arg0)).thenReturn(PrimitiveObjectInspectorFactory.javaStringObjectInspector);
        reverseString.initialize(arg0);

        Text text = new Text("Ans");
        GenericUDF.DeferredObject[] deferredObjects = { new GenericUDF.DeferredJavaObject(text) };
        when(reverseString.reverse("Ans")).thenReturn(testcase());

        Object result = reverseString.evaluate(deferredObjects);
        System.out.println(result);



    }

    public Object testcase() {
        return new Text("snA") ;

    }

}

我们可以使用“spy”来代替“mock”，其中对象的一部分将被模拟，而一部分将使用真正的方法调用（如果您想考虑这一点，这也有助于提高代码覆盖率）。

您可以参考下面的链接来了解更多关于 Mockito 'mock' 和 'spy' 的信息。

Mockito - @Spy 与 @Mock

由于我们已经扩展了

GenericUDF

，并且 Hive 实现首先调用

initialize

方法，因此在调用

evaluate

之前模拟正确传递的参数非常重要。