likes
comments
collection
share

一种自动生成趋势比对数据的方法设计及实现

作者站长头像
站长
· 阅读数 40

背景

数据之间两两趋势比较在数据分析应用中是非常常见的应用场景，如下所示：

模拟考批次班级学生语文数学英语
202302三年一班张小明130145133
202302三年一班王二小128138140
202302三年一班谢春花136142139
202301三年一班张小明132140128
202301三年一班王二小125146142
202301三年一班谢春花138143140
202212三年一班张小明135138120
202212三年一班王二小123145138
202212三年一班谢春花136140142

现在有一个需求：各班级的每个学生在不同考试批次的各学科成绩的进退步情况，得出数据如下

模拟考批次班级学生语文数学英语
202302与202301对比三年一班张小明-255
202302与202301对比三年一班王二小3-8-2
202302与202301对比三年一班谢春花-2-1-1
202301与202212对比三年一班张小明-328
202301与202212对比三年一班王二小214
202301与202212对比三年一班谢春花23-2

详细设计及实现

趋势比对定义类 TrendCompare

public class TrendCompare {
    /**
     * 主体的字段列表（如三年一班的张小明，那么主体字段列表为 班级 + 学生姓名）
     */
    private String[] subjectFields;
    /**
     * 在某个字段的（介词） 如三年一班张晓明在不同考试批次的成绩对比结果
     */
    private String atField;

    /**
     * 参与趋势比较的字段集合
     */
    private String[] compareFields;

    /**
     * 赋值映射集合：给结果数据中指定的key设置指定的值
     */
    private Map<String, String> assignValMap;

    public String[] subjectFields() {
        return this.subjectFields;
    }

    public TrendCompare subjectFields(String... fields) {
        this.subjectFields = fields;
        return this;
    }

    public String atField() {
        return this.atField;
    }

    public TrendCompare atField(String field) {
        this.atField = field;
        return this;
    }

    public String[] compareFields() {
        return this.compareFields;
    }

    public TrendCompare compareFields(String... fields) {
        this.compareFields = fields;
        return this;
    }

    /**
     * 赋值操作
     *
     * @param field
     * @param valueEL 值表达式
     * @return
     */
    public TrendCompare assignVal(String field, String valueEL) {
        if (assignValMap == null) {
            assignValMap = new HashMap<>();
        }
        assignValMap.put(field, valueEL);
        return this;
    }

    public Map<String, String> assignValMap() {
        return this.assignValMap;
    }
}

该类定义了如下属性：

  1. 主体的字段列表
  2. 介词字段
  3. 比对的字段列表

如：各班级的每个学生在不同考试批次的各学科成绩的进退步情况

上面的需求映射到定义类的结果如下：

  1. 主体的字段列表（班级、学生）
  2. 介词字段（考试批次）
  3. 比对的字段列表（各学科：语文、数学、英语）

趋势比对执行类

该类提供了一个供外部调用的方法如下

public static <T> List<T> compare(List<T> dataList, TrendCompare trendCompare) {
    Map<String, List<T>> groupMap = group(dataList, null, trendCompare.subjectFields());
    List<T> resultList = new ArrayList<>();
    for (List<T> groupDataList : groupMap.values()) {
        List<T> diffValueList = new ArrayList<>();
        int size = groupDataList.size();
        if (size > 1) {
            for (int i = 0; i < size - 1; i++) {
                //数据之间两两比较 diffValue = minuend - subtrahend
                T minuend = groupDataList.get(i);
                T subtrahend = groupDataList.get(i + 1);
                T diffValue = minus(trendCompare.compareFields(), minuend, subtrahend);
                //设置主体信息
                if (trendCompare.subjectFields() != null) {
                    for (String subjectField : trendCompare.subjectFields()) {
                        setFieldValue(diffValue, subjectField, getFieldValue(minuend, subjectField));
                    }
                }
                //设置介词字段信息
                String atField = trendCompare.atField();
                if (StringUtils.isNotEmpty(atField)) {
                    setFieldValue(diffValue, atField, getFieldValue(minuend, atField) + "与" + getFieldValue(subtrahend, atField) + "对比增减");
                }
                diffValueList.add(diffValue);
            }
        }
        if (diffValueList.size() > 0) {
            T firstData = groupDataList.get(0);
            Map<String, Object> valMap = new HashMap<>();
            //指定的赋值集合进行赋值
            if (trendCompare.assignValMap() != null) {
                for (Map.Entry<String, String> stringStringEntry : trendCompare.assignValMap().entrySet()) {
                    String field = stringStringEntry.getKey();
                    if (!StringUtils.equalsAny(field, trendCompare.compareFields())) {
                        String valueEL = stringStringEntry.getValue();
                        valMap.put(field, executeSpEL(valueEL, firstData));
                    }
                }
            }
            for (Map.Entry<String, Object> entry : valMap.entrySet()) {
                for (T diffValue : diffValueList) {
                    setFieldValue(diffValue, entry.getKey(), entry.getValue());
                }
            }
        }
        resultList.addAll(diffValueList);
    }
    return resultList;
}

可以看到，该方法要求传入

  1. 数据集合
  2. 趋势比对定义

两个参数，并最终返回趋势比对后的结果集合。

该方法的内部逻辑可分为如下2个步骤：

  1. 按主体分组
  2. 分组后组内数据两两比对，并最终返回比对结果。

使用案例

假设有如下这样一组数据

一种自动生成趋势比对数据的方法设计及实现

定义一个学生类：

public class Student {
    private String batch;
    private String banji;
    private String studentNo;
    private String name;
    private String sex;
    private Double yuwen;
    private Double math;
    private Double english;
    private Double physics;
    //extra
    private String maxScoreName1;
    public Student(String batch, String banji, String studentNo, String name, String sex, Double yuwen, Double math, Double english, Double physics) {
        this.batch = batch;
        this.banji = banji;
        this.studentNo = studentNo;
        this.name = name;
        this.sex = sex;
        this.yuwen = yuwen;
        this.math = math;
        this.english = english;
        this.physics = physics;
    }
}

我们写一个方法，返回如上数据：

public List<Student> getDataList() {
    List<Student> dataList = new ArrayList<>();
    dataList.add(new Student("202302", "三年一班", "20001001", "张小明", "男", 130.0, 145.0, 133.0, 92.0));
    dataList.add(new Student("202302", "三年一班", "20001002", "王二小", "男", 128.0, 138.0, 140.0, 98.0));
    dataList.add(new Student("202302", "三年一班", "20001003", "谢春花", "女", 136.0, 142.0, 139.0, 95.0));
    dataList.add(new Student("202302", "三年二班", "20002001", "冯世杰", "男", 129.0, 144.0, 138.0, 96.0));
    dataList.add(new Student("202302", "三年二班", "20002002", "马功成", "男", 130.0, 132.0, 133.0, 98.0));
    dataList.add(new Student("202302", "三年二班", "20002003", "魏翩翩", "女", 136.0, 142.0, 137.0, 92.0));
    dataList.add(new Student("202301", "三年一班", "20001001", "张小明", "男", 132.0, 142.0, 134.0, 92.0));
    dataList.add(new Student("202301", "三年一班", "20001002", "王二小", "男", 126.0, 136.0, 135.0, 94.0));
    dataList.add(new Student("202301", "三年一班", "20001003", "谢春花", "女", 136.0, 145.0, 139.0, 95.0));
    dataList.add(new Student("202301", "三年二班", "20002001", "冯世杰", "男", 124.0, 143.0, 148.0, 90.0));
    dataList.add(new Student("202301", "三年二班", "20002002", "马功成", "男", 140.0, 133.0, 138.0, 90.0));
    dataList.add(new Student("202301", "三年二班", "20002003", "魏翩翩", "女", 126.0, 136.0, 135.0, 92.0));
    return dataList;
}

趋势比对定义并执行比对：

List<Student> dataList = getDataList();
TrendCompare trendCompare = new TrendCompare()
        .subjectFields("banji", "name")
        .atField("batch")
        .compareFields("yuwen", "math", "english")
        //.assignVal("batch", "'环比增减'")
        ;
List<Student> resultList = DataProcessUtil.compare(dataList, trendCompare);
for (Student result : resultList) {
    System.out.println(JSON.toJSONString(result));
}

结果如下：

{"banji":"三年一班","batch":"202302与202301对比增减","english":-1.0,"math":3.0,"name":"张小明","yuwen":-2.0}
{"banji":"三年一班","batch":"202302与202301对比增减","english":5.0,"math":2.0,"name":"王二小","yuwen":2.0}
{"banji":"三年一班","batch":"202302与202301对比增减","english":0.0,"math":-3.0,"name":"谢春花","yuwen":0.0}
{"banji":"三年二班","batch":"202302与202301对比增减","english":-10.0,"math":1.0,"name":"冯世杰","yuwen":5.0}
{"banji":"三年二班","batch":"202302与202301对比增减","english":-5.0,"math":-1.0,"name":"马功成","yuwen":-10.0}
{"banji":"三年二班","batch":"202302与202301对比增减","english":2.0,"math":6.0,"name":"魏翩翩","yuwen":10.0}
转载自:https://juejin.cn/post/7218559384452087868
评论
请登录