CodeQL: GitHub Security Lab CTF

summersec师傅的文章【1】不错，主要参考师傅的文章

0x01 背景

GitHub Security Lab CTF

第四题：GitHub Security Lab CTF 4: CodeQL and Chill - The Java Edition
相关漏洞：CVE-2020-9297
CodeQL database 下载地址

CVE-2020-9297

祖师爷pwntester 发现的，$3000，然后被拿来出题。。。

upload successful

0x02 CVE-2020-9297

项目在 https://github.com/Netflix/titus-control-plane/archive/refs/tags/v0.1.1-rc.263.zip

1
2
3
4
5
6
wget https://github.com/Netflix/titus-control-plane/archive/refs/tags/v0.1.1-rc.263.zip
unzip v0.1.1-rc.263.zip
cd titus-control-plane-0.1.1-rc.263
#几个包缺失，会失败，不折腾了直接用提供的codeql db
#jenv local 1.8
#codeql database create titus_db -l java -c="./gradlew build"

题目是直接给出了Sink点：ConstraintValidatorContext.buildConstraintViolationWithTemplate

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
@Override
public boolean isValid(Container container, ConstraintValidatorContext context) {
    if (container == null) {
        return true;
    }
    Set<String> common = new HashSet<>(container.getSoftConstraints().keySet());
    common.retainAll(container.getHardConstraints().keySet());

    if (common.isEmpty()) {
        return true;
    }

    context.buildConstraintViolationWithTemplate(
            "Soft and hard constraints not unique. Shared constraints: " + common
    ).addConstraintViolation().disableDefaultConstraintViolation();
    return false;
}

2.1 JSR 380 Bean Validation

Java API规范(JSR303)定义了Bean校验的标准validation-api，但没有提供实现。hibernate validation是对这个规范的实现，并增加了校验注解如@Email、@Length等。Spring Validation是对hibernate validation的二次封装，用于支持spring mvc参数自动校验。

使用Bean Validation可以简单的在需要验证的类或者属性上加上对应的注解，就可以使用内置或者自定义验证器对Bean进行验证，优势就是只需要约束一次，不用在需要验证的所有接口处加入大量的if-else 判断，方便代码维护，简化代码量。

详情可以参考【7】

2.2 Bean Stalking: Growing Java beans into RCE

详情参考祖师爷pwntester 【6】的文章，结论是ConstraintValidatorContext.buildConstraintViolationWithTemplate()参数中存在注入EL注入。buildConstraintViolationWithTemplate 作用是invalid 失败时进行报错。

pwntester 祖师爷从CVE-2018-16621 开始，总结了该漏洞类型的原理，给出了CodeQL 规则，并且发现了另外的发现。

upload successful

  • CVE-2018-16621 Nexus
  • CVE-2020-10199 Nexus
  • CVE-2020-10204 Nexus

2.3 调用链

参考【7】中的调用链

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
interpolate:67, ElTermResolver (org.hibernate.validator.internal.engine.messageinterpolation)
interpolate:64, InterpolationTerm (org.hibernate.validator.internal.engine.messageinterpolation)
interpolate:159, ResourceBundleMessageInterpolator (org.hibernate.validator.messageinterpolation)
interpolateExpression:519, AbstractMessageInterpolator (org.hibernate.validator.messageinterpolation)
interpolateMessage:415, AbstractMessageInterpolator (org.hibernate.validator.messageinterpolation)
interpolate:355, AbstractMessageInterpolator (org.hibernate.validator.messageinterpolation)
interpolate:59, MessageSourceMessageInterpolator (org.springframework.boot.validation)
interpolate:51, LocaleContextMessageInterpolator (org.springframework.validation.beanvalidation)
interpolate:313, AbstractValidationContext (org.hibernate.validator.internal.engine.validationcontext)
addConstraintFailure:230, AbstractValidationContext (org.hibernate.validator.internal.engine.validationcontext)
validateConstraints:79, ConstraintTree (org.hibernate.validator.internal.engine.constraintvalidation)
doValidateConstraint:130, MetaConstraint (org.hibernate.validator.internal.metadata.core)
validateConstraint:123, MetaConstraint (org.hibernate.validator.internal.metadata.core)
validateMetaConstraint:555, ValidatorImpl (org.hibernate.validator.internal.engine)
validateConstraintsForSingleDefaultGroupElement:518, ValidatorImpl (org.hibernate.validator.internal.engine)
validateConstraintsForDefaultGroup:488, ValidatorImpl (org.hibernate.validator.internal.engine)
validateConstraintsForCurrentGroup:450, ValidatorImpl (org.hibernate.validator.internal.engine)
validateInContext:400, ValidatorImpl (org.hibernate.validator.internal.engine)
validate:172, ValidatorImpl (org.hibernate.validator.internal.engine)
validate:109, SpringValidatorAdapter (org.springframework.validation.beanvalidation)
validate:66, ValidatorAdapter (org.springframework.boot.autoconfigure.validation)
validate:895, DataBinder (org.springframework.validation)
validateIfApplicable:245, AbstractMessageConverterMethodArgumentResolver (org.springframework.web.servlet.mvc.method.annotation)
resolveArgument:139, RequestResponseBodyMethodProcessor (org.springframework.web.servlet.mvc.method.annotation)
resolveArgument:121, HandlerMethodArgumentResolverComposite (org.springframework.web.method.support)
getMethodArgumentValues:179, InvocableHandlerMethod (org.springframework.web.method.support)
invokeForRequest:146, InvocableHandlerMethod (org.springframework.web.method.support)
invokeAndHandle:117, ServletInvocableHandlerMethod (org.springframework.web.servlet.mvc.method.annotation)
invokeHandlerMethod:895, RequestMappingHandlerAdapter (org.springframework.web.servlet.mvc.method.annotation)
handleInternal:808, RequestMappingHandlerAdapter (org.springframework.web.servlet.mvc.method.annotation)
handle:87, AbstractHandlerMethodAdapter (org.springframework.web.servlet.mvc.method)
doDispatch
……

hibernate validator的8.0.0已经引入constraint_expression_language_feature_level 和custom_violation_expression_language_feature_level 进行EL表达式的限制，默认已经不能利用，具体哪个版本引入的赞未考究。

upload successful

0x03 用CodeQL 进行漏洞分析

3.1 Sink

1
2
3
4
5
6
7
8
9
override predicate isSink(DataFlow::Node sink) { 
    exists(
        MethodAccess ma|
        ma.getMethod().getDeclaringType().hasQualifiedName("javax.validation", "ConstraintValidatorContext") 
        and ma.getMethod().hasName("buildConstraintViolationWithTemplate")
        // and ma.getMethod().getParameter(0) = sink.asParameter()
        and ma.getArgument(0) = sink.asExpr()
    )
}

Tips: Method.getParameter() = sink.asParameter() VS MethodAccess.getArgument() = sink.asExpr()

  • sink.asParameter 最终的结果是buildConstraintViolationWithTemplate 声明时候的参数
  • sink.asExpr 最终的结果是buildConstraintViolationWithTemplate 调用时候的参数
    因为javax.validation.ConstraintValidatorContext#buildConstraintViolationWithTemplate 是在依赖库中，因此需要使用sink.asExpr，如果都是源码，那么asExpr 可以当作是asParameter的TiantFlow上一个节点，结果没有区别；如果是在依赖库中，则需要用sink.asExpr。

3.2 Source

1
2
3
4
5
6
7
8
9
10
11
private class VailidatorSource extends RemoteFlowSource {
    // isValid 参数1 可控
    VailidatorSource(){
        exists(
            Method m | m.getParameter(0) = this.asParameter() |
            m.getDeclaringType().getASourceSupertype().hasQualifiedName("javax.validation", "ConstraintValidator")
            and m.hasName("isValid")
        )
    }
    override string getSourceType() { result = "validator" }
}

Tips: 此处为何不用MethodAccess = this.asExpr ?
Method 不一定会有对应的MethodAccess

3.3 TaintTracking configuration

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
class VulConfig extends TaintTracking::Configuration {
    // class VulConfig extends DataFlow::Configuration {
    VulConfig() { this = "jaas vul" }
    override int fieldFlowBranchLimit() { result = 100 }
    override int explorationLimit() { result = 10 }
    override predicate isSource(DataFlow::Node src) { src instanceof VailidatorSource }
    override predicate isSink(DataFlow::Node sink) { 
		...
    }
}

from VulConfig config, DataFlow::PathNode source, DataFlow::PathNode sink
where config.hasFlowPath(source, sink)
// select source, "Data flow to $@.", sink, sink.toString()
select sink.getNode(), source, sink, "$@.", source.getNode(), "user input"

**执行，但是发现结果为空

upload successful

3.4 Partial Flow 调试

具体参考CodeQL官方的说明文档【8】，Partial Flow可以用于数据流的调试

3.4.1 自查

官方给了两个自查的手段

3.4.1.1 Checking sources and sinks

检查source 和sink，单独quick evaluate 看看结果是否符合预期。

3.4.1.2 fieldFlowBranchLimit

官方解释：

Gets the virtual dispatch branching limit when calculating field flow. This can be overridden to a smaller value to improve performance (a value of 0 disables field flow), or a larger value to get more results.
获取计算字段流时的虚拟调度分支限制。可以将其覆盖为较小的值以提高性能（0 将禁用字段流），或将其覆盖为较大的值以获得更多结果。

这里的Field是指的Object 的Field，还是任何污点节点？没懂

3.4.2 Partial Flow

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
/** 
@kind path-problem
*/
import java
import semmle.code.java.dataflow.TaintTracking
import DataFlow::PartialPathGraph // this is different!

class MyTaintTrackingConfig extends TaintTracking::Configuration {
    MyTaintTrackingConfig() { ... } // same as before
    override predicate isSource(DataFlow::Node source) { ... } // same as before
    override predicate isSink(DataFlow::Node sink) { ... } // same as before
    override int explorationLimit() { result =  10} // this is different!
}

from MyTaintTrackingConfig cfg, DataFlow::PartialPathNode source, DataFlow::PartialPathNode sink
where cfg.hasPartialFlow(source, sink, _)
select sink, source, sink, "Partial flow from unsanitized user data"
  • import 的是DataFlow::PartialPathGraph
  • explorationLimit 个人理解是最大污点传播深度

官方DataFlow::hasPartialFlow 的解释

Holds if there is a partial data flow path from source to node. The approximate distance between node and the closest source is dist and is restricted to be less than or equal to explorationLimit(). This predicate completely disregards sink definitions.
This predicate is intended for data-flow exploration and debugging and may perform poorly if the number of sources is too big and/or the exploration limit is set too high without using barriers.

  • hasPartialFlow 的第三个参数也是要不大于conf内的explorationLimit
  • 忽略sink 点约束
  • source 节点过多/explorationLimit 过大都可能导致性能问题

3.4.3 实战定位

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
/**
 * @name CVE-2020-9297-partial
 * @kind path-problem
 */
import java
import semmle.code.java.dataflow.DataFlow
import semmle.code.java.dataflow.FlowSources
import DataFlow::PartialPathGraph
 

private class VailidatorSource extends RemoteFlowSource {
    // isValid 参数1 可控
    VailidatorSource(){
        exists(
            Method m | m.getParameter(0) = this.asParameter() |
            m.getDeclaringType().getASourceSupertype().hasQualifiedName("javax.validation", "ConstraintValidator")
            and m.hasName("isValid")
            // 增加约束条件
            and m.getFile().getBaseName() = "SchedulingConstraintValidator.java"
        )
    }
    override string getSourceType() { result = "validator" }
}
 
class VulConfig extends TaintTracking::Configuration {
    // class VulConfig extends DataFlow::Configuration {
    VulConfig() { this = "vul" }

    // override int fieldFlowBranchLimit() { result = 100 }

    override int explorationLimit() { result = 8 }

    override predicate isSource(DataFlow::Node src) { src instanceof VailidatorSource }

    override predicate isSink(DataFlow::Node sink) { 
        exists(
        MethodAccess ma|
        ma.getMethod().getDeclaringType().getASupertype*().hasQualifiedName("javax.validation", "ConstraintValidatorContext") 
        and ma.getMethod().hasName("buildConstraintViolationWithTemplate")
        and ma.getArgument(0) = sink.asExpr()
        )
    }
}
 
from VulConfig config, DataFlow::PartialPathNode source, DataFlow::PartialPathNode sink
where config.hasPartialFlow(source, sink, _)
select sink, source, sink, "Partial flow from unsanitized user data"

用VSCode run，或者codeql database analyze titus-control-plane-db --format=sarif-latest dev/titus/partial.ql -o titus.sarif，实际运行会发现只占用了单核，

Tips: VSCode CodeQL 插件通过启动query server进程来执行对应的查询，不用单独起java进程，可以在插件配置内配置Threads 和 Mem大小，但是每个查询实际上还是单进程的。

1
codeql-home/codeql/tools/osx64/java-aarch64/bin/java -Xmx2048M --add-modules jdk.unsupported -cp codeql-home/codeql/tools/codeql.jar com.semmle.cli2.CodeQL execute query-server2 --threads 2 -J-Xmx2048M --off-heap-ram=2048 --evaluator-log xxx/GitHub.vscode-codeql/structured-evaluator-log.json --evaluator-log-level 5 --debug --tuple-counting -v --log-to-stderr

3.4.4 简化

拷贝SchedulingConstraintValidator.java 并做适当语法修改。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
...
// import static com.netflix.titus.common.util.CollectionsExt.asSet;
...
public class SchedulingConstraintValidator implements ConstraintValidator<SchedulingConstraintValidator.SchedulingConstraint, Map<String, String>> {
	...
    private static final Set<String> CONSTRAINT_NAMES = new HashSet<>(Arrays.asList(new String[]{"uniquehost", "exclusivehost", "zonebalance",
    "activehost", "availabilityzone", "machineid", "machinegroup", "machinetype", "toleration"}));
	...
    @Override
    public boolean isValid(Map<String, String> value, ConstraintValidatorContext context) {
        Set<String> namesInLowerCase = value.keySet().stream().map(String::toLowerCase).collect(Collectors.toSet());
        HashSet<String> unknown = new HashSet<>(namesInLowerCase);
        unknown.removeAll(CONSTRAINT_NAMES);
        if (unknown.isEmpty()) {
            return true;
        }
        context.buildConstraintViolationWithTemplate("Unrecognized constraints " + unknown)
                .addConstraintViolation().disableDefaultConstraintViolation();
        return false;
    }
}

option 文件

1
//semmle-extractor-options: --javac-args -cp ${testdir}/../../stub/hibernate-validator-5.4.3.Final.jar:${testdir}/../../stub/validation-api-1.1.0.Final.jar
1
codeql test run .

upload successful

可以看到在…+… 处污染链断了

3.4.5 隐式参数

1
2
3
4
5
6
7
8
9
10
11
12
13
override predicate isAdditionalTaintStep(DataFlow::Node pred, DataFlow::Node succ) {
    pred.asExpr() = succ.asExpr().(AddExpr).getAnOperand()
}

override predicate allowImplicitRead(DataFlow::Node node, DataFlow::ContentSet c) {
    super.allowImplicitRead(node, c)
    or this.isAdditionalTaintStep(node, _)
    and (
        c instanceof DataFlow::ArrayContent
        or c instanceof DataFlow::CollectionContent
        or c instanceof DataFlow::MapValueContent
    )
}

测试环境
upload successful

upload successful

0x04 思考

  1. Sink点ConstraintValidatorContext.buildConstraintViolationWithTemplate 背后的故事，有一系列的漏洞，参考【6】

  2. summersec师傅和l3yx师傅得到的是不同的sink点和路径，其他剩余的路径呢？
    upload successful

  3. CTF题目当时用的codeql版本较低，还存在HashSet/Getter&Setter 之类的传播问题，可以参考summersec师傅和l3yx师傅的文章

0x05 参考

tag:

    缺失模块。
    1、请确保node版本大于6.2
    2、在博客根目录（注意不是yilia根目录）执行以下命令：
    npm i hexo-generator-json-content --save

    3、在根目录_config.yml里添加配置： 

      jsonContent:
    meta: false
    pages: false
    posts:
      title: true
      date: true
      path: true
      text: false
      raw: false
      content: false
      slug: false
      updated: false
      comments: false
      link: false
      permalink: false
      excerpt: false
      categories: false
      tags: true