Skip to content

Commit fcbe4fe

Browse files
wu-shengwu-sheng
authored
Support logback 1.6.x in the logback toolkit layouts (#837)
* Support logback 1.6.x in the logback toolkit layouts Logback 1.6.0 removed PatternLayout.defaultConverterMap, which TraceIdPatternLogbackLayout and TraceIdMDCPatternLogbackLayout wrote their conversion words to in a static initializer, so both failed with NoSuchFieldError: defaultConverterMap (apache/skywalking#14120). The layouts now register their conversion words in the logging context's conversion rule registry (CoreConstants.PATTERN_RULE_REGISTRY) when they start, updating it in place as logback does for <conversionRule>. Logback 1.2.x to 1.6.x all read class names from that registry, so the same toolkit artifact keeps working on older logback. Logback 1.5.13 is not supported because of an upstream regression fixed in 1.5.14 (qos-ch/logback#885). The toolkit now compiles against logback 1.6.5, so an API removal fails the build. LogbackVersionCompatibilityTest renders both layouts against logback 1.2.13, 1.3.16, 1.4.14, 1.5.12, 1.5.14, 1.5.38 and 1.6.5, each loaded in an isolated class loader. Add apm-toolkit-logback-scenario, the first plugin test of the logback toolkit and its agent activation. It runs logback 1.2.x to 1.5.x with the released toolkit and checks the trace ID and SkyWalking context rendered by both layouts, including behind an AsyncAppender, through the gRPC log reporter. * Replace the conversion rule registry instead of modifying it Register the SkyWalking conversion words by replacing the context's conversion rule registry with an updated copy, under the context monitor held only for the copy. Layouts starting concurrently, from any class loader, keep each other's words, and logback reads a registry that is never modified afterwards, so no lock is held while logback creates and starts the converters. Nothing is copied once the words are registered. On logback before 1.5.14, which also writes <conversionRule>s to this registry, a rule logback registers at the same time as a layout starts in another thread may be lost; this is documented.
1 parent ea2fb09 commit fcbe4fe

19 files changed

Lines changed: 1074 additions & 12 deletions

File tree

‎.github/workflows/plugins-jdk17-test.0.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,7 @@ jobs:
8686
- graphql-20plus-scenario
8787
- spring-kafka-3.x-scenario
8888
- undertow-2.3.x-scenario
89+
- apm-toolkit-logback-scenario
8990
steps:
9091
- uses: actions/checkout@v2
9192
with:

‎CHANGES.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,13 @@ Release Notes.
6666
... Could not find artifact org.apache.skywalking:java-agent:pom:9.7.0` (apache/skywalking#13988).
6767
* Stop reporting datasource timeout configuration as metrics. The c3p0 `maxIdleTime`, DBCP `maxWaitMillis`
6868
and HikariCP `connectionTimeout`, `validationTimeout`, `idleTimeout` and `leakDetectionThreshold` gauges are no longer reported.
69+
* Support logback 1.6.x in `apm-toolkit-logback-1.x` (apache/skywalking#14120). `TraceIdPatternLogbackLayout` and
70+
`TraceIdMDCPatternLogbackLayout` failed with `NoSuchFieldError: defaultConverterMap`, because logback 1.6.0 removed
71+
`PatternLayout.defaultConverterMap`. The layouts now register their conversion words in the logging context's
72+
conversion rule registry when they start, which logback 1.2.x to 1.6.x all read, so the same toolkit artifact keeps
73+
working on older logback. Logback 1.5.13 is not supported because of an upstream regression fixed in 1.5.14
74+
(qos-ch/logback#885). The words are no longer registered for the whole JVM, so using `%tid` or `%sw_ctx` in
75+
other encoders, such as a plain `<encoder><pattern>`, requires declaring them as `<conversionRule>`s.
6976

7077
All issues and pull requests are [here](https://github.com/apache/skywalking/milestone/263?closed=1)
7178

‎apm-application-toolkit/apm-toolkit-logback-1.x/pom.xml‎

Lines changed: 118 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,11 @@
2828
<artifactId>apm-toolkit-logback-1.x</artifactId>
2929

3030
<properties>
31-
<logback-classic.version>1.2.3</logback-classic.version>
31+
<!-- Compile against the latest logback, so an API removal fails the build.
32+
Older releases are covered by LogbackVersionCompatibilityTest. -->
33+
<logback-classic.version>1.6.5</logback-classic.version>
3234
<logstash.version>6.1</logstash.version>
35+
<logback-compat.dir>${project.build.directory}/logback-compat</logback-compat.dir>
3336
</properties>
3437

3538
<dependencies>
@@ -46,4 +49,118 @@
4649
<scope>provided</scope>
4750
</dependency>
4851
</dependencies>
52+
53+
<build>
54+
<plugins>
55+
<!-- Logback releases verified by LogbackVersionCompatibilityTest, each loaded in an isolated class loader. -->
56+
<plugin>
57+
<groupId>org.apache.maven.plugins</groupId>
58+
<artifactId>maven-dependency-plugin</artifactId>
59+
<executions>
60+
<execution>
61+
<id>copy-logback-compat-versions</id>
62+
<phase>generate-test-resources</phase>
63+
<goals>
64+
<goal>copy</goal>
65+
</goals>
66+
<configuration>
67+
<skip>${maven.test.skip}</skip>
68+
<outputDirectory>${logback-compat.dir}</outputDirectory>
69+
<artifactItems>
70+
<artifactItem>
71+
<groupId>ch.qos.logback</groupId>
72+
<artifactId>logback-core</artifactId>
73+
<version>1.2.13</version>
74+
</artifactItem>
75+
<artifactItem>
76+
<groupId>ch.qos.logback</groupId>
77+
<artifactId>logback-classic</artifactId>
78+
<version>1.2.13</version>
79+
</artifactItem>
80+
<artifactItem>
81+
<groupId>ch.qos.logback</groupId>
82+
<artifactId>logback-core</artifactId>
83+
<version>1.3.16</version>
84+
</artifactItem>
85+
<artifactItem>
86+
<groupId>ch.qos.logback</groupId>
87+
<artifactId>logback-classic</artifactId>
88+
<version>1.3.16</version>
89+
</artifactItem>
90+
<artifactItem>
91+
<groupId>ch.qos.logback</groupId>
92+
<artifactId>logback-core</artifactId>
93+
<version>1.4.14</version>
94+
</artifactItem>
95+
<artifactItem>
96+
<groupId>ch.qos.logback</groupId>
97+
<artifactId>logback-classic</artifactId>
98+
<version>1.4.14</version>
99+
</artifactItem>
100+
<artifactItem>
101+
<groupId>ch.qos.logback</groupId>
102+
<artifactId>logback-core</artifactId>
103+
<version>1.5.12</version>
104+
</artifactItem>
105+
<artifactItem>
106+
<groupId>ch.qos.logback</groupId>
107+
<artifactId>logback-classic</artifactId>
108+
<version>1.5.12</version>
109+
</artifactItem>
110+
<artifactItem>
111+
<groupId>ch.qos.logback</groupId>
112+
<artifactId>logback-core</artifactId>
113+
<version>1.5.14</version>
114+
</artifactItem>
115+
<artifactItem>
116+
<groupId>ch.qos.logback</groupId>
117+
<artifactId>logback-classic</artifactId>
118+
<version>1.5.14</version>
119+
</artifactItem>
120+
<artifactItem>
121+
<groupId>ch.qos.logback</groupId>
122+
<artifactId>logback-core</artifactId>
123+
<version>1.5.38</version>
124+
</artifactItem>
125+
<artifactItem>
126+
<groupId>ch.qos.logback</groupId>
127+
<artifactId>logback-classic</artifactId>
128+
<version>1.5.38</version>
129+
</artifactItem>
130+
<artifactItem>
131+
<groupId>ch.qos.logback</groupId>
132+
<artifactId>logback-core</artifactId>
133+
<version>1.6.5</version>
134+
</artifactItem>
135+
<artifactItem>
136+
<groupId>ch.qos.logback</groupId>
137+
<artifactId>logback-classic</artifactId>
138+
<version>1.6.5</version>
139+
</artifactItem>
140+
<artifactItem>
141+
<groupId>org.slf4j</groupId>
142+
<artifactId>slf4j-api</artifactId>
143+
<version>1.7.36</version>
144+
</artifactItem>
145+
<artifactItem>
146+
<groupId>org.slf4j</groupId>
147+
<artifactId>slf4j-api</artifactId>
148+
<version>2.0.19</version>
149+
</artifactItem>
150+
</artifactItems>
151+
</configuration>
152+
</execution>
153+
</executions>
154+
</plugin>
155+
<plugin>
156+
<groupId>org.apache.maven.plugins</groupId>
157+
<artifactId>maven-surefire-plugin</artifactId>
158+
<configuration>
159+
<systemPropertyVariables>
160+
<logback.compat.dir>${logback-compat.dir}</logback.compat.dir>
161+
</systemPropertyVariables>
162+
</configuration>
163+
</plugin>
164+
</plugins>
165+
</build>
49166
</project>

‎apm-application-toolkit/apm-toolkit-logback-1.x/src/main/java/org/apache/skywalking/apm/toolkit/log/logback/v1/x/AbstractTraceIdPatternLogbackLayout.java‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*
17+
*/
18+
19+
package org.apache.skywalking.apm.toolkit.log.logback.v1.x;
20+
21+
import ch.qos.logback.classic.PatternLayout;
22+
import ch.qos.logback.core.Context;
23+
import ch.qos.logback.core.CoreConstants;
24+
import java.util.HashMap;
25+
import java.util.Map;
26+
27+
/**
28+
* Registers the SkyWalking conversion words into the logging context before the pattern is compiled.
29+
* <p>
30+
* The context rule registry, keyed by {@link CoreConstants#PATTERN_RULE_REGISTRY} and holding converter class
31+
* names, is read by every logback release from 1.2 to 1.6. Logback 1.5.14+ adapts the class names into converter
32+
* suppliers itself. The static {@code PatternLayout.defaultConverterMap} used before was removed in logback 1.6.0.
33+
* <p>
34+
* The words are registered in the context, so pattern layouts of the context started later can use them too. The
35+
* registry is cleared when the context is reset, and the layouts register them again when they start. The registry
36+
* is replaced by an updated copy, never modified, as layouts starting concurrently may be reading it. On logback
37+
* before 1.5.14, which also writes {@code <conversionRule>}s to this registry, a rule logback registers at the same
38+
* time as a layout starts in another thread may be lost.
39+
* <p>
40+
* Logback 1.5.13 is not supported: it reads this registry as a map of suppliers, which was reverted in 1.5.14.
41+
*/
42+
public abstract class AbstractTraceIdPatternLogbackLayout extends PatternLayout {
43+
44+
@Override
45+
@SuppressWarnings("unchecked")
46+
public void start() {
47+
Context context = getContext();
48+
if (context == null) {
49+
addError("A logging context is required to register the SkyWalking conversion words");
50+
return;
51+
}
52+
53+
Map<String, String> rules = new HashMap<>();
54+
registerConverters(rules);
55+
56+
// Layouts starting concurrently, from any class loader, do not lose each other's words. The monitor is only
57+
// held for the copy, and released before logback creates and starts the converters.
58+
synchronized (context) {
59+
Map<String, String> existing = (Map<String, String>) context.getObject(CoreConstants.PATTERN_RULE_REGISTRY);
60+
if (existing == null || !existing.keySet().containsAll(rules.keySet())) {
61+
Map<String, String> registry = existing == null ? new HashMap<>() : new HashMap<>(existing);
62+
// Rules already registered, such as user-defined <conversionRule>s, keep their precedence.
63+
for (Map.Entry<String, String> rule : rules.entrySet()) {
64+
registry.putIfAbsent(rule.getKey(), rule.getValue());
65+
}
66+
context.putObject(CoreConstants.PATTERN_RULE_REGISTRY, registry);
67+
}
68+
}
69+
70+
super.start();
71+
}
72+
73+
/**
74+
* Puts the conversion words of this layout into {@code rules}, as conversion word to converter class name.
75+
* Subclasses override it to change or add words. Words already registered in the context keep their precedence.
76+
*/
77+
protected abstract void registerConverters(Map<String, String> rules);
78+
}

‎apm-application-toolkit/apm-toolkit-logback-1.x/src/main/java/org/apache/skywalking/apm/toolkit/log/logback/v1/x/TraceIdPatternLogbackLayout.java‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,16 +18,17 @@
1818

1919
package org.apache.skywalking.apm.toolkit.log.logback.v1.x;
2020

21-
import ch.qos.logback.classic.PatternLayout;
21+
import java.util.Map;
2222

2323
/**
2424
* Based on the logback-component convert register mechanism, register {@link LogbackPatternConverter} as a new
2525
* convert, match to "tid" and "sw_ctx". You can use "%tid" or "sw_ctx" in logback config file, "Pattern" section.
2626
* <p>
2727
*/
28-
public class TraceIdPatternLogbackLayout extends PatternLayout {
29-
static {
30-
defaultConverterMap.put("tid", LogbackPatternConverter.class.getName());
31-
defaultConverterMap.put("sw_ctx", LogbackSkyWalkingContextPatternConverter.class.getName());
28+
public class TraceIdPatternLogbackLayout extends AbstractTraceIdPatternLogbackLayout {
29+
@Override
30+
protected void registerConverters(Map<String, String> rules) {
31+
rules.put("tid", LogbackPatternConverter.class.getName());
32+
rules.put("sw_ctx", LogbackSkyWalkingContextPatternConverter.class.getName());
3233
}
3334
}

‎apm-application-toolkit/apm-toolkit-logback-1.x/src/main/java/org/apache/skywalking/apm/toolkit/log/logback/v1/x/mdc/TraceIdMDCPatternLogbackLayout.java‎

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -18,14 +18,17 @@
1818

1919
package org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc;
2020

21-
import ch.qos.logback.classic.PatternLayout;
21+
import java.util.Map;
22+
import org.apache.skywalking.apm.toolkit.log.logback.v1.x.AbstractTraceIdPatternLogbackLayout;
2223

2324
/**
24-
* Override "X" and "mdc",SuperClass run before Subclass.
25+
* Override "X" and "mdc", so "%X{tid}" and "%X{sw_ctx}" print the SkyWalking context. Other MDC keys keep the
26+
* default MDC output.
2527
*/
26-
public class TraceIdMDCPatternLogbackLayout extends PatternLayout {
27-
static {
28-
defaultConverterMap.put("X", LogbackMDCPatternConverter.class.getName());
29-
defaultConverterMap.put("mdc", LogbackMDCPatternConverter.class.getName());
28+
public class TraceIdMDCPatternLogbackLayout extends AbstractTraceIdPatternLogbackLayout {
29+
@Override
30+
protected void registerConverters(Map<String, String> rules) {
31+
rules.put("X", LogbackMDCPatternConverter.class.getName());
32+
rules.put("mdc", LogbackMDCPatternConverter.class.getName());
3033
}
3134
}

‎apm-application-toolkit/apm-toolkit-logback-1.x/src/test/java/org/apache/skywalking/apm/toolkit/log/logback/v1/x/LayoutProbe.java‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*
17+
*/
18+
19+
package org.apache.skywalking.apm.toolkit.log.logback.v1.x;
20+
21+
import ch.qos.logback.classic.Level;
22+
import ch.qos.logback.classic.LoggerContext;
23+
import ch.qos.logback.classic.PatternLayout;
24+
import ch.qos.logback.classic.spi.LoggingEvent;
25+
import org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout;
26+
27+
/**
28+
* Runs inside the class loader of the logback release under test, so it only uses logback APIs that exist from
29+
* 1.2 to 1.6.
30+
*/
31+
public final class LayoutProbe {
32+
33+
private LayoutProbe() {
34+
}
35+
36+
/**
37+
* Starts both SkyWalking layouts in one context, as a logback.xml using both of them does, and renders an event
38+
* with each.
39+
*/
40+
public static String[] render(String tidPattern, String mdcPattern) {
41+
LoggerContext context = new LoggerContext();
42+
PatternLayout tidLayout = start(new TraceIdPatternLogbackLayout(), context, tidPattern);
43+
PatternLayout mdcLayout = start(new TraceIdMDCPatternLogbackLayout(), context, mdcPattern);
44+
45+
LoggingEvent event = new LoggingEvent();
46+
event.setLevel(Level.INFO);
47+
return new String[] {
48+
tidLayout.doLayout(event),
49+
mdcLayout.doLayout(event)
50+
};
51+
}
52+
53+
private static PatternLayout start(PatternLayout layout, LoggerContext context, String pattern) {
54+
layout.setContext(context);
55+
layout.setPattern(pattern);
56+
layout.start();
57+
if (!layout.isStarted()) {
58+
throw new IllegalStateException(layout.getClass().getSimpleName() + " failed to start");
59+
}
60+
return layout;
61+
}
62+
}

0 commit comments

Comments
 (0)