为什么我们需要日志追踪?
想象一下,在一个大型的微服务架构中,一次用户的请求可能会经过网关、用户服务、订单服务、支付服务等多个微服务。当出现问题时,如果没有日志追踪,我们看到的就是一堆杂乱无章的日志,根本不知道哪个请求对应哪些日志,排查问题简直比登天还难!
这就是我们引入日志追踪的根本原因:让每一次请求都有迹可循,让问题定位变得简单高效!
优秀的日志追踪,我们要解决哪些问题?
- Servlet请求:单服务内,串联request到response之间主线程的所有日志
- 微服务:RPC调用传递traceId至下游服务,下游服务同样完成Servlet请求
- 消息队列:生产者传递traceId至消费者
- 定时任务:类似Servlet请求,本质上是单独的触发点
- 异步线程池:并发编程中父线程传递traceId给子线程
要解决以上这些问题,我有两种全局性方案分享给大家(文末附代码链接)
- 传统MDC(Mapped Diagnostic Context)方案
- 王炸级方案SkyWalking
传统MDC方案
1. Servlet请求处理:过滤器从请求头获取traceId(自生成或来自上游服务)
@Component @Order(CommonConstants.FilterOrdered.TRACE) public class TraceFilter extends OncePerRequestFilter { @Autowired private TraceManager traceManager; @Override protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain chain) throws IOException, ServletException { String traceId = request.getHeader(HeaderConstants.TRACE_ID); traceManager.put(traceId); try { chain.doFilter(request, response); } finally { traceManager.remove(); } } }2. 微服务间的RPC调用
- Feign:通过拦截器传递traceId
@Component public class TraceHeaderInterceptor implements RequestInterceptor { @Autowired private TraceManager traceManager; @Override public void apply(RequestTemplate template) { traceManager.headers().forEach((template::header)); } }- RestTemplate:通过拦截器传递traceId
1.实现拦截器 public class TraceHeaderClientHttpRequestInterceptor implements ClientHttpRequestInterceptor { private final TraceManager traceManager; public TraceHeaderClientHttpRequestInterceptor(TraceManager traceManager) { this.traceManager = traceManager; } @Override public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException { Map<String, List<String>> headers = Maps.newHashMap(); // 日志追踪ID headers.putAll(traceManager.headers()); HttpHeaders httpHeaders = request.getHeaders(); if (!headers.isEmpty()) { httpHeaders.putAll(headers); } return execution.execute(request, body); } } 2.装配拦截器 @LoadBalanced @Bean("restTemplate") public RestTemplate restTemplate(TraceManager traceManager) { RestTemplate restTemplate = new RestTemplate(httpRequestFactory()); List<ClientHttpRequestInterceptor> interceptors = new ArrayList<>(); // 将拦截器添加到restTemplate interceptors.add(new TraceHeaderClientHttpRequestInterceptor(traceManager)); restTemplate.setInterceptors(interceptors); return restTemplate; }3. 消息队列:通过切面编程实现
1.生产者将traceId放入header,传给消费者 public class RocketmqMessageSender implements MessageSender { @Autowired private RocketMQTemplate rocketMQTemplate; @Autowired private TraceManager traceManager; private void sendMessage(Object toJson, String topic, String tag) { String paramsStr = JsonUtil.toJsonString(toJson); Map<String, Object> headers = Maps.newHashMap(); // 生产者将traceId放入header headers.put(HeaderConstants.HEADER_MESSAGE_ID, traceManager.get()); MessageHeaders messageHeaders = new MessageHeaders(headers); Message<String> message = MessageBuilder.createMessage(paramsStr, messageHeaders); // 发送消息... } } 2.消费者取出traceId并放入消费者的上下文 public class TraceAspect { @Autowired private TraceManager traceManager; // 执行之前塞入日志ID,用于追踪整个执行链路 @Before("execution(* org.apache.rocketmq.spring.core.RocketMQListener+.onMessage(org.apache.rocketmq.common.message.MessageExt)) && @within(org.apache.rocketmq.spring.annotation.RocketMQMessageListener)") public void setTraceId(JoinPoint joinPoint) { Object[] args = joinPoint.getArgs();// 获取方法参数 MessageExt messageExt = (MessageExt) args[0]; Map<String, String> properties = messageExt.getProperties(); traceManager.put(properties.get(HeaderConstants.HEADER_MESSAGE_ID)); log.info("初始化日志ID"); } // 执行完毕之后清理日志ID @After("execution(* org.apache.rocketmq.spring.core.RocketMQListener+.onMessage(..)) && @within(org.apache.rocketmq.spring.annotation.RocketMQMessageListener)") public void clearTraceId() { log.info("清除日志ID"); traceManager.remove(); } }4. 定时任务:通过切面编程实现
public class TraceAspect { @Autowired private TraceManager traceManager; // 在jobHander执行之前塞入日志ID,用于追踪整个执行链路 @Before("@annotation(com.xxl.job.core.handler.annotation.XxlJob)") public void setTraceId() { traceManager.put(); log.info("初始化日志ID"); } // 执行完毕之后清理日志ID @After("@annotation(com.xxl.job.core.handler.annotation.XxlJob)") public void clearTraceId() { log.info("清除日志ID"); traceManager.remove(); } }5. 异步线程池(难点)
- ThreadPoolTaskExecutor(Spring环境推荐使用):通过其TaskDecorator装饰器
1.自定义Runnable public class TraceRunnable implements Runnable { private final Runnable target; private final TraceManager traceManager; private String traceId; public TraceRunnable(Runnable target, TraceManager traceManager, String traceId) { // 这里是主线程执行,提交Runnable时通过参数方式将traceId保存到成员变量 this.target = target; this.traceManager = traceManager; this.traceId = traceId; } @Override public void run() { try { // 这里是子线程执行,将成员变量traceId赋值给MDC traceManager.put(traceId); target.run(); } finally { traceManager.remove(); } } } 2.实现TaskDecorator public class TraceTaskDecorator implements TaskDecorator { protected TraceManager traceManager; public TraceTaskDecorator(TraceManager traceManager) { this.traceManager = traceManager; } @Override public Runnable decorate(Runnable runnable) { // 这里已经是子线程了,将成员变量traceId赋值给MDC return new TraceRunnable(runnable, traceManager, traceManager.get()); } } 3.装配taskDecorator @Bean @ConditionalOnMissingBean public TaskDecorator taskDecorator(TraceManager traceManager) { return new TraceTaskDecorator(traceManager);// 传递日志id } @Bean @ConditionalOnMissingBean public AsyncTaskExecutor taskExecutor(TaskDecorator taskDecorator) { ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor(); // 将TraceTaskDecorator实例设置到线程池taskDecorator executor.setTaskDecorator(taskDecorator); // 设置线程池其他参数... return executor; }- ThreadPoolExecutor:参考ThreadPoolTaskExecutor通过TaskDecorator装饰器
1.ThreadPoolExecutor没有TaskDecorato参数,自定义线程池参考ThreadPoolTaskExecutor实现 public class TaskDecoratorThreadPoolExecutor extends ThreadPoolExecutor { private TaskDecorator taskDecorator; public TaskDecoratorThreadPoolExecutor(父构造函数参数..., TaskDecorator taskDecorator) { super(父构造函数参数...); this.taskDecorator = taskDecorator; } @Override public void execute(Runnable command) { // 核心重写execute方法,使用taskDecorator处理Runnable Runnable decorated = taskDecorator.decorate(command); super.execute(decorated); } } @Bean // SpringBoot中建议使用ThreadPoolTaskExecutor @ConditionalOnMissingBean public ExecutorService executorService(TaskDecorator taskDecorator) { // 将TraceTaskDecorator实例设置到线程池taskDecorator ExecutorService executor = new TaskDecoratorThreadPoolExecutor(线程池其他参数..., taskDecorator); return executor; }- ForkJoinPool(难点中的难点,parallelStream、CompletableFuture底层线程池):通过TTL(TransmittableThreadLocal)解决
1.引入transmittable-thread-local解决ThreadLocal无法父子线程上下文 <dependency> <groupId>com.alibaba</groupId> <artifactId>transmittable-thread-local</artifactId> <version>2.14.5</version> </dependency> 注:启动参数需加上:-javaagent:path/to/transmittable-thread-local-2.x.x.jar 2.引入logback-mdc-ttl,核心逻辑是自定义TtlMDCAdapter替换掉默认的LogbackMDCAdapter <dependency> <groupId>com.ofpay</groupId> <artifactId>logback-mdc-ttl</artifactId> <version>1.0.2</version> </dependency> 3.logback-spring.xml配置中添加contextListener <!-- 替换普通MDC,利用transmittable-thread-local解决ForkJoinPool日志ID传递问题,启动参数加上:-javaagent:path/to/transmittable-thread-local-2.x.x.jar --> <contextListener class="com.ofpay.logback.TtlMdcListener"/>关于ForkJoinPool的详细可阅读我另外一篇文章《一次解决ForkJoinPool日志追踪的辛酸经历》
王炸级方案SkyWalking
SkyWalking作为一款开源的APM(应用性能管理)工具,为我们提供了更优雅的解决方案:
- 自动化追踪:无需修改业务代码,通过字节码增强技术自动完成追踪
- 全场景覆盖:无论是同步请求、异步处理还是中间件调用,都能完美支持
- 丰富的插件生态:支持主流框架和中间件
1. Servlet请求处理:无需修改代码
2. 微服务间的RPC调用
- Feign:无需修改代码
- RestTemplate:无需修改代码
3. 消息队列:无需修改代码
`skywalking-agent/plugins` 目录已包含 `apm-rocketMQ-5.x-plugin-8.16.0.jar`
4. 定时任务:无需修改代码
`skywalking-agent/plugins` 目录已包含 `apm-xxl-job-2.x-plugin-8.16.0.jar`
5. 异步线程池(不难了)
- ThreadPoolTaskExecutor(Spring环境推荐使用):无需修改代码
将`/skywalking-agent/bootstrap-plugins/apm-jdk-threadpool-plugin-8.16.0.jar`复制到 `/plugins` 目录
- ThreadPoolExecutor:无需修改代码
将`/skywalking-agent/bootstrap-plugins/apm-jdk-threadpool-plugin-8.16.0.jar`复制到 `/plugins` 目录
- ForkJoinPool(不难了,parallelStream、CompletableFuture底层线程池):无需修改代码
将`/skywalking-agent/bootstrap-plugins/apm-jdk-forkjoinpool-plugin-8.16.0.jar`复制到 `/plugins` 目录
MDC vs SkyWalking:优劣对比
| 方案 | 优势 | 劣势 |
| MDC | 无外部依赖灵活可控,可根据业务需求定制 | 代码侵入性强,维护成本高 |
| SkyWalking | 零侵入,全场景覆盖,功能强大 | 依赖外部服务服务不可用traceId输出N/A |
可尝试:两种方案结合使用
MDC的traceId生成策略中,读取SkyWalking的traceId,如果SkyWalking不可用,则回退到随机生成traceId,这样就可以保证traceId永远有值
public class SkywalkingProvider implements TraceIdProvider { private final TraceIdProvider fallbackProvider; public SkywalkingProvider(TraceIdProvider fallbackProvider) { this.fallbackProvider = fallbackProvider; } @Override public String generateTraceId() { String traceId = TraceContext.traceId(); if (StringUtils.isBlank(traceId) || "Ignored_Trace".equalsIgnoreCase(traceId) || "N/A".equalsIgnoreCase(traceId)) { traceId = fallbackProvider.generateTraceId(); // 回退到传统方案 } return traceId; } }总结与展望
通过本文的介绍,我们看到了SkyWalking在微服务日志追踪领域的巨大价值。从传统的MDC手动配置到SkyWalking的自动化追踪,这不仅是技术的进步,更是开发效率的显著提升。
对于正在构建或维护微服务架构的团队来说,SkyWalking绝对是一个值得投入的技术选型。它不仅能帮助我们快速定位问题,还能提供全面的性能监控数据,为系统的优化提供有力支撑。
记住:好的日志追踪不是锦上添花,而是微服务架构的必备基础设施!
如果你也在使用微服务架构,不妨试试SkyWalking,相信它会给你带来意想不到的惊喜!