Rust 发送 Trace 到 Jaeger

站长

2024年07月14日 18:23 · 阅读数 30

1. 安装并启动 Jaeger

此处以Windows下开发环境为例，Jaeger 采用 all-in-one 来方便快速启动开发环境。

1.1. 下载 Jaeger 二进制包

从 github release 页下载最新的 Jaeger 指定平台的二进制包，目前，最新版本为 1.59.0，下载地址为: github.com/jaegertraci…。

1.2. 解压缩 Jaeger 二进制包

使用解压缩工具将 Jaeger 二进制包解压缩。

1.3. 运行 `all-in-one`

打开 Windows Terminal，进入上述解压位置，并启动 all-in-one：

cd D:/opt/JaegerTracing/Jaeger
./jaeger-all-in-one.exe

1.4. 测试 Jaeger 是否正常运行

前提条件：已经安装并正确配置 golang

1.4.1. 安装 telemetrygen

go install github.com/open-telemetry/opentelemetry-collector-contrib/cmd/telemetrygen@latest

1.4.2. 使用 telemetrygen 模拟 trace 信息并发送至 Jaeger

1.4.2.1. 采用 `gRPC` 通信

telemetrygen traces --otlp-insecure --duration 5s --otlp-endpoint 127.0.0.1:4317

1.4.2.1.1. 参数解释

tracs: 模拟 trace
otlp-insecure: 允许使用非加密模式
otlp-endpoint: collector 的地址（Jaeger grpc 端口默认是4317）

1.4.2.1.2. 在 Jaeger UI 中查看

在浏览器中打开 http://127.0.0.1:16686 ，并在 Jaeger UI 中查询是否能够查询到 service.name 为 telemetrygen 的 trace 信息： Rust 发送 Trace 到 Jaeger

1.4.2.2. 采用 `HTTP` 通信

为了更好区分 trace 数据的通信方式，此处可以将 1.3. 运行 all-in-one 进程结束，并重新启动。亦或是根据时间来进行判断

使用如下命令开始模拟 trace 信息：

telemetrygen traces --otlp-insecure --duration 5s --otlp-endpoint 127.0.0.1:4318 --otlp-http

1.4.2.2.1. 参数解释

otlp-http: 采用 HTTP 进行传输而非 gRPC，telemetrygen 默认采用 gRPC 进行传输。Jaeger HTTP 默认端口号为 4318。

1.4.2.2.2. 在 Jaeger UI 中查看

参照1.4.2.1.2. 在 Jaeger UI 中查看步骤进行。

2. Rust 接入

2.1. 安装依赖

两种方式安装相关依赖：

在 Cargo.toml 中声明依赖
使用 cargo 工具安装依赖

可能需要安装tokio tracing-test 依赖可用于测试用例

2.1.1. 在 `Cargo.toml` 中声明依赖

在 Cargo.toml 中添加如下依赖声明：

opentelemetry = { version = "0.23.0", features = ["pin-project-lite", "trace", "logs", "logs_level_enabled", "testing" ] } 
opentelemetry-appender-tracing = { version = "0.4.0", features = [ "logs_level_enabled", "log" ] } 
opentelemetry-otlp = { version = "0.16.0", features = [ "default", "grpc-tonic", "trace", "tonic", "http", "tokio", "http-json", "http-proto", "integration-testing", "logs", "serialize", "serde", "gzip-tonic", "serde_json", "reqwest-client" ] } 
opentelemetry-semantic-conventions = "0.15.0" 
opentelemetry_sdk = { version = "0.23.0", features = [ "default", "trace", "async-trait", "percent-encoding", "testing", "jaeger_remote_sampler", "logs", "rt-tokio", "tokio", "tokio-stream", "serde", "opentelemetry-http", "serde_json", "http", "url", "logs_level_enabled" ] }
tracing = {version = "0.1.40", features = ["log", "async-await", "log-always"]}
tracing-appender = "0.2.3" 
tracing-opentelemetry = {version = "0.24.0", features = ["tracing-log", "async-trait", "futures-util", "thiserror"]} 
tracing-subscriber = {version = "0.3.18", features = ["serde", "serde_json", "time", "tracing", "tracing-serde"]} 
tracing-test = "0.2.5"

2.1.2. 使用 `cargo` 工具安装依赖

cargo install opentelemetry --features=pin-project-lite,trace,logs,logs_level_enabled,testing
cargo install opentelemetry-appender-tracing --features=logs_level_enabled,log
cargo install opentelemetry-otlp --features=default,grpc-tonic,trace,tonic,http,tokio,http-json,http-proto,integration-testing,logs,serialize,serde,gzip-tonic,serde_json,reqwest-client
cargo install opentelemetry-semantic-conventions
cargo install opentelemetry_sdk --features=default,trace,async-trait,percent-encoding,testing,jaeger_remote_sampler,logs,rt-tokio,tokio,tokio-stream,serde,opentelemetry-http,serde_json,http,url,logs_level_enabled
cargo install tracing --features=log,async-await,log-always
cargo install tracing-appender
cargo install tracing-opentelemetry --features=tracing-log,async-trait,futures-util,thiserror
cargo install tracing-subscriber --features=serde,serde_json,time,tracing,tracing-serde
carg install tracing-test

2.2. 文件日志和控制台输出日志

2.2.1. 工具模块

创建一个工具模块，方便项目使用。

// 文件：src/misc/utils.rs
use time::{macros::{offset, format_description}, OffsetDateTime, UtcOffset};

pub struct Utils;

impl Utils {
    /// 默认时区
    ///
    /// 默认时区为：UTC+08:00:00
    ///
    /// # Return
    /// `time::UtcOffset` - 时区偏移
    pub fn default_timezone() -> UtcOffset {
        offset!(+08:00)
    }
    
    /// 获取当前时间
    ///
    /// 时区设定为默认时区
    ///
    /// # Return
    /// `time::OffsetDateTime` - 当前时间
    pub fn now() -> OffsetDateTime {
        OffsetDateTime::now_utc().to_offset(Self::default_timezone())
    }
    
    /// 默认时间格式化格式
    ///
    /// 默认时间格式化格式为：RFC3339，精度为秒
    /// 
    /// 定义方式见：[time-rs.github.io](https://time-rs.github.io/book/api/format-description.html)
    ///
    /// # Return
    /// `&'static [time::format_description::FormatItem<'static>]` - 默认时间格式化格式
    pub fn default_datetime_format() -> &'static [FormatItem<'static>] {
        format_description!("[year]-[month]-[day]T[hour]:[minute]:[second][offset_hour sign:mandatory]:[offset_minute]")
    }
}

2.2.2. tracing 时间配置

定义一个结构体，用于处理日志时间格式。

// 文件：src/misc/tracing_time.rs
use tracing_subscriber::fmt::time::FormatTime;
use super::utils::Utils;

#[derive(Clone)]
pub struct TracingTimer;

impl FormatTime for TracingTimer {
    fn format_time(&self, w: &mut tracing_subscriber::fmt::format::Writer<'_>) -> std::fmt::Result {
        let now_str = Utils::now()
            .format(Utils::default_datetime_format())
            .unwrap_or(String::from("1970-01-01T07:59:59+08:00"));
        w.write_str(&now_str)
    }
}

2.2.3. 配置输出到控制台和日志文件的 tracing 配置

需要在 main 函数中配置

2.2.3.1. 获取日志路径和创建日志路径

笔者采用 clap 从命令行接收日志路径配置，关于 clap 的使用方式不在赘述，此处以 ./logs 目录直接取代其值。

// src/main.rs
use std::fs::{self, Path};
let log_dir = "./logs";
let log_dir_path = Path::new(&log_dir);
if !log_dir_path.exists() {
    if let Err(error) = fs::create_dir_all(log_dir_path) {
        panic!(
            "日志目录不存在且创建失败, 指定的日志路径: {}, 错误信息: {:?}",
            log_dir, error
        )
    }
}

2.2.3.2. 配置默认日志时间格式

// src/main.rs
let timer = TracingTimer {};

2.2.3.3. 配置 stdout 日志 layer

// src/main.rs
let stdout_layer = tracing_subscriber::fmt::layer()
        // 日志中是否包含文件名
        .with_file(true)
        // 日志中是否包含日志级别
        .with_level(true)
        // 日志中是否包含行号
        .with_line_number(true)
        // 是否包含颜色显示
        .with_ansi(true)
        .with_target(true)
        .with_timer(timer.clone())
        .with_writer(std::io::stdout)
        .with_filter(tracing_subscriber::filter::LevelFilter::INFO);

2.2.3.4. 配置输出到文件的日志 layer

// src/main.rs
const APP_NAME: &str = env!("CARGO_PKG_NAME");
let app_name = APP_NAME;
let log_file = format!("{}.log", app_name);
let appender = tracing_appender::rolling::never(log_dir, log_file);
// 如果需要日志轮换，则使用如下方式
// let appender = tracing_appender::rolling::daily(log_dir, log_file);

let (non_blocking, _guard) = tracing_appender::non_blocking(appender);
let file_layer = tracing_subscriber::fmt::layer()
    .with_ansi(false)
    .with_target(true)
    .with_file(true)
    .with_level(true)
    .with_line_number(true)
    .with_timer(timer.clone())
    .with_writer(non_blocking)
    .with_filter(tracing_subscriber::filter::LevelFilter::INFO);

2.2.3.5. 配置 OpenTelemetry（使用 `gRPC`）

协议 Metadata （可选）

此操作是可选的，但如果 Jaeger 启用了 Authorization，则此步骤是必须的

该步骤操作一个 tonic MetadataMap ，用于在传输时携带一些额外的数据至 collector。

use tonic::metadata::{MetadataKey as TonicMetadataKey, MetadataMap as TonicMetadataMap, MetadataValue as TonicMetadataValue};

let mut opentelemetry_metadata = TonicMetadataMap::new();
// 此处以 service.name 为例，可以按需进行配置
opentelemetry_metadata.insert(TonicMetadataKey::from_static(opentelemetry_semantic_conventions::trace::SERVICE_NAME), TonicMetadataValue::from_static(APP_NAME));

构造 SpanExporter

let opentelemetry_trace_exporter = opentelemetry_otlp::new_exporter().tonic().with_metadata(opentelemetry_metadata).with_endpoint("http://127.0.0.1:4317").build_span_exporter().unwrap();

构造 TracerProvider

use opentelemetry::{KeyValue as OpenTelemetryKeyValue, Key as OpenTelemetryKey, Value as OpenTelemetryValue};
use opentelemetry_sdk::Resource as OpenTelemetryResource;
use opentelemetry::trace::TracerProvider as _;
use opentelemetry_sdk::trace::TracerProvider as OpenTelemetryTracerProvider;

// 设置 service.name
let opentelemetry_trace_resource = vec![OpenTelemetryKeyValue{key: OpenTelemetryKey::from(opentelemetry_semantic_conventions::resource::SERVICE_NAME), value: OpenTelemetryValue::from(APP_NAME)}];
let opentelemetry_trace_resource = OpenTelemetryResource::new(opentelemetry_trace_resource);
// 构造 TracerProvider
OpenTelemetryTracerProvider::builder().with_batch_exporter(opentelemetry_trace_exporter, opentelemetry_sdk::runtime::Tokio).with_config(opentelemetry_sdk::trace::Config::default().with_resource(opentelemetr_trace_resource)).build();

桥接 tracing

let opentelemetry_tracer = tracer_provider.tracer(APP_NAME);

构造 layer

let opentelemetry_layer = tracing_opentelemetry::layer().with_tracer(tracer);

let subscriber = tracing_subscriber::Registry::default()
    .with(stdout_layer)
    .with(file_layer)
    .with(opentelemetry_layer);

2.2.3.6. 配置 OpenTelemetry（使用 `http`）

若需要使用 http，只需要将 2.2.3.5. 配置 OpenTelemetry（使用 gRPC）中的第1节中的 MetadataMap 替换为 HashMap，将第2节中的 tonic 替换为 http ，将with_metadata 替换为 with_heade 即可。如下：

let mut opentelemetry_header = HashMap::new();
opentelemetry_header.insert(opentelemetry_semantic_conventions::trace::SERVICE_NAME.to_string(), APP_NAME.to_string());
let logger_exporter = opentelemetry_otlp::new_exporter()
    .http()
    .with_endpoint("http://127.0.0.1:4318")
    .with_headers(opentelemetry_header)
    .build_span_exporter()
    .unwrap();

3. 完整代码片段

3.1. 依赖

// cargo.toml
opentelemetry = { version = "0.23.0", features = ["pin-project-lite", "trace", "logs", "logs_level_enabled", "testing" ] } 
opentelemetry-appender-tracing = { version = "0.4.0", features = [ "logs_level_enabled", "log" ] } 
opentelemetry-otlp = { version = "0.16.0", features = [ "default", "grpc-tonic", "trace", "tonic", "http", "tokio", "http-json", "http-proto", "integration-testing", "logs", "serialize", "serde", "gzip-tonic", "serde_json", "reqwest-client" ] } 
opentelemetry-semantic-conventions = "0.15.0" 
opentelemetry_sdk = { version = "0.23.0", features = [ "default", "trace", "async-trait", "percent-encoding", "testing", "jaeger_remote_sampler", "logs", "rt-tokio", "tokio", "tokio-stream", "serde", "opentelemetry-http", "serde_json", "http", "url", "logs_level_enabled" ] }
tracing = {version = "0.1.40", features = ["log", "async-await", "log-always"]}
tracing-appender = "0.2.3" 
tracing-opentelemetry = {version = "0.24.0", features = ["tracing-log", "async-trait", "futures-util", "thiserror"]} 
tracing-subscriber = {version = "0.3.18", features = ["serde", "serde_json", "time", "tracing", "tracing-serde"]} 
tracing-test = "0.2.5"

3.2. 助手模块

// src/misc/utils.rs
use time::{macros::{offset, format_description}, OffsetDateTime, UtcOffset};

pub struct Utils;

impl Utils {
    /// 默认时区
    ///
    /// 默认时区为：UTC+08:00:00
    ///
    /// # Return
    /// `time::UtcOffset` - 时区偏移
    pub fn default_timezone() -> UtcOffset {
        offset!(+08:00)
    }
    
    /// 获取当前时间
    ///
    /// 时区设定为默认时区
    ///
    /// # Return
    /// `time::OffsetDateTime` - 当前时间
    pub fn now() -> OffsetDateTime {
        OffsetDateTime::now_utc().to_offset(Self::default_timezone())
    }
    
    /// 默认时间格式化格式
    ///
    /// 默认时间格式化格式为：RFC3339，精度为秒
    /// 
    /// 定义方式见：[time-rs.github.io](https://time-rs.github.io/book/api/format-description.html)
    ///
    /// # Return
    /// `&'static [time::format_description::FormatItem<'static>]` - 默认时间格式化格式
    pub fn default_datetime_format() -> &'static [FormatItem<'static>] {
        format_description!("[year]-[month]-[day]T[hour]:[minute]:[second][offset_hour sign:mandatory]:[offset_minute]")
    }
}

3.3. Trace Time 模块

// src/misc/tracing_time.rs
use tracing_subscriber::fmt::time::FormatTime;
use super::utils::Utils;

#[derive(Clone)]
pub struct TracingTimer;

impl FormatTime for TracingTimer {
    fn format_time(&self, w: &mut tracing_subscriber::fmt::format::Writer<'_>) -> std::fmt::Result {
        let now_str = Utils::now()
            .format(Utils::default_datetime_format())
            .unwrap_or(String::from("1970-01-01T07:59:59+08:00"));
        w.write_str(&now_str)
    }
}

3.4. Trace

// src/main.rs
#[tokio::main]
async fn main() {
    use std::fs::{self, Path};
    let log_dir = "./logs";
    let log_dir_path = Path::new(&log_dir);
    if !log_dir_path.exists() {
        if let Err(error) = fs::create_dir_all(log_dir_path) {
            panic!(
                "日志目录不存在且创建失败, 指定的日志路径: {}, 错误信息: {:?}",
                log_dir, error
            )
        }
    }

    let timer = TracingTimer {};
    // 输出到 stdout
    let stdout_layer = tracing_subscriber::fmt::layer()
        // 日志中是否包含文件名
        .with_file(true)
        // 日志中是否包含日志级别
        .with_level(true)
        // 日志中是否包含行号
        .with_line_number(true)
        // 是否包含颜色显示
        .with_ansi(true)
        .with_target(true)
        .with_timer(timer.clone())
        .with_writer(std::io::stdout)
        .with_filter(tracing_subscriber::filter::LevelFilter::INFO);
    // 输出到文件
    let app_name = APP_NAME;
    let log_file = format!("{}.log", app_name);
    let appender = tracing_appender::rolling::never(log_dir, log_file);
    // 如果需要进行日志轮换，则使用如下方式
    let appender = tracing_appender::rolling::daily(log_dir, log_file);

    let (non_blocking, _guard) = tracing_appender::non_blocking(appender);
    let file_layer = tracing_subscriber::fmt::layer()
        .with_ansi(false)
        .with_target(true)
        .with_file(true)
        .with_level(true)
        .with_line_number(true)
        .with_timer(timer.clone())
        .with_writer(non_blocking)
        .with_filter(tracing_subscriber::filter::LevelFilter::INFO);
    let mut opentelemetry_metadata = MetadataMap::new();
    opentelemetry_metadata.insert(MetadataKey::from_static(opentelemetry_semantic_conventions::trace::SERVICE_NAME), MetadataValue::from_static(APP_NAME));
    // let mut opentelemetry_header = HashMap::new();
    // opentelemetry_header.insert(opentelemetry_semantic_conventions::trace::SERVICE_NAME.to_string(), APP_NAME.to_string());
    let logger_exporter = opentelemetry_otlp::new_exporter()
        .tonic()
        // .http()
        .with_endpoint("http://127.0.0.1:4317")
        // .with_headers(opentelemetry_header)
        .with_metadata(opentelemetry_metadata)
        .build_span_exporter()
        .unwrap();
    // let logger_provider = LoggerProvider::builder()
    //     .with_batch_exporter(logger_exporter, opentelemetry_sdk::runtime::Tokio)
    //     .build();
    // let opentelemetry_layer = OpenTelemetryTracingBridge::new(&logger_provider);
    let opentelemetry_trace_resource = vec![OpenTelemetryKeyValue{key: OpenTelemetryKey::from(opentelemetry_semantic_conventions::resource::SERVICE_NAME), value: OpenTelemetryValue::from(APP_NAME)}];
    let opentelemetr_trace_resource = Resource::new(opentelemetry_trace_resource);
    let tracer_provider = TracerProvider::builder().with_batch_exporter(logger_exporter, opentelemetry_sdk::runtime::Tokio).with_config(opentelemetry_sdk::trace::Config::default().with_resource(opentelemetr_trace_resource))/*.with_simple_exporter(logger_exporter)*/.build();
    let tracer = tracer_provider.tracer(APP_NAME);
    let opentelemetry_layer = tracing_opentelemetry::layer().with_tracer(tracer);
    let subscriber = tracing_subscriber::Registry::default()
        .with(console_layer)
        .with(file_layer)
        .with(opentelemetry_layer);

    if let Err(error) = tracing::subscriber::set_global_default(subscriber) {
        panic!("初始化日志模块失败, 错误信息: {:?}", error);
    }
    info!("初始化日志模块完成!");
}

4. 问题

4.1. 日志轮换

不建议在应用中进行日志轮换，可以选择使用 logrotate 结合 crontab 实现日志轮换。

4.2. 关于 Jaeger Collector 的认证说明

Jaeger Collector 不提供认证模块，因此如果需要实现 Jaeger Collector 的认证模块，则可以采用 Nginx、HAProxy、KeyCloak 等服务容器进行转发，并在服务容器中配置身份认证模块。而在代码中则可以在Metadata （gRPC）中或 Header （HTTP）设置authorization 信息。

4.3. 关于 `Jaeger All-In-One` 的说明

Jaeger 默认使用内存作为存储后端，可以自行灵活搭配 Cassandra 或者 ES，同时也可以搭配 kafka 中间件。

4.4. 关于 `HTTP` 模式下 `no-client` 的问题

需要在安装 opentelemetry-otlp 依赖的时候同时启用 reqwest-client

4.5. 关于 `tonic` 的 `MetadataMap` 与 `opentelemetry_otlp` 的 `with_metadata` 不兼容

这是因为 opentelemetry_otlp=0.16.0 与 tonic=0.12.0 不兼容导致，将 tonic 降级为 0.11.0 即可解决

转载自:https://juejin.cn/post/7390629662836637748