跳转至

DDTrace



Datakit 内嵌的 DDTrace Agent 用于接收,运算,分析 DataDog Tracing 协议数据。

DDTrace 文档和示例

Tip

DataKit 安装目录 data 目录下,有预先准备好的 dd-java-agent.jar(推荐使用)。也可以直接去 Maven 下载

观测云也在 DDTrace-Java 基础上 Fork 了自己的分支,增加更多功能及探针,更多版本详细信息请查看 Ddtrace 二次开发版本说明

采集器配置

进入 DataKit 安装目录下的 conf.d/ddtrace 目录,复制 ddtrace.conf.sample 并命名为 ddtrace.conf。示例如下:

[[inputs.ddtrace]]
  ## DDTrace Agent endpoints register by version respectively.
  ## Endpoints can be skipped listen by remove them from the list.
  ## Default value set as below. DO NOT MODIFY THESE ENDPOINTS if not necessary.
  endpoints = ["/v0.3/traces", "/v0.4/traces", "/v0.5/traces"]

  ## customer_tags is a list of keys contains keys set by client code like span.SetTag(key, value)
  ## that want to send to data center. Those keys set by client code will take precedence over
  ## keys in [inputs.ddtrace.tags]. DOT(.) IN KEY WILL BE REPLACED BY DASH(_) WHEN SENDING.
  # customer_tags = ["key1", "key2", ...]

  ## Keep rare tracing resources list switch.
  ## If some resources are rare enough(not presend in 1 hour), those resource will always send
  ## to data center and do not consider samplers and filters.
  # keep_rare_resource = false

  ## By default every error presents in span will be send to data center and omit any filters or
  ## sampler. If you want to get rid of some error status, you can set the error status list here.
  # omit_err_status = ["404"]

  ## Ignore tracing resources map like service:[resources...].
  ## The service name is the full service name in current application.
  ## The resource list is regular expressions uses to block resource names.
  ## If you want to block some resources universally under all services, you can set the
  ## service name as "*". Note: double quotes "" cannot be omitted.
  # [inputs.ddtrace.close_resource]
    # service1 = ["resource1", "resource2", ...]
    # service2 = ["resource1", "resource2", ...]
    # "*" = ["close_resource_under_all_services"]
    # ...

  ## Sampler config uses to set global sampling strategy.
  ## sampling_rate used to set global sampling rate.
  # [inputs.ddtrace.sampler]
    # sampling_rate = 1.0

  # [inputs.ddtrace.tags]
    # key1 = "value1"
    # key2 = "value2"
    # ...

  ## Threads config controls how many goroutines an agent cloud start to handle HTTP request.
  ## buffer is the size of jobs' buffering of worker channel.
  ## threads is the total number fo goroutines at running time.
  # [inputs.ddtrace.threads]
    # buffer = 100
    # threads = 8

  ## Storage config a local storage space in hard dirver to cache trace data.
  ## path is the local file path used to cache data.
  ## capacity is total space size(MB) used to store data.
  # [inputs.ddtrace.storage]
    # path = "./ddtrace_storage"
    # capacity = 5120

配置好后,重启 DataKit 即可。

目前可以通过 ConfigMap 方式注入采集器配置来开启采集器。


Attention
  • 不要修改这里的 endpoints 列表。
endpoints = ["/v0.3/traces", "/v0.4/traces", "/v0.5/traces"]
  • 如果要关闭采样(即采集所有数据),采样率字段需做如下设置:
# [inputs.ddtrace.sampler]
# sampling_rate = 1.0

不要只注释 sampling_rate = 1.0 这一行,必须连同 [inputs.ddtrace.sampler] 也一并注释掉,否则采集器会认为 sampling_rate 被置为 0.0,从而导致所有数据都被丢弃。

HTTP 设置

如果 Trace 数据是跨机器发送过来的,那么需要设置 DataKit 的 HTTP 设置

如果有 ddtrace 数据发送给 DataKit,那么在 DataKit 的 monitor 上能看到:

DDtrace 将数据发送给了 /v0.4/traces 接口

开启磁盘缓存

如果 Trace 数据量很大,为避免给主机造成大量的资源开销,可以将 Trace 数据临时缓存到磁盘中,延迟处理:

[inputs.ddtrace.storage]
  path = "/path/to/ddtrace-disk-storage"
  capacity = 5120

DDtrace SDK 配置

配置完采集器之后,还可以对 DDtrace SDK 端做一些配置。

环境变量设置

  • DD_TRACE_ENABLED: Enable global tracer (部分语言平台支持)
  • DD_AGENT_HOST: DDtrace agent host address
  • DD_TRACE_AGENT_PORT: DDtrace agent host port
  • DD_SERVICE: Service name
  • DD_TRACE_SAMPLE_RATE: Set sampling rate
  • DD_VERSION: Application version (optional)
  • DD_TRACE_STARTUP_LOGS: DDtrace logger
  • DD_TRACE_DEBUG: DDtrace debug mode
  • DD_ENV: Application env values
  • DD_TAGS: Application

除了在应用初始化时设置项目名,环境名以及版本号外,还可通过如下两种方式设置:

  • 通过命令行注入环境变量
DD_TAGS="project:your_project_name,env=test,version=v1" ddtrace-run python app.py
  • 在 ddtrace.conf 中直接配置自定义标签。这种方式会影响所有发送给 DataKit tracing 服务的数据,需慎重考虑:
# tags is ddtrace configed key value pairs
[inputs.ddtrace.tags]
  some_tag = "some_value"
  more_tag = "some_other_value"

在代码中添加业务 tag

在应用代码中,可通过诸如 span.SetTag(some-tag-key, some-tag-value)(不同语言方式不同) 这样的方式来设置业务自定义 tag。对于这些业务自定义 tag,可通过在 ddtrace.conf 中配置 customer_tags 来识别并提取:

customer_tags = [
  "order_id",
  "task_id",
  "some.key",  # 被重命名为 some_key
]

注意,这些 tag-key 中不能包含英文字符 '.',带 . 的 tag-key 会替换为 _

应用代码中添加业务 tag 注意事项
  • 在应用代码中添加了对应的 tag 后,必须在 ddtrace.conf 的 customer_tags 中也同步添加对应的 tag-key 列表,否则 DataKit 不会对这些业务 tag 进行提取
  • 在开启了采样的情况下,部分添加了 tag 的 span 有可能被舍弃

指标集

ddtrace

  • 标签
标签名 描述
container_host container hostname
endpoint endpoint info
env application environment info
http_method http request method name
http_status_code http response code
operation span name
project project name
service service name
source_type tracing source type
span_type span type
status span status
version application version info
  • 指标列表
指标 描述 数据类型 单位
duration duration of span int μs
message origin content of span string -
parent_id parent span ID of current span string -
pid application process id. string -
priority int -
resource resource name produce current span string -
span_id span id string -
start start time of span. int usec
trace_id trace id string -

延伸阅读