Cloudtop 开源前端性能监控系统设计与开发二（日志数据字段设计）

站长

2024年04月12日 20:06 · 阅读数 10

本文主要描述 Cloudtop 前端性能监控日志字段的含义，用于检索和排查问题。

代码完全开源，支持二次开发；如有错漏，欢迎指出。

Github 开源项目地址：

github.com/zhedh/cloud…

公共字段

字段	类型	示例	说明
pid	string	cloudtop_admin	应用 ID 或应用标识，应用唯一值
type	LogType	pv	日志类型
report_time	number	1711540240461	日志上报时间（毫秒）
date	number	1711540240461	默认取 reportTime，其次取服务器时间
env	LogEnv	production	环境
ct	string	4g	网络类型
http_referer	string	example.com	页面 Referer
http_user_agent	string	Mozilla/5.0***605.1.15	浏览器 UserAgent
browser	string	safari	浏览器类型
browser_version	string	14.1.1	浏览器版本
device	string	mac	设备类型
engine	string	webkit	浏览器引擎
engine_version	string	605.1.15	浏览器引擎版本
os	string	macos	操作系统
os_version	string	10.15.7	操作系统版本
device_type	string	pc	设备类型 pc、mobile
sid	string	pv	Session ID
sr	string	400*892	屏幕分辨率
vp	string	400*813	屏幕分辨率
src	string	www.example.com	被监控目标地址
pv_id	string	bkkR****2R2Xp	PV ID
uid	string	e023f1****3de48	系统生成
page	string	/testing	页面、页面 path
login_id	string	123****987	用户登录账号、用于匹配用户、自定义
ext	string	'{"a":1,"b":2}'	扩展字段，不要超过 100 个字符

日志类型（LogType）枚举：

值	说明
pv	PV 日志
error	JS 错误日志
resource_error	资源引用错误日志
perf	性能日志
api	接口日志

日志环境（LogEnv）枚举：

值	说明
production	生产环境
staging	预发布环境
testing	测试环境
development	开发环境

PV 日志

字段	类型	示例	说明
dt	string	cloudtop 性能监控面板	文档标题
dr	string	example.com	文档 Referrer
de	string	utf-8	文档编码类型
dpr	string	1	屏幕像素比
lang	string	en	文档语言
url	string	example.com/tools	页面链接

上报数据示例：

{
  "pid": "cloud***_admin",
  "type": "pv",
  "env": "production",
  "report_time": 1711609330415,
  "date": 1711609330415,
  "ct": "4g",
  "http_referer": "https://www.****.com/",
  "http_user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36 NetType/WIFI MicroMessenger/7.0.20.1781(0x6700143B) WindowsWechat(0x6309092b) XWEB/9079 Flue",
  "browser": "WeChat",
  "browser_version": "7.0.20.1781",
  "device": "",
  "engine": "Blink",
  "engine_version": "116.0.0.0",
  "os": "Windows",
  "os_version": "10",
  "device_type": "pc",
  "ip_isp": "移动",
  "ip_country": "中国",
  "ip_country_id": "CN",
  "ip_region": "重庆",
  "ip_region_id": "50",
  "ip_region_name": "重庆市",
  "ip_city": "重庆城区",
  "ip_city_id": "5001",
  "ip_city_name": "重庆城区",
  "remote_addr": "183.230.119.132",
  "sid": "",
  "pv_id": "eb157b0c-f557-47a8-8a90-88a86950e312",
  "page": "/data/trade",
  "src": "https://www.***.com/data/trade",
  "sr": "1280*800",
  "vp": "1280*719",
  "uid": "ae3525b0-0181-4aab-97d5-eca564e7a87e",
  "login_id": "138***",
  "dt": "品牌云店",
  "dr": "https://login.***.com/",
  "de": "UTF-8",
  "dpr": "1.5",
  "lang": "en",
  "url": "https://www.***.com/data/trade"
}

Error 日志

字段	类型	示例	说明
category	string	ReferenceError	错误分类，默认为 CustomError
msg	string	example.com	错误消息，最大支持 1000 个字符
error	string	-	错误详情
stack	string	-	错误堆栈信息，最大支持 1000 个字符
file	string	example.com/assets/inde…	报错 JS 所属文件
line	string	60	报错发生行
col	string	1844	报错发生列

上报数据示例：

{
  "pid": "cloud***_admin",
  "type": "error",
  "env": "production",
  "report_time": 1711608515413,
  "date": 1711608515413,
  "ct": "4g",
  "http_referer": "https://www.***.com/",
  "http_user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36",
  "browser": "Chrome",
  "browser_version": "122.0.0.0",
  "device": "",
  "engine": "Blink",
  "engine_version": "122.0.0.0",
  "os": "Windows",
  "os_version": "10",
  "device_type": "pc",
  "ip_isp": "联通",
  "ip_country": "中国",
  "ip_country_id": "CN",
  "ip_region": "上海",
  "ip_region_id": "31",
  "ip_region_name": "上海市",
  "ip_city": "上海",
  "ip_city_id": "3101",
  "ip_city_name": "上海市",
  "remote_addr": "210.13.84.10",
  "sid": "",
  "pv_id": "e895589f-5f52-4e28-8dce-9b534626d43c",
  "page": "/commission/commissionlist/commissionadd",
  "src": "https://www.***.com/commission/commissionlist/commissionadd",
  "sr": "1536*864",
  "vp": "1536*714",
  "uid": "4b6468be-fb87-4157-b31b-2d121bd36edc",
  "login_id": "136****",
  "category": "TypeError",
  "msg": "Cannot read properties of null (reading 'bizType')",
  "stack": "TypeError: Cannot read properties of null (reading 'bizType')\n    at Z (https://www.***.com/assets/index.040f44f5.js:60:1844)",
  "file": "",
  "line": 60,
  "col": 1844,
  "error": {
    "name": "TypeError",
    "message": "Cannot read properties of null (reading 'bizType')",
    "stack": "TypeError: Cannot read properties of null (reading 'bizType')\n    at Z (https://www.***.com/assets/index.040f44f5.js:60:1844)"
  }
}

ResourceError 日志

字段	类型	示例	说明
src	string	example.com/suv4/rMainB…	资源地址
node_name	string	SCRIPT	发生错误的 HTML 节点类型
xpath	string	html.body.script	发生错误位置
res_type	string	script	错误资源类型
res_name	string	/suv4/rMainB.bundle.js	错误资源名称
domain	string	example.com	错误资源域名

上报数据示例：

{
  "pid": "cloudshop_admin",
  "type": "resource_error",
  "env": "production",
  "report_time": 1711413678276,
  "date": 1711413678276,
  "ct": "4g",
  "http_referer": "https://www.example.com/",
  "http_user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36",
  "browser": "Chrome",
  "browser_version": "117.0.0.0",
  "device": "",
  "engine": "Blink",
  "engine_version": "117.0.0.0",
  "os": "Windows",
  "os_version": "10",
  "device_type": "pc",
  "ip_isp": "内网IP",
  "ip_country": "",
  "ip_country_id": "",
  "ip_region": "",
  "ip_region_id": "",
  "ip_city": "",
  "ip_city_id": "",
  "remote_addr": "172.**.**.135",
  "sid": "",
  "pv_id": "0c41ee34-cd54-45df-95b9-f1b4e6a0404b",
  "page": "/",
  "src": "https://www.example.com/suv4/rMainB.bundle.js?md5=bc82bf95c5d2dd6abc9dec6fb91f48be",
  "sr": "1280*720",
  "vp": "1280*581",
  "uid": "237b4fd4-2ad0-47ed-af72-4f3ed9bc5059",
  "login_id": "138***",
  "node_name": "SCRIPT",
  "xpath": "html.body.script",
  "res_type": "script",
  "res_name": "/suv4/rMainB.bundle.js?md5=bc82bf95c5d2dd6abc9dec6fb91f48be",
  "domain": "www.example.com"
}

Perf 日志

字段	类型	示例	说明
dns	number	12	DNS 连接耗时
tcp	number	29	TCP 连接耗时
ssl	number	19	SSL 连接耗时
ttfb	number	280	网络请求耗时。等待接收响应的第一个字节所花费的时间
trans	number	2	数据传输耗时
dom	number	681	DOM 解析耗时
res	number	1	资源加载耗时。DOM 解析完成后到页面完全加载完所用的时间
firstbyte	number	309	First Byte 时间
fpt	number	634	首次渲染时间。白屏时间
tti	number	677	首次可操作时间
ready	number	837	HTML 加载完成时间，即 DOM Ready 时间
load	number	838	从开始加载到完全加载时间
lcp	number	1418	最大内容绘制时间

上报数据示例：

{
  "pid": "cloud***",
  "type": "perf",
  "env": "production",
  "report_time": 1711949380452,
  "date": 1711949380452,
  "ct": "4g",
  "http_referer": "https://www.example.com/",
  "http_user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36",
  "browser": "Chrome",
  "browser_version": "123.0.0.0",
  "device": "",
  "engine": "Blink",
  "engine_version": "123.0.0.0",
  "os": "Windows",
  "os_version": "10",
  "device_type": "pc",
  "ip_isp": "内网IP",
  "ip_country": "",
  "ip_country_id": "",
  "ip_region": "",
  "ip_region_id": "",
  "ip_city": "",
  "ip_city_id": "",
  "remote_addr": "172.**.**.155",
  "sid": "",
  "pv_id": "2d3ff32a-1b3d-4b39-b370-7af536957ddc",
  "page": "/data/trade",
  "src": "https://www.example.com/data/trade",
  "sr": "1366*768",
  "vp": "1366*641",
  "uid": "dd3bb22c-e220-4c60-9ac8-cb472fe14004",
  "login_id": "***",
  "dns": 0,
  "tcp": 29,
  "ssl": 19.299999952316284,
  "ttfb": 279.7999999523163,
  "trans": 1.5,
  "dom": 680.7999999523163,
  "res": 0.6000001430511475,
  "firstbyte": 309,
  "fpt": 633.5999999046326,
  "tti": 676.8999998569489,
  "ready": 837.0999999046326,
  "load": 837.7000000476837,
  "lcp": 1418
}

Api 日志

字段	类型	示例	说明
api	string	www.example.com/v1/brand	API 地址时
success	number	1	请求是否成功 1：成功 0：失败时
status	number	200	请求返回状态码时
msg	string		返回的消息体
time	number	2	API 耗时，单位为 ms

上报数据示例：

{
  "pid": "cloudshop***",
  "type": "api",
  "env": "production",
  "report_time": 1711953796402,
  "date": 1711953796402,
  "ct": "4g",
  "http_referer": "https://www.example.com/",
  "http_user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0",
  "browser": "Edge",
  "browser_version": "122.0.0.0",
  "device": "",
  "engine": "Blink",
  "engine_version": "122.0.0.0",
  "os": "Windows",
  "os_version": "10",
  "device_type": "pc",
  "ip_isp": "联通",
  "ip_country": "中国",
  "ip_country_id": "CN",
  "ip_region": "上海",
  "ip_region_id": "31",
  "ip_region_name": "上海市",
  "ip_city": "上海",
  "ip_city_id": "3101",
  "ip_city_name": "上海市",
  "remote_addr": "210.**.**.10",
  "sid": "",
  "pv_id": "672fd630-1793-4516-8267-a8f647e5fd58",
  "page": "/task/tasklist/taskadd",
  "src": "https://www.example.com/task/tasklist/taskadd",
  "sr": "1536*864",
  "vp": "1488*726",
  "uid": "029b6c1b-f83f-475f-987c-78f43c96dfd1",
  "login_id": "183***",
  "api": "https://www.example.com/tiger/buttonLimit",
  "success": 1,
  "status": 200,
  "msg": "",
  "time": 230
}