I. Feature Description
Cloud Monitor data can be forwarded to a specified Kafka, supporting:
- Full data forwarding, such as forwarding all metric data of CVMs.
- Partial forwarding, such as forwarding metric data of a specific CVM or all CVMs under a specific AppID, and forwarding specified metric data of specified instances.
The above features are configured through forwarding rules.
II. Forwarding Configuration
Due to the lack of a frontend page, forwarding rules are configured via CLI-based operations, that is, by inserting forwarding rules into the database. The steps are as follows:
- Connect to the forwarding database named dispense.
- Configure forwarding rules.
-- Create a rule.
insert into t_rule (RuleName, Status) values ('forwarding rule name', 1);
Description:
RuleName: Rule name.
Status: Rule enabling status, 1: enabled; 0: disabled.
-- Configure the metrics that need to be forwarded.
insert into t_metric (RuleId, Metric, Namespace, Measurement, Periods) values (10, 'cpu_usage', 'qce/cvm', 'cvm_device', '60');
Note:
RuleId: The ID field after a rule is created via the t_rule above.
Metric: Metric name.
Namespace: The Namespace where metrics are located.
Measurement: The corresponding Cloud Monitor view. To obtain the view, see Note 1.
Periods: The time granularity of metrics. You can configure a type such as 60,300 to specify multiple time granularities. You can also configure "*" to support all granularities.
For how to determine the Namespace and Measurement for metrics and the supported time granularity, see Appendix 1.
-- Configure filter conditions (Optional. Configure this if you only want to forward data of a specific instance. There are 10 tables in total. You can obtain the information about which table ${n} resides on by performing a ruleId % 10 calculation.)
insert into t_condition_${n} (RuleId, Filters) values (10, '[{"Key":"vm_uuid","Values":["c7fc0b4c-0059-4318-bb0c-dc4a31a071da"],"Expression":"="},{"Key":"__appid__","Values":["251005942", "1"],"Expression":"in"}]');
Note:
RuleId: The ID field after a rule is created via the t_rule above.
Filters: Forwarding filter rules. It is a JSON array, and the relationship between multiple arrays is AND.
-- Configure the forwarding target Kafka.
insert into t_target (RuleId, Target, ProtoType) values (10, '{"brokers":"11.150.215.105:9092","topic":"Nowrite","user":"","password":""}', 1);
Note:
RuleId: The ID field after a rule is created via the t_rule above.
Target: Target Kafka address.
ProtoType: Message type forwarded to Kafka, 1: JSON; 2: Protobuf.
III. Message Protocol
The message protocol forwarded to Kafka is as follows:
// json
{
"Namespace": "qce/cvm",
"MetricName": "cpu_usage",
"Statistic": "MAX",
"Period": 60,
"Dimensions": [{
"Name": "appid",
"Value": "1"
}, {
"Name": "projectid",
"Value": "0"
}, {
"Name": "vm_uuid",
"Value": "4f7639f1-f156-41b4-9072-f7e27af97d3b"
}],
"Timestamp": 1737107700000,
"Value": 22.916666666666668
}
IV. Forwarding Table Structure Description
| Table Name | Meaning | Description |
| t_rule | Forwarding a rule table | Includes rule metadata such as rule ID, rule name, and configuration user.
`Id: 38230` `RuleName: 1118-Phase 2-cvm-cpu // Forwarding rule name.` `AppId: 1300260700 // User AppID for configuring rules, can be empty.` `SubUin: 0 // User UIN for configuring rules, can be empty.` `Status: 1 // Rule enabling status, 1: enabled; 0: disabled.` `CreateTime: 2024-11-20 11:14:23` `UpdateTime: 2024-11-20 11:14:23` `IsAllMetric: 0 // Can be empty.` |
| t_metric | Forwarding a metric table | Includes the metric information associated with the forwarding operation. Note that in a private network, ExtNamespace and ExtMetric are empty.
` Id: 3732457` ` RuleId: 38230` ` ExtMetric: CpuUtil // External metric name, can be empty.` `ExtNamespace: QCE/TI_MODEL // External namespace, can be empty.` ` Measurement: model_svc // View name.` ` Periods: * // * indicates all granularities. If you only want to forward granularities 60 and 300, configure it as 60,300.` ` Metric: cpu_util // Internal metric name.` ` Namespace: qce/ti_model // Internal namespace.` ` CNMeaning: CPU utilization // Metric Chinese name, which can be empty.` ` ENMeaning: cpu_util // Metric English name, which can be empty.` ` CName: ` ` Unit: % // Metric unit, which can be empty.` ` CreateTime: 2024-12-12 18:30:00` ` UpdateTime: 2024-12-12 18:30:00` |
| t_condition_${n} | Forwarding a condition table | There are 10 tables in total (0 to 9), and you can obtain the information about which table the forwarding condition resides on by performing a ruleId % 10 calculation. Full forwarding can be configured in the private network by setting the condition table to be empty. The Expression in Filters only supports = or in. For =, Values has only one value; for in, Values may have multiple values. `Id: 12` `RuleId: 38230` `Filters: [{"Key":"vm_uuid","Values":["c7fc0b4c-0059-4318-bb0c-dc4a31a071da"],"Expression":"="},{"Key":"__appid__","Values":["251005942", "1"],"Expression":"in"}]` `CreateTime: 2024-11-19 17:49:19` `UpdateTime: 2024-11-19 17:49:19` |
| t_target | Forwarding a target table | TargetType currently only supports Kafka, and ProtoType supports two formats: JSON (1) and PB (2). ` Id: 34539` ` RuleId: 38230` `TargetType: Kafka` ` Target: {"brokers":"11.150.215.105:9092","topic":"Nowrite","user":"","password":""}` ` ProtoType: 1` `CreateTime: 2024-11-19 16:48:31` `UpdateTime: 2024-11-19 16:48:31` |
V. Appendix 1
How to view metrics, Namespace, Measurement, and Period?
Execute kubectl get ingress -A | grep isd.barad on the master node.
Access the domain name through a browser.