No output in pusher_power, PowerReport

[masoudbaharlou]

Hello,
I want to integrate powerapi, but nothing is being written to the output.

Here are details to my setup:
I can run hwpc sensor without problem:

I: 25-06-16 12:19:52 build: version unknown (rev: unknown)
I: 25-06-16 12:19:52 uname: Linux 5.15.0-140-generic #150-Ubuntu SMP Sat Apr 12 06:00:09 UTC 2025 x86_64
I: 25-06-16 12:19:52 pmu: found ix86arch 'Intel X86 architectural PMU' having 7 events, 7 counters (4 general, 3 fixed)
I: 25-06-16 12:19:52 pmu: found perf 'perf_events generic PMU' having 198 events, 0 counters (0 general, 0 fixed)
I: 25-06-16 12:19:52 pmu: found rapl 'Intel RAPL' having 2 events, 3 counters (0 general, 3 fixed)
I: 25-06-16 12:19:52 pmu: found perf_raw 'perf_events raw PMU' having 1 events, 0 counters (0 general, 0 fixed)
I: 25-06-16 12:19:52 pmu: found bdw_ep 'Intel Broadwell EP' having 75 events, 11 counters (8 general, 3 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_cbo0 'Intel BroadwellX C-Box 0 uncore' having 28 events, 4 counters (4 general, 0 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_cbo1 'Intel BroadwellX C-Box 1 uncore' having 28 events, 4 counters (4 general, 0 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_cbo2 'Intel BroadwellX C-Box 2 uncore' having 28 events, 4 counters (4 general, 0 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_cbo3 'Intel BroadwellX C-Box 3 uncore' having 28 events, 4 counters (4 general, 0 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_cbo4 'Intel BroadwellX C-Box 4 uncore' having 28 events, 4 counters (4 general, 0 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_cbo5 'Intel BroadwellX C-Box 5 uncore' having 28 events, 4 counters (4 general, 0 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_cbo6 'Intel BroadwellX C-Box 6 uncore' having 28 events, 4 counters (4 general, 0 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_cbo7 'Intel BroadwellX C-Box 7 uncore' having 28 events, 4 counters (4 general, 0 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_ha0 'Intel BroadwellX HA 0 uncore' having 47 events, 4 counters (4 general, 0 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_imc0 'Intel BroadwellX IMC0 uncore' having 42 events, 5 counters (4 general, 1 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_imc1 'Intel BroadwellX IMC1 uncore' having 42 events, 5 counters (4 general, 1 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_imc2 'Intel BroadwellX IMC2 uncore' having 42 events, 5 counters (4 general, 1 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_imc3 'Intel BroadwellX IMC3 uncore' having 42 events, 5 counters (4 general, 1 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_imc4 'Intel BroadwellX IMC4 uncore' having 42 events, 5 counters (4 general, 1 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_pcu 'Intel BroadwellX PCU uncore' having 62 events, 4 counters (4 general, 0 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_qpi0 'Intel BroadwellX QPI0 uncore' having 53 events, 4 counters (4 general, 0 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_qpi1 'Intel BroadwellX QPI1 uncore' having 53 events, 4 counters (4 general, 0 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_ubo 'Intel BroadwellX U-Box uncore' having 3 events, 3 counters (2 general, 1 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_r2pcie 'Intel BroadwellX R2PCIe uncore' having 15 events, 4 counters (4 general, 0 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_r3qpi0 'Intel BroadwellX R3QPI0 uncore' having 28 events, 3 counters (3 general, 0 fixed)
I: 25-06-16 12:19:52 pmu: found bdx_unc_r3qpi1 'Intel BroadwellX R3QPI1 uncore' having 28 events, 3 counters (3 general, 0 fixed)
I: 25-06-16 12:19:52 sensor: configuration is valid, starting monitoring...
I: 25-06-16 12:19:52 perf: monitoring actor started

The config file for hwpc:

{
"name": "sensor",
"verbose": true,
"frequency": 500,
"output": {
"type": "mongodb",
"uri": "mongodb://127.0.0.1",
"database": "db_sensor2",
"collection": "report_0"
},
"system": {
"rapl": {
"events": ["RAPL_ENERGY_PKG", "RAPL_ENERGY_DRAM"],
"monitoring_type": "MONITOR_ONE_CPU_PER_SOCKET"
},
"msr": {
"events": ["TSC", "APERF", "MPERF"]
}
},
"container": {
"core": {
"events": [
"CPU_CLK_THREAD_UNHALTED:REF_P",
"CPU_CLK_THREAD_UNHALTED:THREAD_P",
"LLC_MISSES",
"INSTRUCTIONS_RETIRED"
]
}
}
}

The problem is that when I start the formula in smartwatts, I don't see anything written in the output (PowerReport), the bucket is being created in Influxdb but it's empty

This is the config file I'm using:

{
"verbose": true,
"stream": true,
"input": {
"puller": {
"model": "HWPCReport",
"type": "mongodb",
"uri": "mongodb://127.0.0.1",
"db": "db_sensor2",
"collection": "report_0"
}
},
"output": {
"pusher_power": {
"model": "PowerReport",
"type": "influxdb2",
"uri": "http://127.0.0.1:8086",
"db": "power_consumption",
"org": "{myorg}",
"token": "{mytoken}"
}
},
"cpu-base-freq": 1200,
"cpu-error-threshold": 2.0,
"disable-dram-formula": false,
"sensor-reports-frequency": 1000
}

I manually checked mongodb and there is no "system" event, not sure if this is the problem and how to solve it:

Thanks in advance for your help!

[roda82]

Hello,

Thank you for your message. Can you provide us the log related to the SmartWatts Formula as well as your OS and Linux Kernel version ? Are you using last versions of HWPC Sensor and SmartWatts ? You have to use the same report frequency for the Formula and the Sensor. Can you also provide us the complete log for the Sensor ? And can you confirm that both (Sensor and Formula) are running on the same machine ?

Kind regards

[roda82]

And can you tell us the cgroup version used by your machine ? Can you provide us the output of ls -la /sys/fs/cgroup command ?

Thank you.

Regards,

[masoudbaharlou]

Thanks for your answer,
I'm using, Ubuntu 22.04 LTS, 5.15.0-140-generic, and I confirm that both sensor and formula are running on the same machine.
I pulled the latest tag docker images for both of them. HWPC doesn't print the sensor but SmarWatts version based on its log is: SmartWatts version 2.4.0 based on PowerAPI version 2.10.0.

Here attached are logs for Sensor and SmartWatts:
formula.log
hwpc.log

Here is the output related to cgroup:

dr-xr-xr-x 14 root root 0 May 22 10:30 .
drwxr-xr-x 11 root root 0 May 22 10:23 ..
-r--r--r-- 1 root root 0 May 22 10:23 cgroup.controllers
-rw-r--r-- 1 root root 0 May 22 10:26 cgroup.max.depth
-rw-r--r-- 1 root root 0 May 22 10:26 cgroup.max.descendants
-rw-r--r-- 1 root root 0 May 22 10:23 cgroup.procs
-r--r--r-- 1 root root 0 May 22 10:26 cgroup.stat
-rw-r--r-- 1 root root 0 Jun 13 13:43 cgroup.subtree_control
-rw-r--r-- 1 root root 0 May 22 10:26 cgroup.threads
-rw-r--r-- 1 root root 0 May 22 10:26 cpu.pressure
-r--r--r-- 1 root root 0 May 22 10:26 cpu.stat
-r--r--r-- 1 root root 0 May 22 10:26 cpuset.cpus.effective
-r--r--r-- 1 root root 0 May 22 10:26 cpuset.mems.effective
drwxr-xr-x 2 root root 0 May 22 10:26 dev-hugepages.mount
drwxr-xr-x 2 root root 0 May 22 10:26 dev-mqueue.mount
drwxr-xr-x 2 root root 0 May 22 10:23 init.scope
-rw-r--r-- 1 root root 0 May 22 10:26 io.cost.model
-rw-r--r-- 1 root root 0 May 22 10:26 io.cost.qos
-rw-r--r-- 1 root root 0 May 22 10:26 io.pressure
-rw-r--r-- 1 root root 0 May 22 10:26 io.prio.class
-r--r--r-- 1 root root 0 May 22 10:26 io.stat
drwxr-xr-x 4 root root 0 May 22 10:30 kubepods.slice
-r--r--r-- 1 root root 0 May 22 10:26 memory.numa_stat
-rw-r--r-- 1 root root 0 May 22 10:26 memory.pressure
-r--r--r-- 1 root root 0 May 22 10:26 memory.stat
-r--r--r-- 1 root root 0 May 22 10:26 misc.capacity
drwxr-xr-x 2 root root 0 May 22 10:26 proc-fs-nfsd.mount
drwxr-xr-x 2 root root 0 May 22 10:26 proc-sys-fs-binfmt_misc.mount
drwxr-xr-x 2 root root 0 May 22 10:26 sys-fs-fuse-connections.mount
drwxr-xr-x 2 root root 0 May 22 10:26 sys-kernel-config.mount
drwxr-xr-x 2 root root 0 May 22 10:26 sys-kernel-debug.mount
drwxr-xr-x 2 root root 0 May 22 10:26 sys-kernel-tracing.mount
drwxr-xr-x 54 root root 0 Jun 17 11:28 system.slice
drwxr-xr-x 3 root root 0 Jun 13 18:30 user.slice

Best regards

[masoudbaharlou]

Hello again!
This reply is just to bring it up.
And I have another question:
If I want to start over, what is the standard way to implement powerAPI on a K8s cluster?

[roda82]

Hello. We have tried to reproduce your issue without success. We used your configuration file with mongo as input and output and then with mongo as input and influx as output and the Formula always generates PowerReports. Did you try the last release of SmartWatts (i.e., 2.4.1) ?

[masoudbaharlou]

Did you try the last release of SmartWatts (i.e., 2.4.1) ?

Thank you roda82 for your answer, I will try again with new version and let you know.

I have another question:
This is for only one node.
I have 7 nodes and a K8s cluster, what is the best and correct way to implement PowerAPI for this usecase?