Skip to content

Collect all weather data from the Gateway #676

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bouwew wants to merge 37 commits into from
Closed

Collect all weather data from the Gateway #676

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
283e62e
Add weather-related measurements
bouwew Dec 19, 2024
b69515e
Create _get_gateway_measurements() function
bouwew Dec 19, 2024
accfc33
Try
bouwew Dec 19, 2024
0eb9837
Try 2
bouwew Dec 19, 2024
8f3479c
Let _object_value() handle string
bouwew Dec 19, 2024
5d97c1d
Debug
bouwew Dec 19, 2024
786b58f
Fix
bouwew Dec 19, 2024
62e945f
Correct some attrs
bouwew Dec 19, 2024
dba6817
Use attr for formatting
bouwew Dec 19, 2024
0bd47f3
Fix humidity
bouwew Dec 19, 2024
8f8318f
Try 3
bouwew Dec 19, 2024
cd47ef7
Debug
bouwew Dec 19, 2024
a419843
Try 4
bouwew Dec 19, 2024
a5c81a3
Another attr-fix
bouwew Dec 19, 2024
ed33552
Try 4
bouwew Dec 19, 2024
056ac48
Simplify
bouwew Dec 20, 2024
e4af18a
Collect as weather-dict
bouwew Dec 20, 2024
b47cd50
Fix
bouwew Dec 20, 2024
6067b22
Update Adam testdata, device-items
bouwew Dec 20, 2024
5bf9c87
Update Anna testdata, device-items
bouwew Dec 20, 2024
2d42d51
Revert attrs -> attr
bouwew Dec 20, 2024
5735dd2
Remove debugging
bouwew Dec 20, 2024
5561f35
Add typing
bouwew Dec 20, 2024
acd0cee
Further typing fixes
bouwew Dec 20, 2024
11f9287
Update manual_fixtures script
bouwew Dec 20, 2024
653e1e7
Save updated fixtures
bouwew Dec 20, 2024
f52bbb5
Improve
bouwew Dec 20, 2024
1d4ffb8
Revert missed attrs_uom change
bouwew Dec 20, 2024
5b14d73
Back to normal test-output
bouwew Dec 20, 2024
6488ec9
Optimize
bouwew Dec 21, 2024
898ccf8
Add comment
bouwew Dec 21, 2024
aa74d4f
Bump to v1.7.0a0 test-version
bouwew Dec 21, 2024
5d0022c
Make solar_irradiance a sensor
bouwew Dec 21, 2024
ec9bbc2
Update testdata jsons
bouwew Dec 21, 2024
d90bc59
Correct typing
bouwew Dec 21, 2024
32922e8
Save updated fixtures
bouwew Dec 21, 2024
9250faf
Bump to a1
bouwew Dec 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions fixtures/adam_heatpump_cooling/all_data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -306,9 +306,16 @@
"select_gateway_mode": "full",
"select_regulation_mode": "cooling",
"sensors": {
"outdoor_temperature": 13.4
"solar_irradiance": 208.2
},
"vendor": "Plugwise",
"weather": {
"humidity": 88,
"outdoor_temperature": 13.4,
"weather_description": "clouded",
"wind_bearing": 228.0,
"wind_speed": 1.79
},
"zigbee_mac_address": "ABCD012345670101"
},
"7fda9f84f01342f8afe9ebbbbff30c0f": {
Expand Down Expand Up @@ -809,7 +816,7 @@
"cooling_present": true,
"gateway_id": "7d97fc3117784cfdafe347bcedcbbbcb",
"heater_id": "0ca13e8176204ca7bf6f09de59f81c83",
"item_count": 497,
"item_count": 502,
"notifications": {},
"reboot": true,
"smile_name": "Adam"
Expand Down
11 changes: 9 additions & 2 deletions fixtures/adam_jip/all_data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,9 +230,16 @@
"select_gateway_mode": "full",
"select_regulation_mode": "heating",
"sensors": {
"outdoor_temperature": 24.9
"solar_irradiance": 449.2
},
"vendor": "Plugwise",
"weather": {
"humidity": 68,
"outdoor_temperature": 24.9,
"weather_description": "overcast",
"wind_bearing": 216.0,
"wind_speed": 2.68
},
"zigbee_mac_address": "ABCD012345670101"
},
"d27aede973b54be484f6842d1b2802ad": {
Expand Down Expand Up @@ -373,7 +380,7 @@
"cooling_present": false,
"gateway_id": "b5c2386c6f6342669e50fe49dd05b188",
"heater_id": "e4684553153b44afbef2200885f379dc",
"item_count": 244,
"item_count": 249,
Copy link
Contributor

@coderabbitai coderabbitai bot Dec 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

The item count of 249 in the JSON file is incorrect and should be updated to 287

The verification reveals that there are actually 287 items in the data structure (excluding the "item_count" field itself). This count represents all the actual data fields (strings, numbers, and booleans) within the JSON structure. The current value of 249 is outdated and needs to be corrected to reflect the actual count.

🔗 Analysis chain

Verify the updated item count.

The item count has increased from 244 to 249, which aligns with the addition of new weather attributes.

Let's verify if the item count matches the actual number of items:

Let me verify the specific structure of the JSON to understand why there's a discrepancy between the stated count (249) and our initial count (288).

Let me check the actual count of items that should be included based on the data structure, excluding metadata fields like "item_count" itself.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Count the actual number of items in the JSON
jq '[.. | select(type == "string" or type == "number" or type == "boolean")] | length' fixtures/adam_jip/all_data.json

Length of output: 124

Script:

#!/bin/bash
# Let's examine the structure of the JSON file to understand what should be counted
jq 'keys' fixtures/adam_jip/all_data.json

# Also get the keys at the second level to understand the data organization
jq '.[] | keys' fixtures/adam_jip/all_data.json | sort | uniq

Length of output: 821

Script:

#!/bin/bash
# Count the number of actual data fields, excluding structural/metadata fields
jq '
  def count_items:
    if type == "object" then
      to_entries | map(
        if .key == "item_count" then 0
        else .value | count_items
        end
      ) | add
    elif type == "array" then
      map(count_items) | add
    elif type == "string" or type == "number" or type == "boolean" then
      1
    else
      0
    end;
count_items
' fixtures/adam_jip/all_data.json

Length of output: 392

"notifications": {},
"reboot": true,
"smile_name": "Adam"
Expand Down
11 changes: 9 additions & 2 deletions fixtures/adam_multiple_devices_per_zone/all_data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -581,17 +581,24 @@
"name": "Adam",
"select_regulation_mode": "heating",
"sensors": {
"outdoor_temperature": 7.81
"solar_irradiance": 157.5
Copy link
Contributor

@coderabbitai coderabbitai bot Dec 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Item count verification failed - discrepancy found

The actual count of items in the JSON is 440, but there is no item_count field present in the JSON structure (returns null). This indicates a potential issue where the item count tracking is missing from the data structure.

  • The JSON contains 440 primitive values (strings, numbers, or booleans)
  • The item_count field is missing from the JSON structure
  • Consider adding the item_count field to maintain data integrity
🔗 Analysis chain

LGTM! Verify the updated item count.

The weather data structure has been properly added to the gateway section, and the sensors field has been updated accordingly. The changes align with the PR objectives to collect weather data.

Run the following script to verify the item count:

Also applies to: 587-593, 601-601

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that the item_count matches the actual number of items in the JSON
# Expected: The item_count should match the number of items in the JSON

# Count all items in the JSON (excluding the item_count itself)
jq '[.. | select(type == "string" or type == "number" or type == "boolean") | tostring] | length' fixtures/adam_multiple_devices_per_zone/all_data.json

Length of output: 157

Script:

#!/bin/bash
# Let's verify the item_count field in the JSON
jq '.item_count' fixtures/adam_multiple_devices_per_zone/all_data.json

Length of output: 75

},
"vendor": "Plugwise",
"weather": {
"humidity": 75,
"outdoor_temperature": 7.81,
"weather_description": "cloudy",
"wind_bearing": 40.0,
"wind_speed": 8.2
},
"zigbee_mac_address": "ABCD012345670101"
}
},
"gateway": {
"cooling_present": false,
"gateway_id": "fe799307f1624099878210aa0b9f1475",
"heater_id": "90986d591dcd426cae3ec3e8111ff730",
"item_count": 369,
"item_count": 374,
"notifications": {
"af82e4ccf9c548528166d38e560662a4": {
"warning": "Node Plug (with MAC address 000D6F000D13CB01, in room 'n.a.') has been unreachable since 23:03 2020-01-18. Please check the connection and restart the device."
Expand Down
11 changes: 9 additions & 2 deletions fixtures/adam_onoff_cooling_fake_firmware/all_data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,16 @@
"select_gateway_mode": "full",
"select_regulation_mode": "cooling",
"sensors": {
"outdoor_temperature": 13.4
"solar_irradiance": 208.2
},
"vendor": "Plugwise",
"weather": {
"humidity": 88,
"outdoor_temperature": 13.4,
"weather_description": "clouded",
"wind_bearing": 228.0,
"wind_speed": 1.79
},
"zigbee_mac_address": "ABCD012345670101"
},
"ca79d23ae0094120b877558734cff85c": {
Expand Down Expand Up @@ -114,7 +121,7 @@
"cooling_present": true,
"gateway_id": "7d97fc3117784cfdafe347bcedcbbbcb",
"heater_id": "0ca13e8176204ca7bf6f09de59f81c83",
"item_count": 64,
"item_count": 69,
"notifications": {},
"reboot": true,
"smile_name": "Adam"
Expand Down
11 changes: 9 additions & 2 deletions fixtures/adam_plus_anna/all_data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,9 +87,16 @@
"name": "Adam",
"select_regulation_mode": "heating",
"sensors": {
"outdoor_temperature": 11.9
"solar_irradiance": 53.8
},
"vendor": "Plugwise",
"weather": {
"humidity": 58,
"outdoor_temperature": 11.9,
"weather_description": "overcast",
"wind_bearing": 220.0,
"wind_speed": 2.6
},
"zigbee_mac_address": "ABCD012345670101"
},
"ee62cad889f94e8ca3d09021f03a660b": {
Expand Down Expand Up @@ -128,7 +135,7 @@
"cooling_present": false,
"gateway_id": "b128b4bbbd1f47e9bf4d756e8fb5ee94",
"heater_id": "2743216f626f43948deec1f7ab3b3d70",
"item_count": 80,
"item_count": 85,
"notifications": {
"6fb89e35caeb4b1cb275184895202d84": {
"error": "There is no OpenTherm communication with the boiler."
Expand Down
11 changes: 9 additions & 2 deletions fixtures/adam_plus_anna_new/all_data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,9 +173,16 @@
"select_gateway_mode": "full",
"select_regulation_mode": "heating",
"sensors": {
"outdoor_temperature": 9.19
"solar_irradiance": 3.0
},
"vendor": "Plugwise",
"weather": {
"humidity": 90,
"outdoor_temperature": 9.19,
"weather_description": "rain",
"wind_bearing": 291.0,
"wind_speed": 6.26
},
"zigbee_mac_address": "000D6F000D5A168D"
},
"e2f4322d57924fa090fbbc48b3a140dc": {
Expand Down Expand Up @@ -288,7 +295,7 @@
"cooling_present": false,
"gateway_id": "da224107914542988a88561b4452b0f6",
"heater_id": "056ee145a816487eaa69243c3280f8bf",
"item_count": 177,
"item_count": 182,
Copy link
Contributor

@coderabbitai coderabbitai bot Dec 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Item count in JSON file needs correction

The declared item_count of 182 in the JSON file appears to be incorrect. The actual count of scalar properties in the JSON structure is 208, and this count remains unchanged from the previous version. This suggests that either:

  • The item_count field is not meant to represent the total number of scalar properties
  • Or there's an error in the count that needs to be corrected
🔗 Analysis chain

Verify the updated item count.

The item count has been increased from 177 to 182, which aligns with the addition of new weather attributes.

Let me try a different approach to count the items correctly.

Let's try one more approach to understand the structure and count the items correctly.

Let's verify the exact item count by checking all the properties in the JSON structure.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that the item count matches the actual number of items
# Count all unique keys in the JSON structure
jq '[.. | objects | keys[]] | unique | length' fixtures/adam_plus_anna_new/all_data.json

Length of output: 93

Script:

#!/bin/bash
# Let's check the actual structure of the JSON file first
jq 'keys' fixtures/adam_plus_anna_new/all_data.json

# Then count the total number of items in the main data array
jq '.data | length' fixtures/adam_plus_anna_new/all_data.json

Length of output: 146

Script:

#!/bin/bash
# Let's examine the actual content structure
jq '.devices | length' fixtures/adam_plus_anna_new/all_data.json

# Also check the gateway structure
jq '.gateway | keys' fixtures/adam_plus_anna_new/all_data.json

# Count all non-null values in the entire JSON structure
jq '[.. | select(. != null)] | length' fixtures/adam_plus_anna_new/all_data.json

Length of output: 339

Script:

#!/bin/bash
# Count all properties in the JSON structure
jq '[paths(scalars)] | length' fixtures/adam_plus_anna_new/all_data.json

# Also let's see the previous version's count for comparison
git show HEAD^:fixtures/adam_plus_anna_new/all_data.json | jq '[paths(scalars)] | length'

Length of output: 176

"notifications": {},
"reboot": true,
"smile_name": "Adam"
Expand Down
11 changes: 9 additions & 2 deletions fixtures/adam_plus_anna_new_regulation_off/all_data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,9 +173,16 @@
"select_gateway_mode": "full",
"select_regulation_mode": "off",
"sensors": {
"outdoor_temperature": 9.19
"solar_irradiance": 3.0
},
"vendor": "Plugwise",
"weather": {
"humidity": 90,
"outdoor_temperature": 9.19,
"weather_description": "rain",
"wind_bearing": 291.0,
"wind_speed": 6.26
},
"zigbee_mac_address": "000D6F000D5A168D"
},
"e2f4322d57924fa090fbbc48b3a140dc": {
Expand Down Expand Up @@ -288,7 +295,7 @@
"cooling_present": false,
"gateway_id": "da224107914542988a88561b4452b0f6",
"heater_id": "056ee145a816487eaa69243c3280f8bf",
"item_count": 177,
"item_count": 182,
"notifications": {},
"reboot": true,
"smile_name": "Adam"
Expand Down
11 changes: 9 additions & 2 deletions fixtures/adam_zone_per_device/all_data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -578,17 +578,24 @@
"name": "Adam",
"select_regulation_mode": "heating",
"sensors": {
"outdoor_temperature": 7.69
"solar_irradiance": 86.2
},
"vendor": "Plugwise",
"weather": {
"humidity": 70,
"outdoor_temperature": 7.69,
"weather_description": "cloudy",
"wind_bearing": 40.0,
"wind_speed": 7.2
},
"zigbee_mac_address": "ABCD012345670101"
}
},
"gateway": {
"cooling_present": false,
"gateway_id": "fe799307f1624099878210aa0b9f1475",
"heater_id": "90986d591dcd426cae3ec3e8111ff730",
"item_count": 369,
"item_count": 374,
"notifications": {
"af82e4ccf9c548528166d38e560662a4": {
"warning": "Node Plug (with MAC address 000D6F000D13CB01, in room 'n.a.') has been unreachable since 23:03 2020-01-18. Please check the connection and restart the device."
Expand Down
13 changes: 10 additions & 3 deletions fixtures/anna_elga_2/all_data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,16 +84,23 @@
"model_id": "smile_thermo",
"name": "Smile Anna",
"sensors": {
"outdoor_temperature": 13.0
"solar_irradiance": 0.0
},
"vendor": "Plugwise"
"vendor": "Plugwise",
"weather": {
"humidity": 42,
"outdoor_temperature": 13.0,
"weather_description": "cloudy",
"wind_bearing": 120.0,
"wind_speed": 3.6
}
}
},
"gateway": {
"cooling_present": true,
"gateway_id": "fb49af122f6e4b0f91267e1cf7666d6f",
"heater_id": "573c152e7d4f4720878222bd75638f5b",
"item_count": 63,
"item_count": 68,
"notifications": {},
"reboot": true,
"smile_name": "Smile Anna"
Expand Down
13 changes: 10 additions & 3 deletions fixtures/anna_elga_2_cooling/all_data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,16 +84,23 @@
"model_id": "smile_thermo",
"name": "Smile Anna",
"sensors": {
"outdoor_temperature": 31.0
"solar_irradiance": 0.0
},
"vendor": "Plugwise"
"vendor": "Plugwise",
"weather": {
"humidity": 42,
"outdoor_temperature": 31.0,
"weather_description": "cloudy",
"wind_bearing": 120.0,
"wind_speed": 3.6
}
}
},
"gateway": {
"cooling_present": true,
"gateway_id": "fb49af122f6e4b0f91267e1cf7666d6f",
"heater_id": "573c152e7d4f4720878222bd75638f5b",
"item_count": 63,
"item_count": 68,
"notifications": {},
"reboot": true,
"smile_name": "Smile Anna"
Expand Down
13 changes: 10 additions & 3 deletions fixtures/anna_elga_2_schedule_off/all_data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,16 +84,23 @@
"model_id": "smile_thermo",
"name": "Smile Anna",
"sensors": {
"outdoor_temperature": 13.0
"solar_irradiance": 0.0
},
"vendor": "Plugwise"
"vendor": "Plugwise",
"weather": {
"humidity": 42,
"outdoor_temperature": 13.0,
"weather_description": "cloudy",
"wind_bearing": 120.0,
"wind_speed": 3.6
}
}
},
"gateway": {
"cooling_present": true,
"gateway_id": "fb49af122f6e4b0f91267e1cf7666d6f",
"heater_id": "573c152e7d4f4720878222bd75638f5b",
"item_count": 63,
"item_count": 68,
"notifications": {},
"reboot": true,
"smile_name": "Smile Anna"
Expand Down
13 changes: 10 additions & 3 deletions fixtures/anna_elga_no_cooling/all_data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,16 @@
"model_id": "smile_thermo",
"name": "Smile Anna",
"sensors": {
"outdoor_temperature": 20.2
"solar_irradiance": 653.6
},
"vendor": "Plugwise"
"vendor": "Plugwise",
"weather": {
"humidity": 42,
"outdoor_temperature": 20.2,
"weather_description": "clear",
"wind_bearing": 50.0,
"wind_speed": 4.1
}
},
"1cbf783bb11e4a7c8a6843dee3a86927": {
"available": true,
Expand Down Expand Up @@ -95,7 +102,7 @@
"cooling_present": false,
"gateway_id": "015ae9ea3f964e668e490fa39da3870b",
"heater_id": "1cbf783bb11e4a7c8a6843dee3a86927",
"item_count": 63,
"item_count": 68,
"notifications": {},
"reboot": true,
"smile_name": "Smile Anna"
Expand Down
13 changes: 10 additions & 3 deletions fixtures/anna_heatpump_cooling/all_data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,16 @@
"model_id": "smile_thermo",
"name": "Smile Anna",
"sensors": {
"outdoor_temperature": 22.0
"solar_irradiance": 207.2
},
"vendor": "Plugwise"
"vendor": "Plugwise",
"weather": {
"humidity": 35,
"outdoor_temperature": 22.0,
"weather_description": "overcast",
"wind_bearing": 0.0,
"wind_speed": 2.1
}
},
"1cbf783bb11e4a7c8a6843dee3a86927": {
"available": true,
Expand Down Expand Up @@ -94,7 +101,7 @@
"cooling_present": true,
"gateway_id": "015ae9ea3f964e668e490fa39da3870b",
"heater_id": "1cbf783bb11e4a7c8a6843dee3a86927",
"item_count": 64,
"item_count": 69,
"notifications": {},
"reboot": true,
"smile_name": "Smile Anna"
Expand Down
13 changes: 10 additions & 3 deletions fixtures/anna_heatpump_cooling_fake_firmware/all_data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,16 @@
"model_id": "smile_thermo",
"name": "Smile Anna",
"sensors": {
"outdoor_temperature": 22.0
"solar_irradiance": 207.2
},
"vendor": "Plugwise"
"vendor": "Plugwise",
"weather": {
"humidity": 35,
"outdoor_temperature": 22.0,
"weather_description": "overcast",
"wind_bearing": 0.0,
"wind_speed": 2.1
}
},
"1cbf783bb11e4a7c8a6843dee3a86927": {
"available": true,
Expand Down Expand Up @@ -94,7 +101,7 @@
"cooling_present": true,
"gateway_id": "015ae9ea3f964e668e490fa39da3870b",
"heater_id": "1cbf783bb11e4a7c8a6843dee3a86927",
"item_count": 64,
"item_count": 69,
"notifications": {},
"reboot": true,
"smile_name": "Smile Anna"
Expand Down
Loading
Loading