Migrated pricing functionality to v4 #8424
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
VIKTORVAV99
reviewed
Oct 10, 2025
Member
VIKTORVAV99
left a comment
VIKTORVAV99
left a comment
There was a problem hiding this comment.
Some initial feedback. Realtime works good but historical data is currently not working.
Comment on lines
+244
to
+245
| if params: | ||
| url += "?" + "&".join(params) |
Member
There was a problem hiding this comment.
We should return the params as a dict, then we can pass it directly to requests.
Comment on lines
+250
to
+278
| def _derive_interval_from_datetime(target_datetime: datetime) -> str: | ||
| """ | ||
| Derive appropriate interval based on time difference from target datetime. | ||
| Args: | ||
| target_datetime: The target datetime to compare against current time | ||
| Returns: | ||
| Appropriate interval string for the API | ||
| """ | ||
| delta = datetime.now(tz=timezone.utc) - target_datetime.replace(tzinfo=timezone.utc) | ||
| days = delta.days | ||
|
|
||
| if delta <= timedelta(hours=1): | ||
| return "5m" | ||
| elif delta <= timedelta(days=1): | ||
| return "1h" | ||
| elif days <= 7: | ||
| return "1d" | ||
| elif days <= 31: | ||
| return "7d" | ||
| elif days <= 92: | ||
| return "1M" | ||
| elif days <= 184: | ||
| return "3M" | ||
| elif days <= 366: | ||
| return "1y" | ||
| else: | ||
| return "fy" |
Member
There was a problem hiding this comment.
We should not derive the interval, we should always use the lowest granularity available for each kind. Price should be 5 min for example.
Comment on lines
396
to
400
| zone_key=zone_key, | ||
| session=session, | ||
| target_datetime=target_datetime, | ||
| kind="power", | ||
| ) |
Member
There was a problem hiding this comment.
If I run this: poetry run test_parser "AU-WA" --target_datetime="2022-01-01" this fails with:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/viktor/Repos/electricitymaps-contrib/.venv/lib/python3.10/site-packages/click/core.py", line 1161, in __call__
return self.main(*args, **kwargs)
File "/Users/viktor/Repos/electricitymaps-contrib/.venv/lib/python3.10/site-packages/click/core.py", line 1082, in main
rv = self.invoke(ctx)
File "/Users/viktor/Repos/electricitymaps-contrib/.venv/lib/python3.10/site-packages/click/core.py", line 1443, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/viktor/Repos/electricitymaps-contrib/.venv/lib/python3.10/site-packages/click/core.py", line 788, in invoke
return __callback(*args, **kwargs)
File "/Users/viktor/Repos/electricitymaps-contrib/test_parser.py", line 74, in test_parser
res = parser(*args, target_datetime=parsed_target_datetime, logger=logger)
File "/Users/viktor/Repos/electricitymaps-contrib/electricitymap/contrib/parsers/lib/config.py", line 77, in wrapped_f
result = f(*args, **kwargs)
File "/Users/viktor/Repos/electricitymaps-contrib/electricitymap/contrib/parsers/OPENNEM.py", line 395, in fetch_production
datasets = fetch_datasets(
File "/Users/viktor/Repos/electricitymaps-contrib/electricitymap/contrib/parsers/OPENNEM.py", line 180, in fetch_datasets
response.raise_for_status()
File "/Users/viktor/Repos/electricitymaps-contrib/.venv/lib/python3.10/site-packages/requests/models.py", line 1024, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 416 Client Error: Range Not Satisfiable for url: https://api.openelectricity.org.au/v4/data/network/WEM?metrics=power&interval=fy
This is likely because not all kinds are aggregated to all resolutions and because they have different time ranges allowed. 5 min for example is max 7 days.
Comment on lines
+191
to
+192
| date_start: str | None = None, | ||
| date_end: str | None = None, |
Member
There was a problem hiding this comment.
end should be either now or target_datetime and start should be now or target - 3 days.
| derived_interval = _derive_interval_from_datetime(target_datetime) | ||
|
|
||
| # Build query parameters | ||
| params = [] |
Member
There was a problem hiding this comment.
So this should be a dict with a key value pair.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue
Closes #8415
Description
This PR aims to migrate the Open Electricity API used for Australian data to V4. Specifically for pricing.
Preview
Double check
poetry run test_parser "zone_key"pnpx prettier@2 --write .andpoetry run formatin the top level directory to format my changes.