-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Add NASA POWER to iotools #2500
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
IoannisSifnaios
wants to merge
22
commits into
pvlib:main
Choose a base branch
from
IoannisSifnaios:add_nasa_power
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+261
−1
Open
Changes from all commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
794cc5d
Create nasa_power.py
IoannisSifnaios ff8d010
Updated API
IoannisSifnaios 9a99395
Update __init__.py
IoannisSifnaios 88c2efa
fixed minor typos
IoannisSifnaios 7222882
Implement Adam's feedback
IoannisSifnaios 773dfab
update metadata
IoannisSifnaios 85cc8de
Add tests
IoannisSifnaios fc3f520
fix lint
IoannisSifnaios 2775387
2nd try
IoannisSifnaios 2f127e4
update whatsnew
IoannisSifnaios 1b1793d
Apply suggestions from code review
IoannisSifnaios 0ae7346
suggestions from code review vol.2
IoannisSifnaios 095f311
Apply suggestions from code review
IoannisSifnaios d726591
lint vol.2
IoannisSifnaios 11eedcc
Apply suggestions from code review
IoannisSifnaios 7241196
Adam's corrections
IoannisSifnaios 3720b85
Update tests according to new code
IoannisSifnaios c508058
come on now linter!
IoannisSifnaios a2c9a44
minor change
IoannisSifnaios 36a337f
Apply suggestions from code review
IoannisSifnaios 9c0d58f
Kevin's suggestions (function)
IoannisSifnaios b8ff9eb
Kevin's suggestions (tests)
IoannisSifnaios File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,153 @@ | ||
"""Functions for reading and retrieving data from NASA POWER.""" | ||
|
||
import pandas as pd | ||
import requests | ||
import numpy as np | ||
|
||
URL = 'https://power.larc.nasa.gov/api/temporal/hourly/point' | ||
|
||
DEFAULT_PARAMETERS = [ | ||
'dni', 'dhi', 'ghi', 'temp_air', 'wind_speed' | ||
] | ||
|
||
VARIABLE_MAP = { | ||
'ALLSKY_SFC_SW_DWN': 'ghi', | ||
'ALLSKY_SFC_SW_DIFF': 'dhi', | ||
'ALLSKY_SFC_SW_DNI': 'dni', | ||
'CLRSKY_SFC_SW_DWN': 'ghi_clear', | ||
'T2M': 'temp_air', | ||
'WS2M': 'wind_speed_2m', | ||
'WS10M': 'wind_speed', | ||
} | ||
|
||
|
||
def get_nasa_power(latitude, longitude, start, end, | ||
parameters=DEFAULT_PARAMETERS, community='re', url=URL, | ||
elevation=None, wind_height=None, wind_surface=None, | ||
map_variables=True): | ||
""" | ||
Retrieve irradiance and weather data from NASA POWER. | ||
|
||
A general description of NASA POWER is given in [1]_ and the API is | ||
described in [2]_. A detailed list of the available parameters can be | ||
found in [3]_. | ||
|
||
Parameters | ||
---------- | ||
latitude: float | ||
In decimal degrees, north is positive (ISO 19115). | ||
longitude: float | ||
In decimal degrees, east is positive (ISO 19115). | ||
start: datetime like | ||
First timestamp of the requested period. | ||
end: datetime like | ||
Last timestamp of the requested period. | ||
parameters: str, list | ||
IoannisSifnaios marked this conversation as resolved.
Show resolved
Hide resolved
|
||
List of parameters. The default parameters are mentioned below; for the | ||
full list see [3]_. Note that the pvlib naming conventions can also be | ||
used. | ||
|
||
* Global Horizontal Irradiance (GHI) [Wm⁻²] | ||
* Diffuse Horizontal Irradiance (DHI) [Wm⁻²] | ||
* Direct Normal Irradiance (DNI) [Wm⁻²] | ||
* Air temperature at 2 m [C] | ||
* Wind speed at 10 m [m/s] | ||
|
||
community: str, default 're' | ||
Can be one of the following depending on which parameters are of | ||
interest. Note that in many cases this choice | ||
might affect the units of the parameter. | ||
|
||
* ``'re'``: renewable energy | ||
* ``'sb'``: sustainable buildings | ||
* ``'ag'``: agroclimatology | ||
|
||
elevation: float, optional | ||
The custom site elevation in meters to produce the corrected | ||
atmospheric pressure adjusted for elevation. | ||
wind_height: float, optional | ||
The custom wind height in meters to produce the wind speed adjusted | ||
for height. Has to be between 10 and 300 m; see [4]_. | ||
wind_surface: str, optional | ||
The definable surface type to adjust the wind speed. For a list of the | ||
surface types see [4]_. If you provide a wind surface alias please | ||
include a site elevation with the request. | ||
map_variables: bool, default True | ||
When true, renames columns of the Dataframe to pvlib variable names | ||
where applicable. See variable :const:`VARIABLE_MAP`. | ||
|
||
Raises | ||
------ | ||
requests.HTTPError | ||
Raises an error when an incorrect request is made. | ||
|
||
Returns | ||
------- | ||
data : pd.DataFrame | ||
Time series data. The index corresponds to the start (left) of the | ||
interval. | ||
meta : dict | ||
Metadata. | ||
|
||
References | ||
---------- | ||
.. [1] `NASA Prediction Of Worldwide Energy Resources (POWER) | ||
<https://power.larc.nasa.gov/>`_ | ||
.. [2] `NASA POWER API | ||
<https://power.larc.nasa.gov/api/pages/>`_ | ||
.. [3] `NASA POWER API parameters | ||
<https://power.larc.nasa.gov/parameters/>`_ | ||
.. [4] `NASA POWER corrected wind speed parameters | ||
<https://power.larc.nasa.gov/docs/methodology/meteorology/wind/>`_ | ||
""" | ||
start = pd.Timestamp(start) | ||
end = pd.Timestamp(end) | ||
|
||
IoannisSifnaios marked this conversation as resolved.
Show resolved
Hide resolved
|
||
# allow the use of pvlib parameter names | ||
parameter_dict = {v: k for k, v in VARIABLE_MAP.items()} | ||
parameters = [parameter_dict.get(p, p) for p in parameters] | ||
|
||
params = { | ||
'latitude': latitude, | ||
'longitude': longitude, | ||
'start': start.strftime('%Y%m%d'), | ||
'end': end.strftime('%Y%m%d'), | ||
'community': community, | ||
'parameters': ','.join(parameters), # make parameters in a string | ||
'format': 'json', | ||
'user': None, | ||
'header': True, | ||
'time-standard': 'utc', | ||
'site-elevation': elevation, | ||
'wind-elevation': wind_height, | ||
'wind-surface': wind_surface, | ||
} | ||
|
||
response = requests.get(url, params=params) | ||
if not response.ok: | ||
# response.raise_for_status() does not give a useful error message | ||
raise requests.HTTPError(response.json()) | ||
|
||
# Parse the data to dataframe | ||
data = response.json() | ||
hourly_data = data['properties']['parameter'] | ||
df = pd.DataFrame(hourly_data) | ||
df.index = pd.to_datetime(df.index, format='%Y%m%d%H').tz_localize('UTC') | ||
IoannisSifnaios marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
# Create metadata dictionary | ||
meta = data['header'] | ||
meta['times'] = data['times'] | ||
meta['parameters'] = data['parameters'] | ||
|
||
meta['longitude'] = data['geometry']['coordinates'][0] | ||
meta['latitude'] = data['geometry']['coordinates'][1] | ||
meta['altitude'] = data['geometry']['coordinates'][2] | ||
|
||
# Replace NaN values | ||
df = df.replace(meta['fill_value'], np.nan) | ||
|
||
# Rename according to pvlib convention | ||
if map_variables: | ||
df = df.rename(columns=VARIABLE_MAP) | ||
|
||
return df, meta |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,92 @@ | ||
import pandas as pd | ||
import pytest | ||
import pvlib | ||
from requests.exceptions import HTTPError | ||
from tests.conftest import RERUNS, RERUNS_DELAY | ||
|
||
|
||
@pytest.fixture | ||
def data_index(): | ||
index = pd.date_range(start='2025-02-02 00:00+00:00', | ||
end='2025-02-02 23:00+00:00', freq='1h') | ||
return index | ||
|
||
|
||
@pytest.fixture | ||
def ghi_series(data_index): | ||
ghi = [ | ||
0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 50.25, 184.2, 281.55, 368.3, 406.48, | ||
386.45, 316.05, 210.1, 109.05, 12.9, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 | ||
] | ||
return pd.Series(data=ghi, index=data_index, name='ghi') | ||
|
||
|
||
@pytest.mark.remote_data | ||
@pytest.mark.flaky(reruns=RERUNS, reruns_delay=RERUNS_DELAY) | ||
def test_get_nasa_power(data_index, ghi_series): | ||
data, meta = pvlib.iotools.get_nasa_power(latitude=44.76, | ||
longitude=7.64, | ||
start=data_index[0], | ||
end=data_index[-1], | ||
parameters=['ALLSKY_SFC_SW_DWN'], | ||
map_variables=False) | ||
# Check that metadata is correct | ||
assert meta['latitude'] == 44.76 | ||
assert meta['longitude'] == 7.64 | ||
assert meta['altitude'] == 705.88 | ||
assert meta['start'] == '20250202' | ||
assert meta['end'] == '20250202' | ||
assert meta['time_standard'] == 'UTC' | ||
assert meta['title'] == 'NASA/POWER Source Native Resolution Hourly Data' | ||
# Assert that the index is parsed correctly | ||
pd.testing.assert_index_equal(data.index, data_index) | ||
# Test one column | ||
pd.testing.assert_series_equal(data['ALLSKY_SFC_SW_DWN'], ghi_series, | ||
check_freq=False, check_names=False) | ||
|
||
|
||
def test_get_nasa_power_pvlib_params_naming(data_index, ghi_series): | ||
data, meta = pvlib.iotools.get_nasa_power(latitude=44.76, | ||
longitude=7.64, | ||
start=data_index[0], | ||
end=data_index[-1], | ||
parameters=['ghi']) | ||
# Assert that the index is parsed correctly | ||
pd.testing.assert_index_equal(data.index, data_index) | ||
# Test one column | ||
pd.testing.assert_series_equal(data['ghi'], ghi_series, | ||
check_freq=False) | ||
|
||
|
||
def test_get_nasa_power_map_variables(data_index): | ||
# Check that variables are mapped by default to pvlib names | ||
data, meta = pvlib.iotools.get_nasa_power(latitude=44.76, | ||
longitude=7.64, | ||
start=data_index[0], | ||
end=data_index[-1]) | ||
mapped_column_names = ['ghi', 'dni', 'dhi', 'temp_air', 'wind_speed'] | ||
for c in mapped_column_names: | ||
assert c in data.columns | ||
assert meta['latitude'] == 44.76 | ||
assert meta['longitude'] == 7.64 | ||
assert meta['altitude'] == 705.88 | ||
|
||
|
||
def test_get_nasa_power_wrong_parameter_name(data_index): | ||
# Test if HTTPError is raised if a wrong parameter name is asked | ||
with pytest.raises(HTTPError, match=r"ALLSKY_SFC_SW_DLN"): | ||
pvlib.iotools.get_nasa_power(latitude=44.76, | ||
longitude=7.64, | ||
start=data_index[0], | ||
end=data_index[-1], | ||
parameters=['ALLSKY_SFC_SW_DLN']) | ||
|
||
|
||
def test_get_nasa_power_duplicate_parameter_name(data_index): | ||
# Test if HTTPError is raised if a duplicate parameter is asked | ||
with pytest.raises(HTTPError, match=r"ALLSKY_SFC_SW_DWN"): | ||
pvlib.iotools.get_nasa_power(latitude=44.76, | ||
longitude=7.64, | ||
start=data_index[0], | ||
end=data_index[-1], | ||
parameters=2*['ALLSKY_SFC_SW_DWN']) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.