Recreate iron mapping functionality

[dakotaramos]

Recreating iron mapping functionality with geopandas, matplotlib, and contextily

Recreating the iron mapping functionality from previous usa_iron repo. Leverages Geopandas, Matplotlib, and Contextily to plot geospatial point heat maps and simple straight line shipping / transport routes with multiple layers and open source basemaps. Core functionality added to /h2integrate/postprocess/mapping.py, open to changing file and function names.

GeospatialMapConfig config class to handle validation and default values for all parameters needed to format the plots in plot_geospatial_point_heat_map() and plot_straight_line_shipping_routes()

plot_geospatial_point_heat_map() function to create or add layers to a geospatial point heat map. Functionally this plots a point heat map across a range of sites (lat,long) with the color map indicates the value of a metric of interest (ie: AEP, LCOE, LCOI, etc.) with an openstreetmap (or other) basemap.

plot_straight_line_shipping_routes() function to plot straight line shipping routes. Can either be used to create or add layers to an existing map. NOTE: this was developed for proof of concept work for the ITO Iron electrowinning project, this is a very simplified method for plotting straight line routes, I anticipate this will be revamped or replaced entirely with more complex transport / shipping is developed or integrated into H2I in the future.

calculate_geodataframe_total_bounds() helper function to determine the total bounds / extent (range of lat/long) the data covers in a single or set of Geopandas GeoDataFrames.

auto_detect_lat_long_columns() helper function to automatically detect which columns in a Pandas DataFrame represent the latitude and longitude of a given site. Used if the user does not provide the column names as arguments in plot_geospatial_point_heat_map() and plot_straight_line_shipping_routes()

validate_gdfs_are_same_crs() helper function to validate if 2+ geodataframes are using the same coordinate reference system (CRS)

auto_colorbar_limits() helper function to automatically set visually please / even limits for the colorbar legends, this is also used to set the limits for normalizing the colormap and colorbar. Users can also specify the limits manually in the map_preferences dictionary (colorbar_limits key)

/examples/26_iron_map/ contains standard config files with additional csvs (with some dummy data for ore and shipping costs) used in run_iron.py. Lines 15 onward show an example flow of how this would be used to add 3 heatmap layers (LCOI, LCO iron ore, waterway shipping costs in $/tonne-X) and 3 straight line shipping route layers and saves the plot in in the same location as the cases.sql the design of experiment H2I runs produces /examples/26_iron_map/ex_26_out/

docs/user_guide/plotting_geospatial_data_with_geopandas.md simple workflow of how to use this code and an example figure output

Type of Contribution

• Feature Enhancement
  • New Technology Model
• Bug Fix
• Documentation Update
• CI Changes
• Other (please describe):

General PR Checklist

• CHANGELOG.md has been updated to describe the changes made in this PR
• Documentation
  • Docstrings are up-to-date
  • Related docs/ files are up-to-date, or added when necessary
  • Documentation has been rebuilt successfully
  • Examples have been updated (if applicable)
• Tests pass (If not, and this is expected, please elaborate in the tests section)
• Added tests for new functionality or bug fixes
• PR description thoroughly describes the new feature, bug fix, etc.

New Technology Checklist

• Performance Model: Technology performance model has been implemented and follows H2Integrate patterns (if applicable)
• Cost Model: Technology cost model has been implemented (if applicable)
• Tests: Unit tests have been added for the new technology
  • Performance model tests (if applicable)
  • Cost model tests (if applicable)
  • Integration tests with H2Integrate system
• Example: A working example demonstrating the new technology has been created
  • Example has been tested and runs successfully in test_all_examples.py
  • Example is documented with clear explanations in examples/README.md
    • Input file comments
    • Run file comments
• Documentation:
  • Technology documentation page added to docs/technology_models/
  • Technology added to the main technology models list in docs/technology_models/technology_overview.md
• Integration: Technology has been properly integrated into H2Integrate
  • Added to supported_models.py
  • If a new commodity_type is added, update create_financial_model in h2integrate_model.py
  • Follows established naming conventions outlined in docs/developer_guide/coding_guidelines.md

Related issues

#300

Impacted areas of the software

• path/to/file.extension
  • method1: What and why something was changed in one sentence or less.

Additional supporting information

Test results, if applicable

[dakotaramos]

This is in draft status, still need to add a /docs page, add pytest, and double check changes necessary for geopandas and contextily install process (update main readme.md)

Will up the description of the PR tomorrow

[dakotaramos]

Ok, I think this is ready for review. A couple of thoughts for those who review this:

1. I see a bunch of tests are now failing as of 12/22/2025 at 4:51 pm MT (commit 7abddc5) but I don't think that was from anything I did

2. @johnjasa and @RHammond2, I think the way that I have it now resolves all of the dependency issues with geopandas and contextily that I was running into previously. Maybe I'm misunderstood on the intended workflow of the pip install h2integrate installers / users but I first created a conda environment conda create --name h2i_test python=3.11 -y; conda activate h2i_test and then tried both pip install -e [.all] and pip install [.all] (in separate workflows) so that it installs things from the pyproject.toml and had no issues with my code (TBD if there are other issues once the tests start passing again?). I think my original issue occurred when trying to conda install or pip install geopandas and contextily after having already done the conda install and pip install commands in the README.md, meaning I think it manages the dependencies properly if geopandas and contextily are included within the pyproject.toml.

SIDE NOTE: on the pyproject.toml, there are a few duplicates that could be cleaned up, ie: i see geopy in there atleast 3 times.

3. I created tests in examples/test/test_all_examples.py which indirectly is testing the functionality of all of the helper functions in mapping.py. Should I also create explicit tests for each individual helper function (calculate_geodataframe_total_bounds(), auto_detect_lat_long_columns(), validate_gdfs_are_same_crs(), auto_colorbar_limits())? If yes, where / in what file should those test live?

4. The current docs/user_guide/plotting_geospatial_data_with_geopandas.md is pretty minimal / basic. Happy to add / expand if anyone has thoughts on what else should be included there (or if it should live somewhere else in the online docs hierarchy). But I left it as is thinking this could also expand with additional geopandas plotting functionality examples if they are developed in the future.

5. This is specifically pointed at @elenya-grant, but if other have thoughts please let me know. Elenya, I'd particularly like feedback and thoughts on the GeospatialMapConfig() class and how it is used in the subsequent functions. Since you have worked with geopandas and mapping results in the past I'm curious if you would recommend any changes that would make integrating other geopandas plotting functions (that you or others have made) in the future easier.

6. I still planning to create an issue on the SSL self-signed certificate error that I ran into when developing / debugging this functionality. My original solution (on macOS) ended up causing issues with conda's ability to download / install packages. Will link this PR to that issues (or the other way around) next week.

[dakotaramos]

All good, my suggestion to shorten the example test runtime was implemented. I think we should even cut that out of the example script by default, rather than suggesting the user do it by commenting out a line. Since the example is about mapping there's no need to run the model, especially if it only takes a few minutes. See my suggested changes.

Also: running run_iron.py on Win11, python 3.11.13, my NREL machine on VPN, I was getting the following error:

File "C:\Users\jmartin4\AppData\Local\anaconda3\envs\h2i-fork\Lib\site-packages\requests\adapters.py", line 675, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='tile.openstreetmap.org', port=443): Max retries exceeded with url: /6/14/22.png (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1016)')))

BUT I was able to fix it pretty easily via:

pip install pip-system-certs

That's what Google AI told me to do and I can now run run_iron.py just fine. Hopefully just a matter of adding that package to the pyproject.toml but probably want to flag it for @RHammond2 to address in their re-review.

We can add this detail to the issue once created, I'll do that this week. Given the discovery on these SSL errors in general, this is more so an issue at the OS / virtual environment level where not all libraries / packages use the same .pem / SSL certification file because they install their own (ie: requests, certifi, openssl, etc) and is not specific to the use of contextily & API calls to get tile basemaps (in this case from openstreetmap), I would vote the installation of pip-system-certs lives in the base dependencies in the pyproject.toml so these types of errors are auto caught / resolved for other packages that may have similar issues.

[elenya-grant]

I created tests in examples/test/test_all_examples.py which indirectly is testing the functionality of all of the helper functions in mapping.py. Should I also create explicit tests for each individual helper function (calculate_geodataframe_total_bounds(), auto_detect_lat_long_columns(), validate_gdfs_are_same_crs(), auto_colorbar_limits())? If yes, where / in what file should those test live?

@dakotaramos - I think that it'd be great to have tests for these individual functions if its not a big lift. These could live in h2integrate/postprocess/tests/test_mapping_tools.py or something like that.

[elenya-grant]

Looks good Dakota! I left a few notes in mapping.py but they're all just light suggestions/ideas and don't require any changes at this time. Thanks for all the great work on this! This is really great!

[elenya-grant]

I think in the future (not necessary for this PR) we could add a new variable like norm_type (right now it seems to default to a BoundaryNorm, which is likely the most common use-case) and then expand/update the colorbar_limits to handle different inputs for different norm types.

[dakotaramos]

I'd be curious to learn more about other norm types, to my knowledge the normalization that is happening is normalizing the colormap variable (LCOE, LCOI, etc) between [0.0,1.0] such that it can be mapped to the colormap color scheme. But definitely open to the idea if there are applicable use cases for it!

[dakotaramos]

Leaving as is for now and will keep this unresolved as reference for future PRs.

[elenya-grant]

Not-necessary for this PR but I've before made a function that takes a reference marker size and figure size to auto-size the marker size if the fig size changes. A marker size of 36 may look good on the default figure size but may appear way too small on a figure that's 2x the size. This kind of auto-calc could be useful to prevent having to fine-tune that input parameter but not necessary for this PR.

[dakotaramos]

Yea I'm all for more automation in this workflow for stuff like this, if you have already made some similar functionality feel free to add on in a future PR.

[elenya-grant]

Is basemap_provider how we specify the underlying map or is that done with basemap_provider and another variable?

When I've made map-plots in the past, I normally just use Census geodata files to plot the outlines of states (and counties if wanted). In the future, I wonder if we could offer an option to use geodata files that is user-provided or pulled from open-source datasets like the census. This is just a thought - no changes necessary!

[dakotaramos]

basemap_provider is the contextily object that interfaces with the xyzservices.TileProvider API to pull in the underlying basemap .pngs to the extent and zoom / detail level needed based on the extent (lat/long range) of the data. It is the main variable for specifying but there are some things under the hood that contextily does to interface with the geopandas plot to add the basemap.

I do see value in having the ability to use state outlines (ie: census TIGER state.zip files).

Can you share an existing workflow / code that you have that plots points on a state outline map, for my own reference? I like the idea of having that as an option instead of contextily (especially with all the SSL errors around that), but I would have to play around with the workflow a bit to make sure it works smoothly. Right now contextily auto detects the current gdf / plot and adds to that, with the state outline maps that may need to be added as the basemap at the beginning? TBD.

[elenya-grant]

I'm not sure whether this should go in the configuration class or as an input to the function, but its sometimes nice to be able to specify the dpi (I think the default is 100 or 200) if saving a figure:

if Path(save_plot_fpath).suffix != ".pdf":
   fig.savefig(save_plot_fpath,bbox_inches="tight",dpi=save_plot_dpi)
else:
   fig.savefig(save_plot_fpath,bbox_inches="tight")

[dakotaramos]

Yea it would be good to have atleast the format and dpi arguments.

Might be worth making a savefig_preference object in the config class for all the keyword args https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.savefig.html

[RHammond2]

Feel free to copy/take inspiration from some of the work I've done in the OpenOA plotting module that falls into the same category, just don't repeat my unfixed mutable defaults that I've had you fix in this PR. The major difference from what you've proposed is that if a user provides a bad argument to matplotlib, it's on them, not us to identify.

[dakotaramos]

I think for now I'm simply going to add a save_plot_dpi argument to the functions, but would like to revisit this in future PRs. It almost sounds like this could be part of Elenya's previously suggested PlotOptionsBaseConfig or even its own SavefigOptionsBaseConfig class

[RHammond2]

Also: running run_iron.py on Win11, python 3.11.13, my NREL machine on VPN, I was getting the following error:

File "C:\Users\jmartin4\AppData\Local\anaconda3\envs\h2i-fork\Lib\site-packages\requests\adapters.py", line 675, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='tile.openstreetmap.org', port=443): Max retries exceeded with url: /6/14/22.png (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1016)')))

BUT I was able to fix it pretty easily via:

pip install pip-system-certs

That's what Google AI told me to do and I can now run run_iron.py just fine. Hopefully just a matter of adding that package to the pyproject.toml but probably want to flag it for @RHammond2 to address in their re-review.

I would recommend against adding this library to the H2Integrate dependency stack because it's not an H2I problem, but a corporate security layer problem. Some other institutional users may encounter these issues, but much of our errors stem from the fact that we're operating on a VPN with certifications that don't always play nicely.

I believe all of Rob's comments have been addressed, dismissing this review to allow for merging

[dakotaramos]

Also: running run_iron.py on Win11, python 3.11.13, my NREL machine on VPN, I was getting the following error:
File "C:\Users\jmartin4\AppData\Local\anaconda3\envs\h2i-fork\Lib\site-packages\requests\adapters.py", line 675, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='tile.openstreetmap.org', port=443): Max retries exceeded with url: /6/14/22.png (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1016)')))
BUT I was able to fix it pretty easily via:
pip install pip-system-certs
That's what Google AI told me to do and I can now run run_iron.py just fine. Hopefully just a matter of adding that package to the pyproject.toml but probably want to flag it for @RHammond2 to address in their re-review.

I would recommend against adding this library to the H2Integrate dependency stack because it's not an H2I problem, but a corporate security layer problem. Some other institutional users may encounter these issues, but much of our errors stem from the fact that we're operating on a VPN with certifications that don't always play nicely.

Copy that, I'll be sure to include this sentiment in the issue that is created along with all of the solutions we've found for the different operating systems

[dakotaramos]

I created tests in examples/test/test_all_examples.py which indirectly is testing the functionality of all of the helper functions in mapping.py. Should I also create explicit tests for each individual helper function (calculate_geodataframe_total_bounds(), auto_detect_lat_long_columns(), validate_gdfs_are_same_crs(), auto_colorbar_limits())? If yes, where / in what file should those test live?

@dakotaramos - I think that it'd be great to have tests for these individual functions if its not a big lift. These could live in h2integrate/postprocess/tests/test_mapping_tools.py or something like that.

@elenya-grant added in some tests for the helper functions, if you can review and let me know if there is anything else you would add, thanks!