Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Add EIA Provider & Weekly Petroleum Status Report #6693

Open
wants to merge 6 commits into
base: develop
Choose a base branch
from

Conversation

deeleeramone
Copy link
Contributor

Note: This is marked as Do Not Merge because the package name - openbb-eia - is currently occupied and may need to change. This PR is being opened for testing and preliminary review and I will remove the label when those details are sorted.

  1. Why?:

The EIA's Weekly Petroleum Status Report is a closely followed event for anyone watching the fossil fuels

  1. What?:

The WPSR is comprised of thirteen (excludes discontinued series) high-level categories with each containing a subset of tables. Data is from the static Excel files published here, and each file represents a single category.

All data from a single category is returned by supplying "all" to the table parameter of the WPSR endpoint.

Tables from the WPSR are returned in a flat format in the same order as presented in the Excel files. The response is suitable for pivot tables and SQL storage.

  1. Impact:

    • ALRU cache (async drop-in replacement to functools.lru_cache) is used for the WPSR files that retains each "category" requested for the duration of the Python session. Setting use_cache as False will invalidate the cache and download the file again.
    • The table parameter has defined choices that represent all possible tables from all categories. A custom validator checks this parameter against the category and the list of tables within that category; it will raise with a message indicating which choices are valid for the given category.
  2. Testing Done:

    • Unit/integration/
  3. Reviewer Notes:

    • This endpoint does not require an API key, but the field is populated for future expansion.
    • [BugFix] Make User Preferences -> Defaults Work With Any Parameter #6687 will allow this command to be run without defining some placeholder value for eia_api_key.
    • The combination of (category="weekly_estimates", table="all") will result in a very large response that will take 20 seconds or so to download, parse, and combine 21 sheets and more than 1 million rows. It's best to avoid "all" for that category.

Screenshot 2024-09-24 at 7 32 38 PM

@deeleeramone deeleeramone added enhancement Enhancement do not merge Label to prevent pull request merge platform OpenBB Platform v4 PRs for v4 labels Sep 25, 2024
@deeleeramone deeleeramone changed the title add eia provider and weekly petroleum status report [Feature] Add EIA Provider & Weekly Petroleum Status Report Sep 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do not merge Label to prevent pull request merge enhancement Enhancement platform OpenBB Platform v4 PRs for v4
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant