Merge pull request #171 from WISDEM/release-2.3.5

Release 2.3.5

All automated and manual testing has passed on this PR. Integrating into `master` branch.

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,30 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+## Description
+_Describe the bug here_
+
+### Steps to reproduce issue
+_Please provide a minimum working example (MWE) if possible_
+
+1. …
+2. …
+3. …
+
+### Current behavior
+…
+
+### Expected behavior
+…
+
+
+### Code versions
+_List versions only if relevant_
+- Python
+- …

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,14 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+# Description of feature
+Describe the feature here and provide some context. Under what scenario would this be useful?
+
+# Potential solution
+Can you think of ways to implement this?

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,28 @@
+Delete the text explanations below these headers and replace them with information about your PR.
+Please first consult the [developer guide](https://weis.readthedocs.io/en/latest/how_to_contribute_code.html) to make sure your PR follows all code, testing, and documentation conventions.
+
+## Purpose
+Explain the goal of this pull request. If it addresses an existing issue be sure to link to it. Describe the big picture of your changes here, perhaps using a bullet list if multiple changes are done to accomplish a single goal. If it accomplishes multiple goals, it may be best to create separate PR's for each. 
+
+## Type of change
+What types of change is it?
+_Select the appropriate type(s) that describe this PR_
+
+- [ ] Bugfix (non-breaking change which fixes an issue)
+- [ ] New feature (non-breaking change which adds functionality)
+- [ ] Breaking change (non-backwards-compatible fix or feature)
+- [ ] Code style update (formatting, renaming)
+- [ ] Refactoring (no functional changes, no API changes)
+- [ ] Documentation update
+- [ ] Maintenance update
+- [ ] Other (please describe)
+
+## Testing
+Explain the steps needed to test the new code to verify that it does indeed address the issue and produce the expected behavior.
+
+## Checklist
+_Put an `x` in the boxes that apply._
+
+- [ ] I have run existing tests which pass locally with my changes
+- [ ] I have added new tests or examples that prove my fix is effective or that my feature works
+- [ ] I have added necessary documentation

.github/workflows/CI_LandBOSSE.yml

@@ -0,0 +1,83 @@
+name: CI_LandBOSSE
+
+# We run CI on push commits and pull requests on all branches
+on: [push, pull_request]
+
+# A workflow run is made up of one or more jobs that can run sequentially or in parallel
+jobs:
+    build_pip:
+        name: Pip Build (${{ matrix.os }})
+        runs-on: ${{ matrix.os }}
+        strategy:
+            fail-fast: False
+            matrix:
+                os: ["ubuntu-latest", "windows-latest"]
+                python-version: [3.6, 3.7, 3.8]
+
+        steps:
+            - uses: actions/checkout@v2
+            - name: Set up Python ${{ matrix.python-version }}
+              uses: actions/setup-python@v2
+              with:
+                  python-version: ${{ matrix.python-version }}
+
+            - name: Pip Install Dependencies
+              shell: pwsh
+              run: |
+                  python -m pip install --upgrade pip install pytest pandas numpy scipy xlsxwriter openpyxl openmdao
+
+            - name: Pip Install LandBOSSE
+              shell: pwsh
+              run: |
+                  pip install -e .
+
+            # Validate
+            - name: Pip Validation
+              shell: pwsh
+              run: |
+                  python main.py --input project_input_template --output project_input_template --validate
+
+            # Run tests
+            - name: Pip Run pytest
+              shell: pwsh
+              run: |
+                  pytest landbosse/tests
+
+
+    build_conda:
+        name: Conda Build (${{ matrix.os }})
+        runs-on: ${{ matrix.os }}
+        strategy:
+            fail-fast: False
+            matrix:
+                os: ["ubuntu-latest", "windows-latest"]
+                python-version: [3.8]
+
+        steps:
+            - uses: actions/checkout@v2
+            - uses: conda-incubator/setup-miniconda@v2
+              # https://github.com/marketplace/actions/setup-miniconda
+              with:
+                  miniconda-version: "latest"
+                  auto-update-conda: true
+                  python-version: 3.8
+                  environment-file: environment.yml
+
+            # Install 
+            - name: Conda Install LandBOSSE
+              shell: pwsh
+              run: |
+                  python setup.py develop
+
+            # Validate
+            - name: Conda Validation
+              shell: pwsh
+              run: |
+                  python main.py --input project_input_template --output project_input_template --validate
+
+            # Run tests
+            - name: Conda Run pytest
+              shell: pwsh
+              run: |
+                  pytest landbosse/tests
+

.gitignore

@@ -131,6 +131,9 @@ dmypy.json
 # VSCode
 .vscode/
 
+# Emacs
+*~
+
 # Ignore Pandas _libs files
 pandas/_libs/
 

environment.yml

@@ -0,0 +1,18 @@
+name: test
+
+channels:
+  - conda-forge
+  - defaults
+
+dependencies:
+  - python
+  - setuptools
+  - et-xmlfile
+  - pytest
+  - openpyxl
+  - xlsxwriter
+  - pandas
+  - numpy
+  - scipy
+  - openmdao
+  - pip

landbosse/excelio/XlsxDataframeCache.py

@@ -72,8 +72,10 @@ def read_all_sheets_from_xlsx(cls, xlsx_basename, xlsx_path=None):
         else:
             xlsx_filename = os.path.join(xlsx_path, f'{xlsx_basename}.xlsx')
 
-        xlsx = pd.ExcelFile(xlsx_filename)
+        xlsx = pd.ExcelFile(xlsx_filename, engine='openpyxl')
         sheets_dict = {sheet_name: xlsx.parse(sheet_name) for sheet_name in xlsx.sheet_names}
+        for sheet_name in xlsx.sheet_names:
+            sheets_dict[sheet_name].dropna(inplace=True, how='all')
         cls._cache[xlsx_basename] = sheets_dict
         return cls.copy_dataframes(sheets_dict)
 

landbosse/excelio/XlsxValidator.py

@@ -36,7 +36,8 @@ def compare_expected_to_actual(self, expected_xlsx, actual_module_type_operation
         # the raw_cost and raw_cost_total_or_per_turbine columns.
         actual_df = pd.DataFrame(actual_module_type_operation_list)
         actual_df.drop(['raw_cost', 'raw_cost_total_or_per_turbine'], axis=1, inplace=True)
-        expected_df = pd.read_excel(expected_xlsx, 'costs_by_module_type_operation')
+        expected_df = pd.read_excel(expected_xlsx, 'costs_by_module_type_operation', engine='openpyxl')
+        #expected_df = expected_df.dropna(inplace=True, how='all')
         expected_df.rename(columns={
             'Project ID with serial': 'project_id_with_serial',
             'Number of turbines': 'num_turbines',

landbosse/landbosse_omdao/CsvGenerator.py

@@ -0,0 +1,113 @@
+import pandas as pd
+
+
+class CsvGenerator:
+    """
+    This class generates CSV files.
+    """
+
+    def __init__(self, file_ops):
+        """
+        Parameters
+        ----------
+        file_ops : XlsxFileOperations
+            An instance of XlsxFileOperations to manage file names.
+        """
+        self.file_ops = file_ops
+
+    def create_details_dataframe(self, details):
+        """
+        This writes the details .csv.
+
+        Parameters
+        ----------
+        details : list[dict]
+            A list of dictionaries to be converted into a Pandas dataframe
+
+        Returns
+        -------
+        pd.DataFrame
+            The dataframe that can be written to a .csv file.
+        """
+
+        # This the list of details to write to the .csv
+        details_to_write_to_csv = []
+        for row in details:
+            new_row = {}
+            new_row["Project ID with serial"] = row["project_id_with_serial"]
+            new_row["Module"] = row["module"]
+            new_row["Variable name"] = row["variable_df_key_col_name"]
+            new_row["Unit"] = row["unit"]
+
+            value = row["value"]
+            value_is_number = self._is_numeric(value)
+            if value_is_number:
+                new_row["Numeric value"] = value
+            else:
+                new_row["Non-numeric value"] = value
+
+            # If there is a last_number, which means this is a dataframe row that has a number
+            # at the end, write this into the numeric value column. This overrides automatic
+            # type detection.
+
+            if "last_number" in row:
+                new_row["Numeric value"] = row["last_number"]
+
+            details_to_write_to_csv.append(new_row)
+
+        details = pd.DataFrame(details_to_write_to_csv)
+
+        return details
+
+    def create_costs_dataframe(self, costs):
+        """
+        Parameters
+        ----------
+        costs : list[dict]
+            The list of dictionaries of costs.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe to be written as a .csv
+        """
+        new_rows = []
+        for row in costs:
+            new_row = {
+                "Project ID with serial": row["project_id_with_serial"],
+                "Number of turbines": row["num_turbines"],
+                "Turbine rating MW": row["turbine_rating_MW"],
+                "Rotor diameter m": row["rotor_diameter_m"],
+                "Module": row["module"],
+                "Type of cost": row["type_of_cost"],
+                "Cost per turbine": row["cost_per_turbine"],
+                "Cost per project": row["cost_per_project"],
+                "Cost per kW": row["usd_per_kw_per_project"],
+            }
+            new_rows.append(new_row)
+        costs_df = pd.DataFrame(new_rows)
+        return costs_df
+
+    def _is_numeric(self, value):
+        """
+        This method tests if a value is a numeric (that is, can be parsed
+        by float()) or non numeric (which cannot be parsed).
+
+        The decision from this method determines whether values go into
+        the numeric or non-numeric columns.
+
+        Parameters
+        ----------
+        value
+            The value to be tested.
+
+        Returns
+        -------
+        bool
+            True if the value is numeric, False otherwise.
+        """
+        try:
+            float(value)
+        except ValueError:
+            return False
+        return True