Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Technical debt #131

Merged
merged 5 commits into from
Jun 26, 2023
Merged

Technical debt #131

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f5334fe
Continuous Comparison and Tech Debt PT1
GregoryPetrochenkov-NOAA Jun 22, 2023
d2e0e63
Multi plotting and tech debt part 2
GregoryPetrochenkov-NOAA Jun 24, 2023
da42247
Adjust size of datasets for memory
GregoryPetrochenkov-NOAA Jun 24, 2023
134d978
Reupload to s3
GregoryPetrochenkov-NOAA Jun 24, 2023
93efdde
Decrease size of geotiffs
GregoryPetrochenkov-NOAA Jun 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions CONTRIBUTING.MD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,14 +51,15 @@ automatically with flake8-black but in some cases manual changes will be necessa
in code coverage and increase in memory. Also run local_benchmark_test to make sure there is no significant
cpu time performance loss. To run everything (in root project directory):
`pytest --memray --cov=gval --cov-report term-missing && python ./tests/local_benchmark.py`.
7. If any changes were made to the documentation, make sure to run `python docs/compile_readme.py`. (The README is made
up of other docs located in Markdown. Edit those directly if need be before running the preceding command.)
To build sphinx documentation locally, change to the docs/sphinx folder and run `make clean && make html`.
7. The README is made up of other docs located in /docs/markdown. To make changes to the README edit them directly
then run the following script: `python docs/compile_readme_and_arrange_docs.py`.
8. To build sphinx documentation locally, change to the docs/sphinx folder and run `make clean && make html`.
The html will be created in the _build/html folder. Open index.html in a browser to preview docs.
8. Commit your changes: `git commit -m 'feature message'` This will invoke pre-commit hooks mentioned on step 5
9. Commit your changes: `git commit -m 'feature message'` This will invoke pre-commit hooks mentioned on step 5
that will lint the code. Make sure all of these checks pass, if not make changes and re-commit.
9. Push to the branch: `git push origin <your_username> <your_feature>`
10. Open a pull request (review checklist in PR template before requesting a review)
10. Push to the branch: `git push -u origin`, or if the branch is not pushed up yet:
`git push --set-upstream origin <your branch>`
11. Open a pull request (review checklist in PR template before requesting a review)


## Standards
Expand All @@ -85,7 +86,7 @@ that will lint the code. Make sure all of these checks pass, if not make changes
* [Jupyter](https://pypi.org/project/jupyter/)


## Development Installation
## Docker Use

(In this case, the image name, "gval-image", and container name, "gval-python" can be changed
to whatever name is more suitable. Script, "test.py", does not exist and is an arbitrary placeholder for
Expand Down
0 docs/compile_readme.py → docs/compile_readme_and_arrange_docs.py
View file
Open in desktop
File renamed without changes.
15 changes: 8 additions & 7 deletions docs/sphinx/SPHINX_CONTRIBUTING.MD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,14 +51,15 @@ automatically with flake8-black but in some cases manual changes will be necessa
in code coverage and increase in memory. Also run local_benchmark_test to make sure there is no significant
cpu time performance loss. To run everything (in root project directory):
`pytest --memray --cov=gval --cov-report term-missing && python ./tests/local_benchmark.py`.
7. If any changes were made to the documentation, make sure to run `python docs/compile_readme.py`. (The README is made
up of other docs located in Markdown. Edit those directly if need be before running the preceding command.)
To build sphinx documentation locally, change to the docs/sphinx folder and run `make clean && make html`.
7. The README is made up of other docs located in /docs/markdown. To make changes to the README edit them directly
then run the following script: `python docs/compile_readme_and_arrange_docs.py`.
8. To build sphinx documentation locally, change to the docs/sphinx folder and run `make clean && make html`.
The html will be created in the _build/html folder. Open index.html in a browser to preview docs.
8. Commit your changes: `git commit -m 'feature message'` This will invoke pre-commit hooks mentioned on step 5
9. Commit your changes: `git commit -m 'feature message'` This will invoke pre-commit hooks mentioned on step 5
that will lint the code. Make sure all of these checks pass, if not make changes and re-commit.
9. Push to the branch: `git push origin <your_username> <your_feature>`
10. Open a pull request (review checklist in PR template before requesting a review)
10. Push to the branch: `git push -u origin`, or if the branch is not pushed up yet:
`git push --set-upstream origin <your branch>`
11. Open a pull request (review checklist in PR template before requesting a review)


## Standards
Expand All @@ -85,7 +86,7 @@ that will lint the code. Make sure all of these checks pass, if not make changes
* [Jupyter](https://pypi.org/project/jupyter/)


## Development Installation
## Docker Use

(In this case, the image name, "gval-image", and container name, "gval-python" can be changed
to whatever name is more suitable. Script, "test.py", does not exist and is an arbitrary placeholder for
Expand Down
82 changes: 42 additions & 40 deletions docs/sphinx/SphinxTutorial.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 1,
"id": "a9fa8470",
"metadata": {},
"outputs": [],
Expand All @@ -45,7 +45,7 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 2,
"id": "f91c0b8c",
"metadata": {},
"outputs": [],
Expand All @@ -72,7 +72,7 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 3,
"id": "541857a7",
"metadata": {},
"outputs": [],
Expand Down Expand Up @@ -132,17 +132,17 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 4,
"id": "b1ef13a0",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.collections.QuadMesh at 0x7f3190123eb0>"
"<matplotlib.collections.QuadMesh at 0x7f31b0842a70>"
]
},
"execution_count": 9,
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
},
Expand Down Expand Up @@ -179,7 +179,7 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": 5,
"id": "fdc9df2b",
"metadata": {},
"outputs": [
Expand Down Expand Up @@ -256,7 +256,7 @@
"3 1 2.0 2.0 24.0 2473405.0"
]
},
"execution_count": 10,
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -283,7 +283,7 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": 6,
"id": "16cb3626",
"metadata": {},
"outputs": [
Expand Down Expand Up @@ -386,7 +386,7 @@
"[1 rows x 22 columns]"
]
},
"execution_count": 11,
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
Expand Down Expand Up @@ -433,7 +433,7 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": 7,
"id": "7264ffc9",
"metadata": {},
"outputs": [],
Expand All @@ -452,7 +452,7 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": 8,
"id": "e3917e34",
"metadata": {},
"outputs": [],
Expand Down Expand Up @@ -488,17 +488,17 @@
},
{
"cell_type": "code",
"execution_count": 14,
"execution_count": 9,
"id": "c6e3c35c",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.collections.QuadMesh at 0x7f318dc0c130>"
"<matplotlib.collections.QuadMesh at 0x7f31ae66e380>"
]
},
"execution_count": 14,
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
},
Expand Down Expand Up @@ -534,17 +534,17 @@
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": 10,
"id": "a2310a98",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.collections.QuadMesh at 0x7f318dc84b20>"
"<matplotlib.collections.QuadMesh at 0x7f31ae42eb60>"
]
},
"execution_count": 15,
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
},
Expand Down Expand Up @@ -591,17 +591,17 @@
},
{
"cell_type": "code",
"execution_count": 16,
"execution_count": 11,
"id": "f6567376",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.collections.QuadMesh at 0x7f318c2a9f30>"
"<matplotlib.collections.QuadMesh at 0x7f31ac9ec250>"
]
},
"execution_count": 16,
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
},
Expand Down Expand Up @@ -643,17 +643,17 @@
},
{
"cell_type": "code",
"execution_count": 17,
"execution_count": 12,
"id": "972f07aa",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.collections.QuadMesh at 0x7f318c12ed10>"
"<matplotlib.collections.QuadMesh at 0x7f31aca8f280>"
]
},
"execution_count": 17,
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
},
Expand Down Expand Up @@ -708,7 +708,7 @@
},
{
"cell_type": "code",
"execution_count": 18,
"execution_count": 13,
"id": "18b9c315",
"metadata": {},
"outputs": [
Expand Down Expand Up @@ -767,7 +767,7 @@
"1 1 2.0 2.0 4.0 2624301.0"
]
},
"execution_count": 18,
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
Expand Down Expand Up @@ -799,7 +799,7 @@
},
{
"cell_type": "code",
"execution_count": 19,
"execution_count": 14,
"id": "2ba3fc06",
"metadata": {},
"outputs": [
Expand Down Expand Up @@ -856,16 +856,16 @@
"0 0.213711 "
]
},
"execution_count": 19,
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"metrics = []\n",
"metric_table_select = crosstab_table.gval.compute_metrics(negative_categories= [0, 1],\n",
" positive_categories = [2],\n",
" metrics=['true_positive_rate', 'prevalence'])\n",
"metric_table_select = crosstab_table.gval.compute_categorical_metrics(negative_categories= [0, 1],\n",
" positive_categories = [2],\n",
" metrics=['true_positive_rate', 'prevalence'])\n",
"metric_table_select"
]
},
Expand All @@ -879,7 +879,7 @@
},
{
"cell_type": "code",
"execution_count": 20,
"execution_count": 15,
"id": "67938408",
"metadata": {},
"outputs": [],
Expand All @@ -901,7 +901,7 @@
},
{
"cell_type": "code",
"execution_count": 21,
"execution_count": 16,
"id": "1e8eeb59",
"metadata": {},
"outputs": [],
Expand All @@ -928,19 +928,21 @@
},
{
"cell_type": "code",
"execution_count": 22,
"execution_count": 17,
"id": "6a41eee3",
"metadata": {},
"outputs": [],
"source": [
"metric_table_register = crosstab_table.gval.compute_metrics(negative_categories= None,\n",
" positive_categories = [2],\n",
" metrics=['error_balance', 'arbitrary1', 'arbitrary2'])"
"metric_table_register = crosstab_table.gval.compute_categorical_metrics(negative_categories= None,\n",
" positive_categories = [2],\n",
" metrics=['error_balance', \n",
" 'arbitrary1', \n",
" 'arbitrary2'])"
]
},
{
"cell_type": "code",
"execution_count": 23,
"execution_count": 18,
"id": "6ab884b7",
"metadata": {},
"outputs": [
Expand Down Expand Up @@ -992,7 +994,7 @@
"0 1 639227.0 512277.0 NaN 2473405.0 0.801401"
]
},
"execution_count": 23,
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -1019,7 +1021,7 @@
},
{
"cell_type": "code",
"execution_count": 24,
"execution_count": 19,
"id": "899a1da9",
"metadata": {},
"outputs": [],
Expand Down
7 changes: 7 additions & 0 deletions docs/sphinx/continuous_stat_funcs.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Continuous Statistics Functions
###############################

:doc:`Return to Homepage <../index>`

.. automodule:: gval.statistics.continuous_stat_funcs
:members:
7 changes: 7 additions & 0 deletions docs/sphinx/continuous_statistics.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Continuous Statistics
#####################

:doc:`Return to Homepage <../index>`

.. automodule:: gval.statistics.continuous_statistics
:members:
1 change: 1 addition & 0 deletions docs/sphinx/schemas.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ Schemas

.. automodule:: gval.utils.schemas
:members:
:undoc-members:
2 changes: 2 additions & 0 deletions docs/sphinx/statistics.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,5 @@ Statistics

categorical_stat_funcs
categorical_statistics
continuous_stat_funcs
continuous_statistics
Loading