Skip to content

Commit

Permalink
* Updated docs and bumped to version 1.0.1
Browse files Browse the repository at this point in the history
  • Loading branch information
MCMcCallum committed Dec 3, 2022
1 parent feda1b5 commit 1ecf18c
Show file tree
Hide file tree
Showing 10 changed files with 28 additions and 32 deletions.
1 change: 0 additions & 1 deletion docs/api/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ Docs for scooch functions and classes are found here.
configurable
config
param
configurable_param
config_list
config_collection
configurable_factory
Expand Down
6 changes: 0 additions & 6 deletions docs/api/configurable_param.rst

This file was deleted.

9 changes: 4 additions & 5 deletions docs/benefits_of_scooch.rst
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,6 @@ Often in ML pipelines you will want to reuse functionality (and hence configurat
from features import FeatureTransform
from batcher import Batcher
from scooch import Configurable
from scooch import ConfigurableParam
class Task(Configurable):
"""
Expand All @@ -70,17 +69,17 @@ Often in ML pipelines you will want to reuse functionality (and hence configurat
Batches data.
"""
_feature = ConfigurableParam(FeatureTransform, doc="Transforms features for neural net input")
_batcher = ConfigurableParam(Batcher, doc="Batches feature Data")
_feature = Param(FeatureTransform, doc="Transforms features for neural net input")
_batcher = Param(Batcher, doc="Batches feature Data")
...
class InferenceTask(Task):
"""
Applies inference.
"""
_feature = ConfigurableParam(FeatureTransform, doc="Transforms features for neural net input")
_model = ConfigurableParam(Model, doc="A trained model for analyzing features")
_feature = Param(FeatureTransform, doc="Transforms features for neural net input")
_model = Param(Model, doc="A trained model for analyzing features")
...
Both classes reuse the same :code:`FeatureTransform` interface, and in the Scooch :code:`config.yaml` file, we can reuse the same configuration:
Expand Down
4 changes: 2 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,11 @@
# -- Project information -----------------------------------------------------

project = 'Scooch'
copyright = '2021 Pandora Media, LLC.'
copyright = '2023 Pandora Media, LLC.'
author = 'Matt C. McCallum'

# The full version, including alpha/beta/rc tags
release = '0.0.9'
release = '1.0.1'


# -- General configuration ---------------------------------------------------
Expand Down
8 changes: 4 additions & 4 deletions docs/getting_started/existing_codebase.rst
Original file line number Diff line number Diff line change
Expand Up @@ -87,19 +87,19 @@ For example, to add all keras loss functions to our Scooch hierarchy we could do
pass
clsmembers = inspect.getmembers(sys.modules[tf.keras.losses.__name__], inspect.isclass)
configurable_tf_losses = [configurize(mem[0], base_class) for mem in clsmembers if mem[0] != 'Loss']
configurable_tf_losses = [configurize(mem[0], Loss) for mem in clsmembers if mem[0] != 'Loss']
With the above code, classes can now be defined with a :code:`ConfigurableParam` of type :code:`Loss`, which will now be able to use all keras loss functions in your :code:`config.yaml` file:
With the above code, classes can now be defined with a :code:`Param` of type :code:`Loss`, which will now be able to use all keras loss functions in your :code:`config.yaml` file:

.. code-block:: python
...
from scooch import ConfigurableParam
from scooch import Param
class Experiment(Configurable):
ConfigurableParam(Loss, doc="A Loss function to train a model with.")
Param(Loss, doc="A Loss function to train a model with.")
...
Configurizing Code that is not Object Oriented
Expand Down
10 changes: 5 additions & 5 deletions docs/getting_started/new_codebase.rst
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ For data augmentations we'll want to parameterize that augmentation itself. Let'
noise_data = np.random.normal(scale=power_linear, size=sample.shape)
return sample + noise_data
We can now employ this new :code:`Configurable` inside the :code:`Batcher` class by adding a :code:`ConfigurableParam` in the class definition of :code:`Batcher`, e.g.,
We can now employ this new :code:`Configurable` inside the :code:`Batcher` class by adding a :code:`Param` with a type of another :code:`Configurable` class, (i.e., the :code:`NoiseAugmenter` class), in the class definition of :code:`Batcher`, e.g.,

.. code-block:: python
Expand All @@ -192,7 +192,7 @@ We can now employ this new :code:`Configurable` inside the :code:`Batcher` class
_batch_size = Param(int, default=128, doc="The number of samples in each mini-batch")
_audio_samples_per_smaple = Param(int, default=1024, doc="The number of audio samples to extract each feature from")
_augmenter = ConfigurableParam(NoiseAugmenter, doc="An augmentation transformation to be applied to each sample")
_augmenter = Param(NoiseAugmenter, doc="An augmentation transformation to be applied to each sample")
...
Expand Down Expand Up @@ -270,7 +270,7 @@ Scooch configures not only classes, but class hierarchies. As this codebase deve
# Produce a random dB value for the noise
...
We now adjust the :code:`ConfigurableParam` in the :code:`Batcher` class to refer to any class that derives from :code:`Augmenter`:
We now adjust the :code:`Configurable` \ :code:`Param` in the :code:`Batcher` class to refer to any class that derives from :code:`Augmenter`:

.. code-block:: python
Expand All @@ -287,7 +287,7 @@ We now adjust the :code:`ConfigurableParam` in the :code:`Batcher` class to refe
_batch_size = Param(int, default=128, doc="The number of samples in each mini-batch")
_audio_samples_per_smaple = Param(int, default=1024, doc="The number of audio samples to extract each feature from")
_augmenter = ConfigurableParam(Augmenter, doc="An augmentation transformation to be applied to each sample")
_augmenter = Param(Augmenter, doc="An augmentation transformation to be applied to each sample")
...
Expand Down Expand Up @@ -360,4 +360,4 @@ The structure of configuration for a :code:`Configurable` can become quite compl
scooch construct -c ./default_config.yaml -m batcher -f Batcher
This will prompt for the type of each :code:`ConfigurableParam` in the class hierarchy, and construct a configuration for the :code:`Batcher` class in the :code:`batcher` module and place it in the file :code:`./default_config.yaml`.
This will prompt for the type of each :code:`Param` that is of type :code:`Configurable` in the class hierarchy, and construct a configuration for the :code:`Batcher` class in the :code:`batcher` module and place it in the file :code:`./default_config.yaml`.
13 changes: 9 additions & 4 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,6 @@ Each class in this configuration corresponds directly to a Scooch :code:`Configu
from scooch import Configurable
from scooch import Param
from scooch import ConfigurableParam
from scooch import ConfigList
class SpectrogramFeature(Configurable):
Expand Down Expand Up @@ -109,12 +108,12 @@ Each class in this configuration corresponds directly to a Scooch :code:`Configu
class Batcher(Configurable):
_batch_size = Param(int, default=256, doc="Number of samples per batch")
_feature = ConfigurableParam(SpectrogramFeature, doc="The feature to produce samples of")
_augmenters = ConfigurableParam(ConfigList(Augmenter), doc="A list of data augmenters to sample from")
_feature = Param(SpectrogramFeature, doc="The feature to produce samples of")
_augmenters = Param(ConfigList(Augmenter), doc="A list of data augmenters to sample from")
...
In the above snippet, we can see abstraction, polymorphism, inheritance, and encapsulation employed within the classes, and their Scooch parameters. Once configured, within each of the classes above the :code:`Param`\ s and :code:`ConfigurableParam`\ s will become accessible as attributes of the encapsulating :code:`Configurable` class instance. Furthermore, the Scooch :code:`Param` / :code:`ConfigurableParam` documentation will be added to the :code:`Configurable` class's doc string for accessibilty in any auto-generated documentation.
In the above snippet, we can see abstraction, polymorphism, inheritance, and encapsulation employed within the classes, and their Scooch parameters. Once configured, within each of the classes above the :code:`Param`\ s will become accessible as attributes of the encapsulating :code:`Configurable` class instance. Furthermore, the Scooch :code:`Param` documentation will be added to the :code:`Configurable` class's doc string for accessibilty in any auto-generated documentation.

With the class definitions and the :code:`config.yaml` file provided above, configuring the :code:`Batcher` class and running the code in a script could be as simple as:

Expand All @@ -135,6 +134,12 @@ With the class definitions and the :code:`config.yaml` file provided above, conf
# Use the class to produce a batch with the configured parameters
samples = a_batcher.get_batch()
As your codebases grow having a tool to explore the available functionality of that codebase will become more important. Fortunately, if your codebase is SCOOCH configurable, the SCOOCH CLI can help. For example to view the details of all of the :code:`Augmenter` types in the `batcher` module described above, you can simply type the following on the command line:

.. code-block:: bash
scooch options -m batcher -f Augmenter
Index
==================

Expand Down
3 changes: 1 addition & 2 deletions examples/batcher_example/batcher/batcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@
import random
import numpy as np
from scooch import Configurable
from scooch import ConfigurableParam
from scooch import Param
from .augmenters import Augmenter

Expand All @@ -27,7 +26,7 @@ class Batcher(Configurable):

_batch_size = Param(int, default=128, doc="The number of samples in each mini-batch")
_audio_samples_per_smaple = Param(int, default=1024, doc="The number of audio samples to extract each feature from")
_augmenter = ConfigurableParam(Augmenter, doc="An augmentation transformation to be applied to each sample")
_augmenter = Param(Augmenter, doc="An augmentation transformation to be applied to each sample")

def set_data(self, data):
# Save a reference to a data array, to sample / batch from
Expand Down
2 changes: 1 addition & 1 deletion scooch/configurable_meta.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ def __new__(cls, name, bases, attrs):
if attr_name.isnumeric():
raise ValueError(f"The Configurable class, {cls.__name__}, has a numeric parameter named {attr_name}, which is disallowed")

# Create Param / ConfigurableParam attributes that don't already exist from the scooch Configurable dictionaries
# Create Param attributes that don't already exist from the scooch Configurable dictionaries
for attr_name in attrs['__PARAMS__']:
if '_'+attr_name not in attrs and attr_name not in attrs:
if attr_name in attrs['__CONFIGURABLES__']:
Expand Down
4 changes: 2 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,15 +105,15 @@ def run(self):
'make_reqs': MakeReqsCommand
},
name='scooch',
version='1.0.0',
version='1.0.1',
description='A python module for configuring hierarchical class structures in yaml with defaults',
long_description=long_description,
long_description_content_type="text/markdown",
url="http://www.mattcmccallum.com/scooch/docs",
author="Matt C. McCallum",
author_email="scooch@mattcmccallum.com",
classifiers=[
'Development Status :: 3 - Alpha',
'Development Status :: 5 - Production/Stable',
'Intended Audience :: Developers',
'Intended Audience :: Education',
'Intended Audience :: Science/Research',
Expand Down

0 comments on commit 1ecf18c

Please sign in to comment.