Skip to content

Commit

Permalink
Merge pull request #669 from rkingsbury/rmdrone
Browse files Browse the repository at this point in the history
Proposal: remove Drone class
  • Loading branch information
munrojm committed Aug 4, 2022
2 parents d07b41c + 88a47aa commit bce2abb
Show file tree
Hide file tree
Showing 7 changed files with 0 additions and 600 deletions.
12 changes: 0 additions & 12 deletions docs/concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,3 @@ Another challenge in building complex data-transformation codes is keeping track

Maggma solves this by putting the configuration with the pipeline definition in JSON or YAML files. This is done using the `MSONable` pattern, which requires that any Maggma object (the databases and transformation steps) can convert itself to a python dictionary with it's configuration parameters in a process called serialization. These dictionaries can then be converted back to the origianl Maggma object without having to know what class it belonged. `MSONable` does this by injecting in `@class` and `@module` keys that tell it where to find the original python code for that Maggma object.

## Drone
Drone is a standardized class to synchronize local files and data in your database. It breaks down the process in 4 steps:

1. `get_items`
- Given a folder path to a data folder, read all the files, and return a dictionary
that maps each RecordKey -> List of `RecordIdentifier`
2. `should_update_records`
- Given a list of `RecordIdentifier`, it query the database return a list of `RecordIdentifier` that requires update
3. `process_item` (from `Builder`)
- Given a single `RecordIdentifier`, return the data that it refers to and add meta data
4. `update_targets`
- updates the database given a list of data
99 changes: 0 additions & 99 deletions docs/getting_started/simple_drone.md

This file was deleted.

1 change: 0 additions & 1 deletion docs/reference/core_drone.md

This file was deleted.

2 changes: 0 additions & 2 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,11 @@ nav:
- Advanced Builders: getting_started/advanced_builder.md
- Working with MapBuilder: getting_started/map_builder.md
- Working with GroupBuilder: getting_started/group_builder.md
- Writing a Drone: getting_started/simple_drone.md
- Reference:
Core:
Store: reference/core_store.md
Builder: reference/core_builder.md
Validator: reference/core_validator.md
Drone: reference/core_drone.md
Stores: reference/stores.md
Builders: reference/builders.md
- Changelog: CHANGELOG.md
Expand Down
223 changes: 0 additions & 223 deletions src/maggma/core/drone.py

This file was deleted.

Loading

0 comments on commit bce2abb

Please sign in to comment.