Skip to content

Samthesimpsons/Project-Weather-ETL-Application

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Assignment

Chose the Data Engineering Question for my job application - ML Engineer. This repository contains all the code and documentation. My attempt landed me a ML Engineer role which I did not accept.

Deliverables:

  1. Documentation.md:
    • Design Documentation (Includes introduction, system architecture, assumptions);
    • Setup Documentation on steps to set up the system;
    • Maintenance for common problems;
  2. results.csv
    • Example of a daily weather result CSV file that is stored in an AWS S3 bucket.

Prerequisites

For this walkthrough, you should have the following prerequisites:

References:

https://aws.amazon.com/blogs/devops/micro-etl-pipeline-how-to-fetch-process-and-refresh-small-data-using-aws-lambda-and-aws-sam/

License

MIT

Releases

No releases published

Packages

No packages published

Languages