A simple ELT project pulling Airbnb data.
Data was extracted into an S3 Bucket. From here it was transfered into Snowflake. DBT was used to clean up and transform the data. Ensuring that the neccessary tests and packages were used for this.
Dashboard was produced using preset highlighting number of hosts, new listings, distribution of super hosts, price distribution and the effects of the full moon.
Motivation:
Project was based on an interest in Airbnb listing data and user reviews based on varying obstacles.
It also provided a good opportunity to develop skills and experience in a range of tools. As such, project is more complex than required, utilising dbt, cloud based storagen and visualisation tools.
Architecture:
- Extract data using Reddit API
- Load into AWS S3
- Copy into Snowflake
- Transform using dbt
- Create PowerBI or Google Data Studio Dashboard
- Orchestrate with Airflow