Skip to content

Getting started as a developer and data engineer

Nigel Leck edited this page Apr 29, 2021 · 13 revisions

Contents

Github

All code is stored in either the govau or ausdto organisation. To request access to these organisations, contact support@cloud.gov.au and send them your github username.

Cloud.gov.au

All of our web products are stored on cloud.gov.au, which is built upon cloud foundry.

To get access to cloud.gov.au send the following email to support@cloud.gov.au

G`Day cloud.gov.au team

Could you please create a cloud.gov.au account for me?

email: <your email here>
Mobile number: <your mobile number here>

Could I please have access to the Analytics space.

Regards,

Jane Doe
<your team name here>

You can read more on cloud.gov.au on their website.


AWS

Email support@cloud.gov.au for access to AWS, with the role of datagovau-support. This is used to do maintenance work on data.gov.au/data


Data responsibility at Observatory

• Google Analytics data for whole of Government
• Government organisations data published on data.gov.au
• Geo dataset published on nationalmap.gov.au
• Other varied data sources, data pipelines outputs in Google Cloud Storage

Data store at Observatory

• The Google Analytics data is ingested in cloud data warehouse hosted on Google Cloud Platform BigQuery service.
• Data on data.gov.au is hosted using CKAN
• Geo dataset on nationalmap.gov.au is hosted on MAGDA platform which is maintained by Geoscience Australia.

Github Repository for publishing data scripts

The data scripts developed in SQL and Python are available in following Github repository. 
The scripts are used to build data pipelines.
Galileo repository
Observatory Service – the details of data scripts can be found in documentation guide.