adding Google BigQuery to Insights as data sources and use queries to visualize data

[sambhxv]

Hi,
I was working on adding Google's BigQuery as an Insights data source but have been confused while doing so.
I was expecting the entire flow to be bit modular, but turns out I would have to make a lot more changes than presumed.
It was kind of obvious to make a whole lot of changes given the fact that PostgresDB / MariaDB / Frappe site-db provide username and password based connection. Connection string can also be used to establish the connection in these cases. But when it comes to BigQuery, the connection is established using a service-account based authorization method.

I'm having some issues primarily with the understanding of what are the steps after a connection is established.
Until now, I have created a doctype called custom-data-source which takes a project ID and service-account as parameters and establishes a connection to the database.

I have made the changes to this fork.
I'm looking for a quick review of the progress I've made. There is a lot of clutter to clean up, bear with me🙂

I have a few more questions:

• Am I following the right path?
• What are the steps further?
• How do I display the tables of my BigQuery DB to Insights dashboard from here?
• How do query builders work?
• Are query builders generic to all relational databases? Or is there a need to create new ones for BigQuery?
• Once I have the data from a data source, how are they visualized on the dashboard?
• Is there a complete workbook of how things work behind the scene for reference?