Architecture

• Introduction
• Using the final setup
  • Using the included Generic Data Viewer
  • Using the included Comunica Webclient
    • Adding a Solid pod to the list of datasources for a query
• Building and running
  • The localhost setup case
    • Prerequisites for the localhost setup case
    • Environment variables for the localhost setup case
    • Building the localhost setup case
    • Running the localhost setup case
    • Adding pod contents for the additional use cases to the localhost setup case
    • Using the localhost setup case
    • Stopping the localhost setup case
  • The public Docker based setup case
    • Prerequisites for the public Docker based setup case
    • Environment variables for the public Docker based setup case
    • Building the public Docker based setup case
    • Running the public Docker based setup case
    • Adding pod contents for the additional use cases to the public Docker based setup case
    • Using the public Docker based setup case
    • Bonus: local Generic Data Viewer for querying public pods
    • Stopping the public Docker based setup case
  • The private Docker based setup case
    • Prerequisites for the private Docker based setup case
    • Environment variables for the private Docker based setup case
    • Building the private Docker based setup case
    • Running the private Docker based setup case
    • Using the private Docker based setup case
    • Stopping the private Docker based setup case
• Other documentation resources

Introduction

This repository contains the implementation of an Open Circularity Platform as part of the Onto-DESIDE Horizon Europe project.

We demonstrate the Open Circularity Platform through some example use cases:

• example use case within the Construction domain;
• example use case within the Textile domain;
• some additional use cases:
  • extended example use case within the Textile domain, with data using the CEON ontology and with verifiable credentials;
  • evaluation use cases within different domains.

This repository supports different setup cases. Some are only valid during development. The final and publicly available setup of the Open Circularity Platform relies on Docker containers and Docker Compose and illustrates the decentralized nature of the Solid-based data sharing platform.

Within the platform, we have set up:

• multiple data providers each publishing their data behind a secure access layer using Solid pods,
• a Generic Data Viewer providing an easy to use Web UI to execute queries on these Solid pods,
• a Comunica Webclient providing a more technical Web UI to execute similar or user-modified queries on these Solid pods.

During the setup-flow, Solid pods are created and prefilled with example WebIDs and other Resource Description Framework (RDF) formatted data.

During the usage-flow, an end user navigates to one of the Web UIs to query the data.

Using the final setup

The final setup is available online and needs no further action prior to usage.

Using the included Generic Data Viewer

To use this Generic Data Viewer, navigate to https://viewer.onto-deside.ilabt.imec.be/.

The list of predefined queries is shown in expandible groups at the left side.

Some queries may require you to login as one of the actors, described in the use cases. That is because read permissions to resources may be restricted to specific actors, as can be seen in the overview of permissions.
Find actors' email addresses and passwords in the overview of actors' WebIDs, emails and passwords.

To login, click on the icon in the top right.

To execute a query, click on it in the dashboard.

This screenshot shows the result of a query about Texon's components and materials. The result shown was obtained when logged in to identity provider https://css5.onto-deside.ilabt.imec.be as info@texon.com.

Using the included Comunica Webclient

This Comunica Webclient is intended for viewing low level details of the results of some predefined queries or for experimenting with new queries derived from these predefined queries.

To use this Comunica Webclient, navigate to https://webclient.onto-deside.ilabt.imec.be/.

Some predefined queries are available in the Type or pick a query: field.

The above remark about queries vs. logging in applies here too.

The login dialogue is available after clicking the Settings asterisk (top right.)

This screenshot demonstrates querying Lindner Group's products. The result shown was obtained when logged in to identity provider https://css1.onto-deside.ilabt.imec.be as info@lindner-group.com.

Adding a Solid pod to the list of datasources for a query

All predefined queries are configured to work on a predefined list of datasources (Solid pods in our case). Selecting a query preloads the Choose datasources: dialogue.

It is possible to add your own Solid pod to the list, as illustrated here ("any additional Solid pod").

If you don't have a pod yet:

• you can obtain one following the Solid project instructions
• or try out Solid pods with the SolidLab Pod Playground (be warned for this one: "The pods and data on the instance is removed daily and there are no guarantees regarding uptime.").

To add an external Solid pod to the list of datasources in the Comunica Webclient, place the cursor in the field 'Choose datasources'.

Next type the baseURL of an additional Solid pod and confirm with the <Return> key.

The additional datasource will be taken into account when the query is executed, and the datasource will be available as a preset datasource from here on.

Building and running

We split up this section for the different setup cases.

Unless specified otherwise, execute all commands below from a bash shell in the repository root.

The different setup cases depend on common files.

Some common files read environment variables, that need to be set upfront by sourcing the environment variables files.

Some files cannot read environment variables. In these cases derived files are created from common templates.

The localhost setup case

Prerequisites for the localhost setup case

• a bash shell
• Node >= 18 with npm
• yarn classic
• Java version 17, e.g. 17.0.10-tem

Environment variables for the localhost setup case

Execute this command in any terminal window, before executing any other command in the remainder of this section:

source env-localhost

Building the localhost setup case

Before continuing, make sure the result of a previous build isn't running. See Stopping the localhost setup case.

Next, execute:

# install node dependencies
yarn install
# all further build actions
./scripts/setup/finalize-setup.sh

Running the localhost setup case

./scripts/local-run/start-csss.sh

The command above starts the pods on the localhost in the background and waits until they are all ready (listening). This takes some time to complete.

The pods log files can be consulted at ./local-run/*.log.

The pods data can be viewed at ./local-run/data/css*/.

Adding pod contents for the additional use cases to the localhost setup case

This assumes the following additional prerequisites to the localhost setup case:

• Docker Engine was installed.
• Docker host networking is available. See also here.

Execute:

./scripts/stuff-pods/stuff-pods.sh

Using the localhost setup case

Using the localhost setup case is similar to using the final setup, with the following differences:

• The included Generic Data Viewer:

  • needs to be started as follows in a separate terminal window:

    cd ../applied-in-architecture-generic-data-viewer-react-admin
    npm run dev

  • and can be used at the localhost port reported in the output of above command.

• The included Comunica Webclient is not available here.

Stopping the localhost setup case

./scripts/local-run/stop-csss.sh

The public Docker based setup case

Prerequisites for the public Docker based setup case

• a bash shell
• Node >= 18 with npm
• yarn classic
• Java version 17, e.g. 17.0.10-tem
• Docker Engine and Docker Compose
  • Depending on your platform, different installation guides are available from the above links.

Environment variables for the public Docker based setup case

Execute this command in any terminal window, before executing any other command in the remainder of this section:

source env-docker-public

Building the public Docker based setup case

Before continuing, make sure the result of a previous build isn't running. See Stopping the public Docker based setup case.

Next, execute:

# install node dependencies
yarn install
# all further build actions
./scripts/setup/finalize-setup.sh

Running the public Docker based setup case

docker compose --profile backend --profile frontend --profile extra-pod up --wait

The command above starts all services in a Docker environment and and waits until they are all ready (listening). This takes some time to complete.

Optional: if you're interested in what's happening while the previous command executes, you may open a new terminal window and in it, execute:

docker compose --profile backend --profile frontend --profile extra-pod logs -f

Adding pod contents for the additional use cases to the public Docker based setup case

The following command may be executed on the server or on any other computer fulfilling the prerequisites and having access to the public pods, if all above steps where executed before.

Execute:

./scripts/stuff-pods/stuff-pods.sh

Using the public Docker based setup case

Usage was explained already above in using the final setup.

Bonus: local Generic Data Viewer for querying public pods

There is one pleasant side effect of building the public Docker based setup case: the Generic Data Viewer described in Running the localhost setup case is also available here at the localhost. This time however it addresses the public pods! This is very convenient for further query experiments on the data in the public pods... For more info, see The modified Generic Data Viewer.

Stopping the public Docker based setup case

docker compose --profile backend --profile frontend --profile extra-pod down -t 0

The private Docker based setup case

Prerequisites for the private Docker based setup case

• a bash shell
• Node >= 18 with npm
• yarn classic
• Java version 17, e.g. 17.0.10-tem
• Docker Engine and Docker Compose
  • Depending on your platform, different installation guides are available from the above links.
• OpenSSL
  • Installation depends on your platform. On Linux (especially on Ubuntu 20.04 LTS), if it is not yet installed, execute sudo apt install openssl.

Environment variables for the private Docker based setup case

Execute this command in any terminal window, before executing any other command in the remainder of this section:

source env-docker-private

Building the private Docker based setup case

Before continuing, make sure the result of a previous build isn't running. See Stopping the private Docker based setup case.

Next, execute:

# install node dependencies
yarn install
# all further build actions
./scripts/setup/finalize-setup.sh

Running the private Docker based setup case

Same instructions as for Running the public Docker based setup case.

Using the private Docker based setup case

Using the private Docker based setup case is similar to using the final setup, with the following differences:

• Navigate to http://localhost:5800, to reach the integrated Firefox container, which has access to the resources in the Docker network.
• Let the integrated Firefox browser trust our self-made Certificate Authority: follow the instructions in the Setup section of FIREFOX_CONTAINER.md.
• All further navigation is to be done from the integrated Firefox browser, with URLs identical to the those in the final setup.
• The queries for the additional use cases won't work here, because there is no solution to add pod contents for these use cases.

Stopping the private Docker based setup case

Same instructions as for Stopping the public Docker based setup case.

Other documentation resources

• Overview of actors' WebIDs, emails and passwords
• Overview of permissions
• About the different setup cases
• Testing guide
• Community Solid Server (CSS) configuration
• The Firefox container
• Developer documentation