Skip to content

Commit

Permalink
tutorial and creator tutorial updates part 1
Browse files Browse the repository at this point in the history
  • Loading branch information
brettfiedler committed Dec 5, 2023
1 parent b046a2c commit 250c75b
Show file tree
Hide file tree
Showing 6 changed files with 101 additions and 71 deletions.
29 changes: 12 additions & 17 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,36 +11,31 @@

## Interactive Play Meets Multimodal Web Development

Paper Playground is an open-source project for collaboratively creating multimodal web experiences by means of mapping JavaScript code to real pieces of paper and manipulating the code in your **physical space**.
Paper Playground is an open-source project for collaboratively creating multimodal web experiences by means of mapping JavaScript code to real pieces of paper and manipulating the code in your **physical space**. Everything runs on your machine and you can collaborate locally or using a hosting service to collaborate over projects with others in your shared space.

We aim to support a community interested in bringing physical interaction as a means to collaboratively solve problems in codesigning technology.
While Paper Playground can support many uses, our aim to support a community interested in bringing physical interaction as a means to collaboratively solve problems in codesigning technology.

Paper Playground is based on the [Paper Programs](https://paperprograms.org) open-source project and has been extended to incorporate [the library stack](https://github.com/scenerystack) used by [PhET Interactive Simulations](https://phet.colorado.edu) as a robust 2D scene creator and manager. The project focuses on enabling quick prototyping of web projects using JavaScript, with a particular emphasis on easy addition of audio features (like sounds and sonifications), speech description (both TTS engines and screen reader descriptions), and other non-visual features that are often difficult to design and develop alongside visual elements in these projects.
Paper Playground is based on the [Paper Programs](https://paperprograms.org) open-source project and has been extended to incorporate [the library stack](https://github.com/scenerystack) used by [PhET Interactive Simulations](https://phet.colorado.edu) as a robust 2D scene creator and manager. The project focuses on enabling quick prototyping of web projects based in JavaScript. In creating Paper Playground, we are developing with a particular emphasis on easy addition of multimodal display such as audio features (like sounds and sonifications), speech description (both TTS engines and screen reader descriptions), and other non-visual features that are often difficult to design and develop alongside visual elements in complex web projects.

## Overview of Paper Playground Use

The ideas behind Paper Playground are simple, but the possibilities are infinite.

1. Create programs, as many as you would like, to represent your project.
2. Print out the dot-covered papers that belong to those programs.
1. Using code or our low-code interface, Creator, create a project that factors in physical paper movement into the programs.
2. Print out the dot-covered papers assigned to those programs.
3. Put those *paper programs* in front of a webcam.
4. Watch and interact with the output of that code in your browser.
5. Move programs around and trigger the mappings between your papers position in space and your code!
6. Change your program code and keep the creativity going!
4. Interact with and experience the output of your programs in your browser or in the world around you using a projector (Display).
5. Add/remove/slide/rotate/add markers to programs and trigger the mappings between your papers position in space and your code!
6. Change your programs, iterate, and keep the creativity going!

<figure markdown>
![Paper Playground setup detecting paper programs that create a moon lander and control vertical thrust](assets/full-interface.png){ width=900 }
<figcaption>The interface in action.</figcaption>
</figure>

## What can you do with it?

<figure markdown>
![create play display collab](assets/paper-features.png){ width=600 }
<figcaption></figcaption>
</figure>

You'll create JavaScript programs to populate a page (Board) with interactive, multimodal components to envision anything you want!
You'll create JavaScript programs to populate a page (Display) with interactive, multimodal components to envision anything you want!

Want to create a simple game? Maybe an interactive art experience? Or maybe you want to prototype auditory displays (speech and sounds) for a project or idea that you have? It's all possible with Paper Playground.
Want to create a card game? Maybe an interactive art experience? Or maybe you want to prototype auditory displays (speech and sounds) for a project or idea that you have? It's all possible with Paper Playground.

Every program you write is associated with a sequence of colored (black, red, green, and blue) dots that you will put (or print) on the corners of a piece of paper. We'll refer to the linked papers and code as paper programs from here on out!

Expand Down
16 changes: 13 additions & 3 deletions docs/setup/device-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ Calibration also sets the average dot size. This will determine how large dots n

#### Fine-tuning program detection

If your setup is not detecting all of the dots on your papers, you won't be able to calibrate them and your programs will not be detected (or will be detected intermittently).
If your setup is not detecting all of the dots on your papers, you will not be able to calibrate them and your programs will not be detected (or will be detected intermittently).

You can fine-tune the requirements for detecting dots on the paper for your specific setup and lighting conditions.

Expand All @@ -103,10 +103,20 @@ You can fine-tune the requirements for detecting dots on the paper for your spec

`maxArea` - good for filtering out anything that is being mistreated as a marker.

Look under the Detection header in the sidebar of Camera.html and adjust the parameters. Even with adjustments, you may need to improve the lighting of your room or find you need to print papers with larger dots to get reliable dot detection.
Look under the Detection header in the sidebar of Camera.html and adjust the controls. Even with adjustments, you may need to improve the lighting of your room or find you need to print papers with larger dots (or more spaced out dots) to get reliable dot detection.

- **Average Dot Centers (checkbox):** Improves performance by using an average center to determine dot position, rather than raw pixel coordinates. May decrease dot detection accuracy.

- **Pixel Value Threshold:** Changes the number of steps used to check for a difference between a dot and background. Large steps might miss dots without enough contrast, but will increase performance. Small steps will find more dots, but slow down performance significantly.
- **Min Pixel Value:** The minimum saturation value (out of 256) to look for a dot. Use this to gate out noise if image features are being detected as dots.
- **Max Pixel Value:** The maximum saturation value (out of 256) to look for a dot. Use this to gate out noise if image features are being detected as dots.
- **Min Dot Area (pixels):** The minimum area in pixels that an image feature must be to be called a dot. Use this to gate out noise if image features are being detected as dots.
- **Max Dot Area (pixels):** The maximum area in pixels that an image feature must be to be called a dot. Use this to gate out noise if image features are being detected as dots. *This is a good slider to use if image features are being erroneously detected as Markers*.
- **Min Dot Separation (pixels):** The minimum numbers of pixels the algorithm expects to see dots. Lower values will help to detect smaller dots or farther away programs, at the cost of detecting more image features that may not be paper dots.
- **Scale Factor:** Recommended to keep at 1. Higher values lower the effective resolution of your camera feed. If you need higher performance and dot detection is not a problem/you are using a very high resolution camera, this will speed up the algorithm significantly to get more performance.

## Canvas Projector Setup

If you're using the legacy Projector page to send visual elements to the web canvas, rather than the Board, you'll follow these intructions instead:
If you're using the legacy Canvas (projector.html) page to send visual elements to the web canvas, rather than the Display (board.html), you'll follow these intructions instead:

Power on the projector and separate [http://localhost:3000/projector.html](http://localhost:3000/projector.html) to its own window. Move this window to the projector's display and enter fullscreen mode (Ctrl/Cmd+Shift+F). If you encounter issues (e.g., a recently created program not displaying), refreshing the page should help.
3 changes: 2 additions & 1 deletion docs/setup/reqs.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
## Required Hardware for Paper Detection and Control

- **Computer**: Any desktop or laptop computer should work. Performance will increase with better computer specifications (CPU, GPU, RAM, etc.). All major operating systems and browsers should also work, though most testing has been done with Windows/MacOS and Chrome/Firefox.
- **Webcam**: 720p is adequate, but be cautious with cameras having low-quality sensors, as they might not perform well under all lighting conditions.
- **Webcam**: 720p is adequate, but be cautious with cameras having low-quality sensors, as they might not perform well under all lighting conditions. 1080p cameras are ideal for use when the camera cannot be close to the play area or paper dots are very small.
- **Color Printer**: Alternatively, a black and white printer with coloring markers will do. Check [Resources](../use/resources.md) for templates.
- A simple piece of paper and a steady hand can also be effective!
- Try colored stickers too!

## Required Hardware for Projector Use

Expand Down
46 changes: 24 additions & 22 deletions docs/use/creator.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,41 +4,43 @@

We are working on updating our documentation - more details coming soon!

The Creator interface is where you can create working paper programs and visualize the components of your paper programs while writing little to no code!
## What is Creator?

Creator is currently organized around the [Model-View-Component framework](mvc.md).
!!! warning "In Development"

You create **Model components**, pieces of your idea that other components will need to use or change to make your project work, to be controlled by **Controller components** and display the information dynamically as a visual (e.g., text, image, shape), sound, and more using **View components**
The Creator interface is in development! The current feature set likely does not reflect the final vision of the tool. Follow the development in [GitHub](https://github.com/phetsims/paper-land) or in our community channels!

??? example "Example project (!! text in progress !!)"
The primary vision of Creator is to bring the relationships between the components of your project to the forefront. The goal is to let you focus on the most important pieces and how they connect to your chosen interactions and outputs (sounds, speech, images, etc).

Let's say you wanted to make a frog jump around your screen. Let's add some interactivity and make it jump to wherever a piece of paper. Say, a lily pad paper?
Creator is organized around the [Model-View-Component framework](mvc.md).

Let's think about the design at a high level and connect it to the needed components:
We want to know...

1. Where the frog is and be able to change where it is (model component > position > "frogPosition")
2. Controlling the value of `frogPosition` with a "lilypad" paper (controller components > paper > paper movement)
3. Drawing a picture of the frog (view component > image > "frogPicture")
Since we want to update the image position separately from the frog's new position (so the image doesn't instantly travel when you move the paper), we'll need to make components for each part (X and Y) of the image position too.

4. We'll create a model component > Number > imageYPosition and imageXPosition.
Edit the frogPicture view component and use the Custom Function to set the Center X and Center Y to frogXPosition and lastly we could add a component for
5. Animating that frog when the "lilypad" paper moves (controller component > animation)
Simply stated: You create **Model components** (pieces of your idea that other components will need to use or change to make your project work), to be controlled by **Controller components** and display the information dynamically as a visual (e.g., text, image, shape), sound, speech, and more using **View components**

Watch the videos for two simple examples below for creating an "Audio Pendulum" or "Paper Organ".

## Using Templates

## Copying Programs
You can load in templates using the New Template button in the top left of the interface. These will create a paper or set of papers that are pre-populated with components to create the desired effect. Want a shape that will follow the position of your paper? Select the *Movable Shape* template and the required papers/components will appear, ready to be customized!

### Creating Templates

Some templates are already included in Paper Playground. However, if you'd like to use templates for common sets of programs/components

## Copying Projects and Programs



## Adding Images and Sounds

Adding your own images (.jpg, .png, .gif, etc.) and sounds (.wav, .mp3, etc.) can be done right in Creator! Add a View Component,
Adding your own images (.jpg, .png, .gif, etc.) and sounds (.wav, .mp3, etc.) can be done right in Creator! Add a View Component, select Image or Sound, and drag/drop your file or select the upload box to open a file dialog.

Once uploaded, files will live in the paper-land directory under `.../www/media/images/upload` or `.../www/media/sound/upload`. You can also directly add files to this directory for bulk upload.

The files will now appear in the dropdown selection for those components!

## Using AI chat to create Custom Code

-

## Audio Pendulum Walkthrough
<figure>
Expand Down
4 changes: 4 additions & 0 deletions docs/use/resources.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,7 @@ Don't have a color printer, but do have markers/colored pencils? Use this templa
[Blank Dot Template (GitHub)](https://github.com/phetsims/paper-land/blob/main/docs/use/resources/draw-dots-paper-template.pdf)

[Blank Dot Template](./resources/draw-dots-paper-template.pdf)

!!! tip

Dot sequences are tied to the program number/ID. When using Creator, you can change the number of the programs you create. This means that you can re-use papers over and over! Just note the number of the paper you have printed and change the program numbers before sending them to the Playground.
Loading

0 comments on commit 250c75b

Please sign in to comment.