diff --git a/docs/index.md b/docs/index.md index 864ac84f..4b433537 100644 --- a/docs/index.md +++ b/docs/index.md @@ -11,36 +11,31 @@ ## Interactive Play Meets Multimodal Web Development -Paper Playground is an open-source project for collaboratively creating multimodal web experiences by means of mapping JavaScript code to real pieces of paper and manipulating the code in your **physical space**. +Paper Playground is an open-source project for collaboratively creating multimodal web experiences by means of mapping JavaScript code to real pieces of paper and manipulating the code in your **physical space**. Everything runs on your machine and you can collaborate locally or using a hosting service to collaborate over projects with others in your shared space. -We aim to support a community interested in bringing physical interaction as a means to collaboratively solve problems in codesigning technology. +While Paper Playground can support many uses, our aim to support a community interested in bringing physical interaction as a means to collaboratively solve problems in codesigning technology. -Paper Playground is based on the [Paper Programs](https://paperprograms.org) open-source project and has been extended to incorporate [the library stack](https://github.com/scenerystack) used by [PhET Interactive Simulations](https://phet.colorado.edu) as a robust 2D scene creator and manager. The project focuses on enabling quick prototyping of web projects using JavaScript, with a particular emphasis on easy addition of audio features (like sounds and sonifications), speech description (both TTS engines and screen reader descriptions), and other non-visual features that are often difficult to design and develop alongside visual elements in these projects. +Paper Playground is based on the [Paper Programs](https://paperprograms.org) open-source project and has been extended to incorporate [the library stack](https://github.com/scenerystack) used by [PhET Interactive Simulations](https://phet.colorado.edu) as a robust 2D scene creator and manager. The project focuses on enabling quick prototyping of web projects based in JavaScript. In creating Paper Playground, we are developing with a particular emphasis on easy addition of multimodal display such as audio features (like sounds and sonifications), speech description (both TTS engines and screen reader descriptions), and other non-visual features that are often difficult to design and develop alongside visual elements in complex web projects. + +## Overview of Paper Playground Use The ideas behind Paper Playground are simple, but the possibilities are infinite. -1. Create programs, as many as you would like, to represent your project. -2. Print out the dot-covered papers that belong to those programs. +1. Using code or our low-code interface, Creator, create a project that factors in physical paper movement into the programs. +2. Print out the dot-covered papers assigned to those programs. 3. Put those *paper programs* in front of a webcam. -4. Watch and interact with the output of that code in your browser. -5. Move programs around and trigger the mappings between your papers position in space and your code! -6. Change your program code and keep the creativity going! +4. Interact with and experience the output of your programs in your browser or in the world around you using a projector (Display). +5. Add/remove/slide/rotate/add markers to programs and trigger the mappings between your papers position in space and your code! +6. Change your programs, iterate, and keep the creativity going!
![Paper Playground setup detecting paper programs that create a moon lander and control vertical thrust](assets/full-interface.png){ width=900 }
The interface in action.
-## What can you do with it? - -
- ![create play display collab](assets/paper-features.png){ width=600 } -
-
- -You'll create JavaScript programs to populate a page (Board) with interactive, multimodal components to envision anything you want! +You'll create JavaScript programs to populate a page (Display) with interactive, multimodal components to envision anything you want! -Want to create a simple game? Maybe an interactive art experience? Or maybe you want to prototype auditory displays (speech and sounds) for a project or idea that you have? It's all possible with Paper Playground. +Want to create a card game? Maybe an interactive art experience? Or maybe you want to prototype auditory displays (speech and sounds) for a project or idea that you have? It's all possible with Paper Playground. Every program you write is associated with a sequence of colored (black, red, green, and blue) dots that you will put (or print) on the corners of a piece of paper. We'll refer to the linked papers and code as paper programs from here on out! diff --git a/docs/setup/device-setup.md b/docs/setup/device-setup.md index 184ee140..f1d6cf44 100644 --- a/docs/setup/device-setup.md +++ b/docs/setup/device-setup.md @@ -83,7 +83,7 @@ Calibration also sets the average dot size. This will determine how large dots n #### Fine-tuning program detection -If your setup is not detecting all of the dots on your papers, you won't be able to calibrate them and your programs will not be detected (or will be detected intermittently). +If your setup is not detecting all of the dots on your papers, you will not be able to calibrate them and your programs will not be detected (or will be detected intermittently). You can fine-tune the requirements for detecting dots on the paper for your specific setup and lighting conditions. @@ -103,10 +103,20 @@ You can fine-tune the requirements for detecting dots on the paper for your spec `maxArea` - good for filtering out anything that is being mistreated as a marker. -Look under the Detection header in the sidebar of Camera.html and adjust the parameters. Even with adjustments, you may need to improve the lighting of your room or find you need to print papers with larger dots to get reliable dot detection. +Look under the Detection header in the sidebar of Camera.html and adjust the controls. Even with adjustments, you may need to improve the lighting of your room or find you need to print papers with larger dots (or more spaced out dots) to get reliable dot detection. + +- **Average Dot Centers (checkbox):** Improves performance by using an average center to determine dot position, rather than raw pixel coordinates. May decrease dot detection accuracy. + +- **Pixel Value Threshold:** Changes the number of steps used to check for a difference between a dot and background. Large steps might miss dots without enough contrast, but will increase performance. Small steps will find more dots, but slow down performance significantly. +- **Min Pixel Value:** The minimum saturation value (out of 256) to look for a dot. Use this to gate out noise if image features are being detected as dots. +- **Max Pixel Value:** The maximum saturation value (out of 256) to look for a dot. Use this to gate out noise if image features are being detected as dots. +- **Min Dot Area (pixels):** The minimum area in pixels that an image feature must be to be called a dot. Use this to gate out noise if image features are being detected as dots. +- **Max Dot Area (pixels):** The maximum area in pixels that an image feature must be to be called a dot. Use this to gate out noise if image features are being detected as dots. *This is a good slider to use if image features are being erroneously detected as Markers*. +- **Min Dot Separation (pixels):** The minimum numbers of pixels the algorithm expects to see dots. Lower values will help to detect smaller dots or farther away programs, at the cost of detecting more image features that may not be paper dots. +- **Scale Factor:** Recommended to keep at 1. Higher values lower the effective resolution of your camera feed. If you need higher performance and dot detection is not a problem/you are using a very high resolution camera, this will speed up the algorithm significantly to get more performance. ## Canvas Projector Setup -If you're using the legacy Projector page to send visual elements to the web canvas, rather than the Board, you'll follow these intructions instead: +If you're using the legacy Canvas (projector.html) page to send visual elements to the web canvas, rather than the Display (board.html), you'll follow these intructions instead: Power on the projector and separate [http://localhost:3000/projector.html](http://localhost:3000/projector.html) to its own window. Move this window to the projector's display and enter fullscreen mode (Ctrl/Cmd+Shift+F). If you encounter issues (e.g., a recently created program not displaying), refreshing the page should help. diff --git a/docs/setup/reqs.md b/docs/setup/reqs.md index 3cdf9ff1..0d1d0a0e 100644 --- a/docs/setup/reqs.md +++ b/docs/setup/reqs.md @@ -1,9 +1,10 @@ ## Required Hardware for Paper Detection and Control - **Computer**: Any desktop or laptop computer should work. Performance will increase with better computer specifications (CPU, GPU, RAM, etc.). All major operating systems and browsers should also work, though most testing has been done with Windows/MacOS and Chrome/Firefox. -- **Webcam**: 720p is adequate, but be cautious with cameras having low-quality sensors, as they might not perform well under all lighting conditions. +- **Webcam**: 720p is adequate, but be cautious with cameras having low-quality sensors, as they might not perform well under all lighting conditions. 1080p cameras are ideal for use when the camera cannot be close to the play area or paper dots are very small. - **Color Printer**: Alternatively, a black and white printer with coloring markers will do. Check [Resources](../use/resources.md) for templates. - A simple piece of paper and a steady hand can also be effective! + - Try colored stickers too! ## Required Hardware for Projector Use diff --git a/docs/use/creator.md b/docs/use/creator.md index 12095815..d2d372b6 100644 --- a/docs/use/creator.md +++ b/docs/use/creator.md @@ -4,41 +4,43 @@ We are working on updating our documentation - more details coming soon! -The Creator interface is where you can create working paper programs and visualize the components of your paper programs while writing little to no code! +## What is Creator? -Creator is currently organized around the [Model-View-Component framework](mvc.md). +!!! warning "In Development" -You create **Model components**, pieces of your idea that other components will need to use or change to make your project work, to be controlled by **Controller components** and display the information dynamically as a visual (e.g., text, image, shape), sound, and more using **View components** + The Creator interface is in development! The current feature set likely does not reflect the final vision of the tool. Follow the development in [GitHub](https://github.com/phetsims/paper-land) or in our community channels! -??? example "Example project (!! text in progress !!)" +The primary vision of Creator is to bring the relationships between the components of your project to the forefront. The goal is to let you focus on the most important pieces and how they connect to your chosen interactions and outputs (sounds, speech, images, etc). - Let's say you wanted to make a frog jump around your screen. Let's add some interactivity and make it jump to wherever a piece of paper. Say, a lily pad paper? +Creator is organized around the [Model-View-Component framework](mvc.md). - Let's think about the design at a high level and connect it to the needed components: - - We want to know... - - 1. Where the frog is and be able to change where it is (model component > position > "frogPosition") - 2. Controlling the value of `frogPosition` with a "lilypad" paper (controller components > paper > paper movement) - 3. Drawing a picture of the frog (view component > image > "frogPicture") - - Since we want to update the image position separately from the frog's new position (so the image doesn't instantly travel when you move the paper), we'll need to make components for each part (X and Y) of the image position too. - - 4. We'll create a model component > Number > imageYPosition and imageXPosition. - - Edit the frogPicture view component and use the Custom Function to set the Center X and Center Y to frogXPosition and lastly we could add a component for - - 5. Animating that frog when the "lilypad" paper moves (controller component > animation) +Simply stated: You create **Model components** (pieces of your idea that other components will need to use or change to make your project work), to be controlled by **Controller components** and display the information dynamically as a visual (e.g., text, image, shape), sound, speech, and more using **View components** Watch the videos for two simple examples below for creating an "Audio Pendulum" or "Paper Organ". ## Using Templates -## Copying Programs +You can load in templates using the New Template button in the top left of the interface. These will create a paper or set of papers that are pre-populated with components to create the desired effect. Want a shape that will follow the position of your paper? Select the *Movable Shape* template and the required papers/components will appear, ready to be customized! + +### Creating Templates + +Some templates are already included in Paper Playground. However, if you'd like to use templates for common sets of programs/components + +## Copying Projects and Programs + + ## Adding Images and Sounds -Adding your own images (.jpg, .png, .gif, etc.) and sounds (.wav, .mp3, etc.) can be done right in Creator! Add a View Component, +Adding your own images (.jpg, .png, .gif, etc.) and sounds (.wav, .mp3, etc.) can be done right in Creator! Add a View Component, select Image or Sound, and drag/drop your file or select the upload box to open a file dialog. + +Once uploaded, files will live in the paper-land directory under `.../www/media/images/upload` or `.../www/media/sound/upload`. You can also directly add files to this directory for bulk upload. + +The files will now appear in the dropdown selection for those components! + +## Using AI chat to create Custom Code + +- ## Audio Pendulum Walkthrough
diff --git a/docs/use/resources.md b/docs/use/resources.md index 44ea93d6..79044bf9 100644 --- a/docs/use/resources.md +++ b/docs/use/resources.md @@ -11,3 +11,7 @@ Don't have a color printer, but do have markers/colored pencils? Use this templa [Blank Dot Template (GitHub)](https://github.com/phetsims/paper-land/blob/main/docs/use/resources/draw-dots-paper-template.pdf) [Blank Dot Template](./resources/draw-dots-paper-template.pdf) + +!!! tip + + Dot sequences are tied to the program number/ID. When using Creator, you can change the number of the programs you create. This means that you can re-use papers over and over! Just note the number of the paper you have printed and change the program numbers before sending them to the Playground. diff --git a/docs/use/tutorial.md b/docs/use/tutorial.md index 705c1ac6..04c54b24 100644 --- a/docs/use/tutorial.md +++ b/docs/use/tutorial.md @@ -1,14 +1,16 @@ # Using Paper Playground -!!! warning "Under Construction" - +!!! warning "Under Construction" + We are working on updating our documentation - more details coming soon! -Before we begin, make sure you have [installed Paper Playground](../setup/install.md) and [set up your camera](../setup/camera-tips.md). +Before we begin, make sure you have [installed Paper Playground](../setup/install.md) and [set up your camera](../setup/device-setup.md). + +## Interface Overview and Startup -## Step 1: Open up all pages of the interface +### Step 1: Open up all pages of the interface Navigate to localhost:3000 in your browser and either click the links or open in your browser: @@ -17,7 +19,7 @@ Navigate to localhost:3000 in your browser and either click the links or open in 3. Board.html 4. Creator.html -## What is the Camera page? +### What is the Camera page? The Camera page is where all the action happens for turning your paper programs interactive! @@ -27,28 +29,35 @@ On this page you will find the preview of your webcam, a preview of the full Jav You'll use this page to match up where your papers are in space with visuals on a projector. Think of it as your play space. -### What are Preview Papers? :eye: +#### What are Preview Papers? :eye: If you are quickly iterating or just not in a position to print physical papers, you can still see how your code runs! Preview paper programs can be placed into the camera view at any time and the code will run as through you put a physical paper in view. ![Previews in the Camera interface](../assets/camera-spaces-preview-print.png) -While a lot more can be done to help the preview papers mimic how you can move a physical piece of paper, there are a few features currently in place to help. +While a lot more can be done to help the preview papers mimic how you can move a physical piece of paper, there are a few features currently in place to help: -- Move the papers by dragging inside boundaries of the preview. +- Move the papers by dragging inside of the boundaries of the preview. - Resize the papers by dragging an edge. - Drag the green square to rotate the paper. -- Select the scissors to break the paper into four independently moveable corners. +- Select the scissors to 'cut' the paper into four independently moveable corners. + - Once cut, you can rejoin the paper with a new button. - Preview papers can be removed at any time by selecting the red square. -#### Preview Markers +!!! tip + + If you'd like to quickly clear the Preview programs, the best option currently is to refresh the Camera page. + +##### Preview Markers Just like preview papers, you can simulate physical paper markers under the Preview Markers sidebar menu. Select the eye icon to place a draggable marker into the camera view that triggers marker-related code in your paper programs. ![Marker Previews](../assets/camera-preview-markers.png) -## Organizing your code: Spaces and Projects -### What is a Space? +### Organizing your code: Spaces and Projects + +#### What is a Space? + A space is way of organizing your programs and how you'll locate your programs in the Camera interface to run them. Some software might refer to this as a "workspace", but we prefer to think of it as a playspace! !!! warning @@ -57,13 +66,15 @@ A space is way of organizing your programs and how you'll locate your programs i ![Camera Spaces](../assets/camera-spaces-tab.png) -All of the programs in a space can be for one project or you can create a series of simpler programs that all exist in the same Space. This is completely up to you and we use them for both purposes. There is no limit to the number of Spaces you can have, so focusing each Space on one idea works well. +All of the programs in a space can be for one project or you can create a series of simpler programs that all exist in the same Space. This is completely up to you and we use them for both purposes. There is no limit to the number of Spaces you can have, so focusing each Space on one idea works well. You might create your own personal space "my-space" :wink: and fill it with ideas you have or create a "frogger-game" space and collect all of the paper programs that will work together to create your project. -#### What is a Creator Project? +##### What is a Creator Project? +!!! info + If you are not familiar with JavaScript development, we highly recommend you begin with the Creator interface! When working from the Creator interface, you may notice you can create Projects after selecting space. This is one level deeper than a Space and allows you to have multiple version or different sets of programs that you can send to the selected Space in the Camera interface via the **Send to Playground** button. @@ -71,33 +82,39 @@ When working from the Creator interface, you may notice you can create Projects If you select **Send to Playground** for your current project, it will OVERWRITE the programs in the Space. This introduces some incompatibility between editing programs directly in the JavaScript Editor (editor.html) and in Creator. If you only work in Creator and regularly **Save Project**, then you can always select your project and your work will be there. - ** If you'd like to use direct JavaScript and Creator, use the Custom Code button on your paper program in Creator.** + *If you'd like to directly write JavaScript and Creator, use the Custom Code button on your paper program in Creator. This code will be added to the end of each listed event function in the interface (e.g., onProgramAdded).* -## What is the Board page? +### What is the Display page? The main display! This page is where your visuals and sounds will be displayed and if you're using a projector, the page you'll want to put onto your projector to overlay visuals in your play area! -![Board Interface with example visuals](../assets/board-example.png) +![Display Interface with example visuals](../assets/board-example.png) There are a few helpful additions to this page to help with your creation and play. -### Controls +#### Display Controls/Features + +##### Position Interval -You can adjust the sensitivity of your programs to paper movement (in the event of detection jitter) by moving the interval slider to the right of the Board display. +You can adjust the sensitivity of your programs to paper movement (in the event of detection jitter) by moving the **Position Interval** slider to the right of the Board display. !!! warning - Keep this value as low as you can. At high values, your paper will be able to move A LOT without the program recognizing it has moved. Values as low as 0.1-0.2 work in many cases. + Keep this value as low as you can. At high values, your paper will be able to move very far without the program recognizing it has moved. Values as low as **0.1-0.2** work in many cases. + +##### Print Speech -### Console +##### Show Console/Console -If you use the `phet.paperland.console.log/warn` functionality in any custom code section of your programs, that code will display here. See the [Board API](board-api.md) for more information. +If you use the `phet.paperland.console.log`/`warn` functionality in any custom code section of your programs, that code will display here. See the [Paper Playground API](board-api.md/#phetpaperlandconsoleerror-args) for more information. -Regular JavaScript usage of `console.log` will display in your browser's developer console as usual, but will not display in the Board Console. +Regular JavaScript usage of `console.log` will display in your browser's developer console (F12 menu) as usual, but will not display in the Display Console. -## What is the Projector page? +### What is the Canvas page? -Given the current structure of the client, all of the code detected in the Camera is run through the Projector. However, in the latest iteration, we have focused all development on paper code outputting to the Board. The Projector page must be opened to run the code, but the tab or window does not need to be visible. +At this time, given the legacy structure of the client, all of the code detected in the Camera is executed through the Canvas. However, in Paper Playground, all development has been focused on paper program output sent to the Display. + +**The Canvas page must be opened to run the code, *but the tab or window does not need to be visible.*** If you would like to use vanilla Web Canvas or make use of the projection overlay feature of the legacy Paper Programs project, then this is the page where the code will output. See [https://github.com/janpaul123/paperprograms/blob/master/docs/tutorial.md](https://github.com/janpaul123/paperprograms/blob/master/docs/tutorial.md) for more on this. @@ -108,6 +125,7 @@ If you would like to use vanilla Web Canvas or make use of the projection overla See [the Creator tutorial](creator.md). [![Creator interface example](../assets/creator-create-component.png)](creator.md) + ### Editor: If you're comfortable with JavaScript and reading API documents Create new paper programs from the Camera interface (Spaces and Programs on the sidebar). You can copy an existing program from any other Space or start from a fresh template. @@ -122,9 +140,9 @@ Open that link (make sure you keep the Camera page open at all times) and select Make some changes in the editor (try changing what text gets drawn on the canvas for starters) and click the "save" button. The changes should now be reflected in the output of the Board or Projector pages. -When you're ready to print that program as a real piece of paper, click the Printer icon beside the program name on the Camera page. Place the newly printed paper in your webcam's view. +When you're ready to print that program as a real piece of paper, click the Printer icon beside the program name on the Camera page. Place the newly printed paper in your webcam's view. -!!! note +!!! note Paper Playground looks for a comment on the first line of the file and uses that as the program's name. It will use the keywords on the second line for search purposes. -At this point you should have everything you need to start building a collaborative Paper Playground "play" Space and a working interactive paper program experience! Look at the [Board API reference](board-api.md) for more information about the functions available to your programs. \ No newline at end of file +At this point you should have everything you need to start building a collaborative Paper Playground "play" Space and a working interactive paper program experience! Look at the [Board API reference](board-api.md) for more information about the functions available to your programs.