Improve definition of ROI

[samcunliffe]

Consider non-square shapes for the ROI.

• Polygon.
• Freeform selection of pixels?

It only makes sense to apply this to a group of videos.

Dependencies

• Common coordinate system for videos #3

[niksirbi]

I will tackle this, together with issue #16 in PR #28

[niksirbi]

I am leaning towards basing the implementation on this example.

[niksirbi]

Some ideas based on recent discussions

These are meant for the short and medium term (before we get to ArUco markers, as discussed under issue #3 )

1. Use one of the defined ROIs (e.g. the enclosure), to infer the approximate positions of other ROIs. Workflow suggested by @samcunliffe :

Workflow: I get the ROI’s perfect on the first video. Then draw one on the next video (i.e. the enclosure but doesn’t have to be). Then all the ROIs are “guessed” relative to the first ROI, i.e. through clicking a "infer ROI positions" button.

2. Draw an origin + x axis + yaxis, to define a coordinate system, suggested by @sfmig . This could be achieved, for example, by having the user draw two lines - one from the bottom left corner of the enclosure to its bottom right corner, the other from bottom left to top left. ROIs can be then transformed relatively to this coordinate system, and it would also serve as the common space for group analyses (across animals/sessions).
3. Perhaps there is a way to combine the above two ideas, i.e. define the enclosure only, and subsequently extract its corners + edges for coordinate system definition.

That said, we are constrained by the shape drawing tools currently available in dash:

• drawline
• drawopenpath
• drawclosedpath
• drawcircle
• drawrect
• eraseshape

The most relevant for the above applications are drawline (for solution 2), drawrect and drawclosedpath (for solutions 1 and 3).

The latter (drawclosedpath) allows you to draw arbitrary polygons (which is what we want for ROIs), but it is not implemented in the "usual way" (clicking one vertex after the other), but instead, as a freehand drawing tool using one continuous hand motion. This makes it quite hard to draw precise shapes. One can drag vertices (or even delete them) to correct the shape afterward, but this is quite time-consuming.

Drawing rectangles is very easy by comparison, but they are constrained to right angles (I know, right?), with no possibility for rotations or shears, meaning they will not exactly fit most ROIs.

To complicate the issue further, Dash stores rectangle and closed paths in different data structures under the hood (just x0, x1, y0, y1 are stored for rectangles, while paths are stored using the SVG syntax. It is possible to convert between the two representations, but we would have to implement the necessary utility functions.

TLDR

There are some neat ideas to both simplify ROI drawing (issues #11 and #16) and define a common coordinate system for videos (issue #3), in a single swoop. However, I'm currently unsure how to best implement the ideas, given the constraints imposed by the available Dash tools.

[sfmig]

@niksirbi I'd suggest we move this to v1, and replace it with #44 in v0, which concerns smaller changes and seems a bit easier to tackle. Let me know thoughts.

[samcunliffe]

Well as per our original description, this has already been completed.

#44 can stay in v0 if you think that's reasonable and we make Niko's 1-3 points as a low-priority future ticket, or just leave them as a comment here and accept the fact that we're limited by dash plotly.

[niksirbi]

@niksirbi I'd suggest we move this to v1, and replace it with #44 in v0, which concerns smaller changes and seems a bit easier to tackle. Let me know thoughts.

Alright, I'll try to get #44 done first

[niksirbi]

Well as per our original description, this has already been completed.

Yes, freeform selection is already there, so in a sense that's a clear improvement on the previous status quo (rectangles only)

#44 can stay in v0 if you think that's reasonable and we make Niko's 1-3 points as a low-priority future ticket, or just leave them as a comment here and accept the fact that we're limited by dash plotly.

Actually the points 1-3 are now essentially #55.

So, I say let's close this one, and I'll work on #44 and #55, which are better-defined chunks of work.