Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remote/client: implement udisks2-using write-files command #1219

Merged
merged 4 commits into from
Mar 25, 2024

Conversation

a3f
Copy link
Contributor

@a3f a3f commented Jun 25, 2023

Description

Adds a write-files client command to copy files onto mass storage. This is useful for bootloader development for processors with bootroms that load code from file systems (e.g. EFI system partition, OMAP, AT91, Zynq, Raspberry ... etc.). The user interface tries to reproduce the semantics of the UNIX cp command as much as possible.

As of now, we can only copy individual files, but not recursively on directories; in the future we might want to extend the ManagedFile class to support this use case as well.

This is mostly based on @sephalon's #548, with some sprinkles of my #820, but with udisksctl instead of pmount.

How to test

labgrid-client write-files a b      # writes files a, b into /mnt/
labgrid-client write-files -t a b c # writes files b, c into /mnt/a/
labgrid-client write-files -T a b   # writes file  b      to /mnt/a

Checklist

  • Documentation for the feature
  • Tests for the feature
  • The arguments and description in doc/configuration.rst have been updated
  • CHANGES.rst has been updated
  • PR has been tested
  • Man pages have been regenerated

@codecov
Copy link

codecov bot commented Jun 25, 2023

Codecov Report

Attention: Patch coverage is 16.56051% with 131 lines in your changes are missing coverage. Please review.

Project coverage is 62.2%. Comparing base (6c8c6e5) to head (a712ee2).

Files Patch % Lines
labgrid/util/agents/udisks2.py 0.0% 70 Missing ⚠️
labgrid/driver/usbstoragedriver.py 24.5% 43 Missing ⚠️
labgrid/remote/client.py 40.0% 18 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##           master   #1219     +/-   ##
========================================
- Coverage    62.7%   62.2%   -0.6%     
========================================
  Files         163     164      +1     
  Lines       12051   12191    +140     
========================================
+ Hits         7560    7584     +24     
- Misses       4491    4607    +116     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Emantor
Copy link
Member

Emantor commented Jun 27, 2023

Hi Ahmad, the idea for the feature looks great, however we would very much prefer a solution which uses the udisks2 DBus API with the agent interface. The regex may not work across versions and the DBus interface should be much more stable.

@a3f
Copy link
Contributor Author

a3f commented Jun 28, 2023

The case-insensitive regexes are only used for error recovery and if they don't match, they would just bubble up the error. The only hard requirement is the Mounted %s at %s message. But yes, using udiskctl in scripts is inferior to pmount. I have reworked the PR to do DBus in an agent. Looking forward to your feedback.

@a3f a3f force-pushed the write-files branch 2 times, most recently from 9fc5087 to 552a8d3 Compare June 28, 2023 13:23
Emantor
Emantor previously approved these changes Jul 14, 2023
write_image currently waits for only the block device to appear, even if
the image should be written to a partition. Improve this a bit by
polling for the partition itself to appear with a non-zero size.

While at it, we refactor the code a bit to improve code reuse when we
add the write_files method in the follow-up commit.

Signed-off-by: Stefan Wiehler <stefan.wiehler@missinglinkelectronics.com>
Co-authored-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
@Bastian-Krause
Copy link
Member

Rebased on latest master and added a few fixups. With those applied, I'm all for finally merging this PR.

@a3f @sephalon Are those fixups okay with you?

@a3f
Copy link
Contributor Author

a3f commented Mar 24, 2024

Thanks, @Bastian-Krause! Changes look ok to me.

a3f and others added 3 commits March 25, 2024 10:29
While udisksctl exists as a client tool to talk to the udisks2 daemon,
it's not well-suited for programmatic use. We thus introduce an agent
that is copied to the remote system and that uses PyGObject to directly
talk DBus to the daemon to mount and unmount file systems.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
Signed-off-by: Bastian Krause <bst@pengutronix.de>
write_image, especially with the seek and skip parameters, can be useful
for fast iterations during bootloader development: Only the bootloader
is copied and the rest of the storage medium stays intact.

Some boot firmware however loads subsequent boot stages from a, usually
FAT32, filesystem, notably UEFI firmware from the EFI system partition.

Make development on such targets easier by adding a write_files method
that temporarily mounts the target device using udisks2 and writes the
specified files there.

Signed-off-by: Stefan Wiehler <stefan.wiehler@missinglinkelectronics.com>
Co-authored-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
Signed-off-by: Bastian Krause <bst@pengutronix.de>
Analogously to write-image, add a write-file command that can be used to
overwrite (or create anew) a single file in a file system located on a
USBStorageDriver. This is useful for faster iteration during bootloader
development when a bootloader is loaded from a file system.

Following usages are supported:

  labgrid-client write-files a b      # writes files a, b into /mnt/
  labgrid-client write-files -t a b c # writes files b, c into /mnt/a/
  labgrid-client write-files -T a b   # writes file  b      to /mnt/a

Where /mnt is some dynamic path on the, possibly remote, host exporting
the USBStorageDriver.

Signed-off-by: Stefan Wiehler <stefan.wiehler@missinglinkelectronics.com>
Co-authored-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
Signed-off-by: Bastian Krause <bst@pengutronix.de>
@Bastian-Krause
Copy link
Member

Fixups squashed.

@Emantor Emantor merged commit aa08ff2 into labgrid-project:master Mar 25, 2024
9 of 11 checks passed
@a3f a3f deleted the write-files branch March 25, 2024 13:24
jluebbe added a commit to jluebbe/labgrid that referenced this pull request Apr 9, 2024
Otherwise, udisks2 and gi are needed for all installations, even if
write_files is never used.

Fixes: labgrid-project#1219
Signed-off-by: Jan Luebbe <jlu@pengutronix.de>
jluebbe added a commit to jluebbe/labgrid that referenced this pull request Apr 9, 2024
Otherwise, udisks2 and gi are needed for all installations, even if
write_files is never used.

Fixes: labgrid-project#1219
Signed-off-by: Jan Luebbe <jlu@pengutronix.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants