Merge pull request #2 from remerge/SCIENCE-210-update-gcsfs-2025.5.0

SCIENCE-210 update gcsfs 2025.5.0

.github/workflows/ci.yml

@@ -18,16 +18,16 @@ jobs:
 
     steps:
       - name: Checkout source
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
 
       - name: Setup Conda Environment
-        uses: mamba-org/provision-with-micromamba@main
+        uses: mamba-org/setup-micromamba@v1
         with:
           cache-downloads: true
           environment-file: environment_gcsfs.yaml
           environment-name: gcsfs_test
-          extra-specs: |
-            python=${{ matrix.python-version }}
+          create-args: >-
+            python=${{ matrix.PY }}
 
       - name: Conda info
         run: |
@@ -49,6 +49,8 @@ jobs:
     name: lint
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - uses: actions/setup-python@v4
-      - uses: pre-commit/action@v2.0.0
+        with:
+          python-version: "3.11"
+      - uses: pre-commit/action@v3.0.0

docs/source/changelog.rst

@@ -1,6 +1,61 @@
 Changelog
 =========
 
+Note: in some releases, there are no changes, because we always guarantee relasing in step
+with fsspec.
+
+2024.5.0
+--------
+
+* swap order of "gcs", "gs" protocols (#620)
+* fix get_file for relative lpath (#618)
+
+2024.3.1
+--------
+
+* fix expiration= for sign() (#613)
+* do populate dircache in ls() (#612)
+* allow passing extra options to mkdir (#610)
+* credentials docs (#609)
+* retry in bulk rm (#608)
+* clean up loop on close (#606)
+
+2024.2.0
+--------
+
+* doc for passing tokens (#603)
+
+2023.12.2
+---------
+
+no changes
+
+2023.12.1
+---------
+
+no changes
+
+2023.12.0
+---------
+
+* use same version when paginating list (#591)
+* fix double asterisk glob test (#589)
+
+2023.10.0
+---------
+
+* Fix for transactions of small files (#586)
+
+2023.9.2
+--------
+
+* CI updates (#582)
+
+2023.9.1
+--------
+
+* small fixes following #573 (#578)
+
 2023.9.0
 --------
 

docs/source/index.rst

@@ -99,19 +99,33 @@ Several modes of authentication are supported:
       or container engine) and fetch the credentials automatically from the
       metadata service.
 
-    - you may supply a token generated by the
-      gcloud_ utility; this is either a python dictionary, or the name of a file
-      containing the JSON returned by logging in with the gcloud CLI tool (e.g.,
-      ``~/.config/gcloud/application_default_credentials.json`` or
-      ``~/.config/gcloud/legacy_credentials/<YOUR GOOGLE USERNAME>/adc.json``)
-      or any value google ``Credentials`` object.
+    - if ``token=dict(...)`` or ``token=<filepath>``, you may supply a token
+      generated by the gcloud_ utility. This can be
+
+      - a python dictionary
+
+      - the path to a file containing the JSON returned by logging in with the
+        gcloud CLI tool (e.g.,
+        ``~/.config/gcloud/application_default_credentials.json`` or
+        ``~/.config/gcloud/legacy_credentials/<YOUR GOOGLE
+        USERNAME>/adc.json``)
+
+      - the path to a service account key
+
+      - a google.auth.credentials.Credentials_ object
+
+      Note that ``~`` will not be automatically expanded to the user home
+      directory, and must be manually expanded with a utility like
+      ``os.path.expanduser()``.
 
     - you can also generate tokens via Oauth2 in the browser using ``token='browser'``,
       which gcsfs then caches in a special file, ~/.gcs_tokens, and can subsequently be accessed with ``token='cache'``.
 
     - anonymous only access can be selected using ``token='anon'``, e.g. to access
       public resources such as 'anaconda-public-data'.
 
+.. _google.auth.credentials.Credentials: https://google-auth.readthedocs.io/en/master/reference/google.auth.credentials.html#google.auth.credentials.Credentials
+
 The acquired session tokens are *not* preserved when serializing the instances, so
 it is safe to pass them to worker processes on other machines if using in a
 distributed computation context. If credentials are given by a file path, however,

environment_gcsfs.yaml

@@ -12,6 +12,7 @@ dependencies:
   - google-auth
   - google-auth-oauthlib
   - google-cloud-core
+  - google-cloud-storage
   - libfuse<3
   - pytest
   - pytest-timeout