Make glob consistent with glob.glob

[mariosasko]

Makes glob(path) consistent with glob.glob(path, recursive=True, include_hidden=True).

This means ** would only be allowed between the path separators. Although breaking, I think this change is good as the current handling of ** is somewhat unexpected (and not particularly useful?).

Also fixes #1380.

[mariosasko]

Should be easy to fix the failing tests:

• gcsfs: replace the line assert fn in gcs.glob(TEST_BUCKET + "/**1") in test_gcs_glob with assert fn in gcs.glob(TEST_BUCKET + "/**/*1")
• downstream (dask) : replace the line root = "http://localhost:8999/" in test_open_glob with root = "http://localhost:8999"

(I can open PRs there after this one gets approved.)

[martindurant]

"**" would only be allowed between the path separators

So I understand, is this the only change in the behaviour of glob that we expect from your change?

[mariosasko]

So I understand, is this the only change in the behaviour of glob that we expect from your change?!

Yes, that's correct.

[martindurant]

@mariosasko , do you have an opinion on #1394 ?

[mariosasko]

Commented on the issue :)

[martindurant]

Due to the test breakages downstream, I will release before merging, and then merge maybe on Monday, so that users won't see this until we've tidied up the other libraries too.

[mariosasko]

Sounds good!

[martindurant]

Will you be available to make the PR to dask? I will merge when you have time to do it, so that we don't cause their CI to break for too long. gcsfs is less urgent, and I can do that anyway.

[mariosasko]

PR opened in dask :)

[martindurant]

Do you intend to fix gcsfs, or will I need to deal with it?

[mariosasko]

I linked a gcsfs PR with the fix.

[efiop]

For the record: turns out this also broke adlfs but we didn't have daily tests to notice (which were also broken for another reason 🤦‍♂️ ), so I've added those and fixed the tests.

[martindurant]

turns out this also broke adlfs

Do you mean that things are now in a good place?

[efiop]

@martindurant Yes, all good now, no action needed. Here's the main PR fixing that fsspec/adlfs#448 and a few more (fsspec/adlfs#447, fsspec/adlfs#446, fsspec/adlfs#445) related to the other reasons for adlfs tests failing. So all fixed now and should be sustainable long term.