TileDB-SOMA 1.14.2

What's Changed

• Bulk-backport Python 3.8 and somacore 1.0.17 mods to release-1.14 branch by @johnkerl in #3074

Full Changelog: 1.14.1...1.14.2

TileDB-SOMA 1.14.2rc0

What's Changed

• Bulk-backport Python 3.8 and somacore 1.0.17 mods to release-1.14 branch by @johnkerl in #3074

Full Changelog: 1.14.1...1.14.2

TileDB-SOMA 1.14.1

What's Changed

• [python] Fix deprecation warnings [release-1.14] by @johnkerl in #3013
• [r] TileDB-SOMA 1.14.1 by @johnkerl in #3016

Full Changelog: 1.14.0...1.14.1

TileDB-SOMA 1.14.0

New features added

• [python] update_uns_by_uri, _update_uns by @ryan-williams in #2876
• [python] Remove pyarrow pin on MacOS by @johnkerl in #3004
• [python] Expose tiledbsoma stats as JSON string or Python-parsed by @johnkerl in #2958
• [r] Timestamp write and read of data frame and array objects by @eddelbuettel in #2926

Bug fixes

• [python] Utilize Arrow schema pa.field nullabilities in DataFrame.create by @johnkerl in #2868
• [python] Offer better guidance on attribute names with . by @johnkerl in #2864
• [python] X-write performance improvement for the non-append case by @johnkerl in #2846
• [c++] Fix ManagedQuery usage in SOMAArray write path by @nguyenv in #2989
• [c++] Only evolve schema when enumerations are extended by @nguyenv in #2851
• [c++] Corner-case bug in extend-enumeration logic by @johnkerl in #2897
• [c++] Remap indexes even if no new enums were added by @nguyenv in #2993

Breaking changes

• [python] Post-deprecation removal of consolidate_and_vacuum from TileDBCreateOptions by @johnkerl in #3000
• [python] Implement scheduled removal of TileDBCreateOptions being passed to write methods by @johnkerl in #2842

Upcoming deprecation notices

• [python/r] Update ETA for new-shape feature by @johnkerl in #2998
• Add tracking links for dense/new-shape support by @johnkerl in #2960

Minor changes

• [python] Ingest/outgest round-trip improvements by @ryan-williams in #2804
• [python] Add CollectionBase.members by @nguyenv in #2795
• [python] Use original error message when re-raising with DoesNotExistError by @nguyenv in #2814
• [python] Debug CI mypy failures related to # type: ignore[misc] by @ryan-williams in #2838
• [python] Misc. ingest/outgest code-neaten improvements by @ryan-williams in #2824
• [python] pin pandas-stubs>=2 during pre-commit mypy hook by @ryan-williams in #2854
• [python] actions{cache,checkout,setup-python} upgrades by @ryan-williams in #2856
• [python] Move _update_column into pybind11 by @nguyenv in #2862
• [python] test_registration_mappings.py::test_pandas_indexing reformat by @ryan-williams in #2853
• [python] Fix a comment by @johnkerl in #2875
• [python] Use somacore 1.0.14 by @johnkerl in #2878
• [python] More AnnData equality assertions in tests by @ryan-williams in #2872
• [python] Update comments, remove unused test params by @ryan-williams in #2873
• [python] Remove some dead code by @johnkerl in #2887
• [python] improve uns DataFrame I/O by @ryan-williams in #2874
• [python] Docstring typo, test-assertion msgs, maybe_raises improvement, verify_logs helper by @ryan-williams in #2892
• [python] Unit-test helper make_df -> make_pd_df by @johnkerl in #2898
• [python] Trivial rename within a unit-test file by @johnkerl in #2908
• [python] Fix ref to method in docstring example by @spencerseale in #2942
• [python/r] Expose shape-related accessors to Python/R bindings by @johnkerl in #2953
• [c++/python] Map core-to-soma domains correctly by @johnkerl in #2957
• [python] Fix nightly-build failure / pybind11 exception-mapping by @johnkerl in #2963
• [python] Minor name-neaten in internals for domain/maxdomain by @johnkerl in #2969
• [python] Trivial dead-strip by @johnkerl in #2968
• [python/r] Array-creation mods for new shape by @johnkerl in #2962
• [c++/python] More exception-mapping fixes by @johnkerl in #2980
• [python] Complete TileDB-Py 0.32.0 pin by @johnkerl in #2984
• [python] Use somacore 1.0.15 by @johnkerl in #2986
• [python/r] Implement resize and tiledbsoma_upgrade_shape by @johnkerl in #2950
• [python] Move temporary feature flag into a module by @johnkerl in #3005
• shellcheck scripts/bld by @ryan-williams in #2927
• Use core 2.26.0 by @johnkerl in #2976
• [r] R DESCRIPTION bump in prep for 1.13 (main branch) by @johnkerl in #2836
• [r] Metadata read/write support via libtiledbsoma by @eddelbuettel in #2819
• [ci] Update autocut rule for current ownership by @johnkerl in #2893
• [r] Small robustification for version comparison by @eddelbuettel in #2914
• [r] Expose timestamp ranges for libtiledbsoma by @mojaveazure in #2929
• [r] Use cached timestamp in $create() and $write() by @mojaveazure in #2930
• [r] Signal script failure to calling script, polish messages by @eddelbuettel in #2932
• [r] Fix bug in blockwise iterator by @mojaveazure in #2965
• [r/ci] Fix bug when installing tiledb-r by @mojaveazure in #2971
• [r] Fix DenseNDArray write after create by @johnkerl in #2970
• [r] Updated groups read/write support via libtiledbsoma by @eddelbuettel in #2973
• [r] Correct a test expression by @eddelbuettel in #2974
• [r] Use cached SOMA context rather than re-creating by @mojaveazure in #2988
• [r] Lower some logging from 'info' to 'debug' by @eddelbuettel in #2991
• [r] Documentation refresher by @eddelbuettel in #2996
• [r] Remove core version check by @johnkerl in #3006
• [r] Add axis checks when initializing a sparse blockwise read by @mojaveazure in #3007
• [r] 1.14 version for release-1.14 branch by @johnkerl in #3010
• [c++] Support option to set log level from environment by @johnkerl in #2972
• [c++] Add dataset_type into SOMAExperiment metadata by @nguyenv in #2884
• [c++] Fix typos in unit-test case by @johnkerl in #2888
• [c++] Remove a temporary gdb hook from #2888 by @johnkerl in #2891
• [c++] Apply ArraySchemaEvolution to specified timestamp by @johnkerl in #2895
• [c++] Final update for Python/R metadata typing by @johnkerl in #2900
• [c++] Trivial parameterizes in test/common.cc by @johnkerl in #2910
• [c++] Methodize timestamped-schema-evolution factory by @johnkerl in #2909
• [c++] Trivial name-neaten by @johnkerl in #2913
• [c++] Remove some dead unit-test code by @johnkerl in #2918
• [c++,r] Update vendored nanoarrow to release 0.5.0 by @eddelbuettel in #2923
• [c++] Revert #2895 by @ryan-williams in #2921
• [c++] Remove some dead code in int64-only shape accessor by @johnkerl in #2915
• [c++] Arrow utils with current-domain option by @johnkerl in #2911
• [c++] Minor function-extract in a unit-test helper by @johnkerl in #2919
• [c++] Pre-neatens before making unit-test helpers for variant-indexed dataframes by @johnkerl in #2936
• [c++] More use_current_domain unit-test parameterization by @johnkerl in #2938
• [c++] Parameterize schema-creation unit-test helpers by @johnkerl in #2939
• [c++] Unit-test resize for SparseNDArray and DenseNDArray by @johnkerl in #2947
• [c++] Make a Catch2 test fixture for dataframes by @johnkerl in #2945
• [c++] Unit-test variant-indexed dataframes by @johnkerl in #2944
• [c++] Performant DataFrame.shape by @johnkerl in #2916
• [c++] Resize for variant-indexed DataFrame by @johnkerl in #2917
• [c++] Implement upgrade_shape for SparseNDArray and DenseNDArray by @johnkerl in #2948
• [c++] Clarify dataframe-shaping test/access points by @johnkerl in #2951
• [c++] Also check for leading slash when checking for 'relative' URLs by @eddelbuettel in #2956
• [c++] Deconfuse a test name by @johnkerl in #2977
• [c++] Remove ArrayBuffers from SOMAArray by @nguyenv in #2979
• [c++] Create specialized templates for SOMAArray casting functions by @nguyenv in #2982
• [c++] Refine concurrency configuration (soma.compute_concurrency_level) by @bkmartinjr in #2978
• [c++] Support writes when attaching Query to ColumnBuffer by @nguyenv in #2983
• [c++] Move cast_bit_to_uint8 into util by @nguyenv in #2992
• [r,c++] Use SOMA Context object throughout by @eddelbuettel in #2997
• [c++] Centralize some nanoarrow helpers by @johnkerl in #2994
• [c++] Use core 2.26.1 by @johnkerl in #3002
• [c++/r] Fix pkgdown build errors by @johnkerl in #3003

New Contributors

• @spencerseale made their first contribution in #2942

Full Changelog: 1.13.1...1.14.0

TileDB-SOMA 1.13.1

Bug fixes

• [Backport release-1.13] [c++] Corner-case bug in extend-enumeration logic by @github-actions in #2903
• [Backport release-1.13] [python] Offer better guidance on attribute names with . by @github-actions in #2865
• [Backport release-1.13] python Utilize Arrow schema pa.field nullabilities in DataFrame.create by @github-actions in #2870
• [Backport release-1.13] [c++] Add dataset_type into SOMAExperiment metadata by @github-actions in #2886
• [Backport release-1.13] [c++] Apply ArraySchemaEvolution to specified timestamp by @github-actions in #2901

Minor changes

• [Backport release-1.13] [c++] Only evolve schema when enumerations are extended by @github-actions in #2852
• [Backport release-1.13] [python] pin pandas-stubs>=2 during pre-commit mypy hook by @github-actions in #2855
• [Backport release-1.13] [python] Fix a comment by @github-actions in #2877
• [Backport release-1.13] [c++] Fix typos in unit-test case by @github-actions in #2889
• [Backport release-1.13] [c++] Remove a temporary gdb hook from #2888 by @github-actions in #2894
• [Backport release-1.13] [c++] Final update for Python/R metadata typing by @github-actions in #2902

Full Changelog: 1.13.0...1.13.1

TileDB-SOMA 1.13.1rc0

Bug fixes

• [Backport release-1.13] [c++] Corner-case bug in extend-enumeration logic by @github-actions in #2903
• [Backport release-1.13] [python] Offer better guidance on attribute names with . by @github-actions in #2865
• [Backport release-1.13] python Utilize Arrow schema pa.field nullabilities in DataFrame.create by @github-actions in #2870
• [Backport release-1.13] [c++] Add dataset_type into SOMAExperiment metadata by @github-actions in #2886
• [Backport release-1.13] [c++] Apply ArraySchemaEvolution to specified timestamp by @github-actions in #2901

Minor changes

• [Backport release-1.13] [c++] Only evolve schema when enumerations are extended by @github-actions in #2852
• [Backport release-1.13] [python] pin pandas-stubs>=2 during pre-commit mypy hook by @github-actions in #2855
• [Backport release-1.13] [python] Fix a comment by @github-actions in #2877
• [Backport release-1.13] [c++] Fix typos in unit-test case by @github-actions in #2889
• [Backport release-1.13] [c++] Remove a temporary gdb hook from #2888 by @github-actions in #2894
• [Backport release-1.13] [c++] Final update for Python/R metadata typing by @github-actions in #2902

Full Changelog: 1.13.0...1.13.1rc0

TileDB-SOMA 1.13.0

Deprecation notices

As noted in #2407, there will be new shape support in the upcoming TileDB-SOMA 1.14, with some impact to user API in the append-to-experiment use-case.

As noted in #2843, TileDBCreateOptions may no longer be passed to write methods.

• [Backport release-1.13] [python] Implement scheduled removal of TileDBCreateOptions being passed to write methods by @github-actions in #2843
• [python] Deprecate tiledb_ctx usage in TileDBSOMAContext by @nguyenv in #2693
• [python] Deprecate tiledb.Filter getters in PlatformConfig by @nguyenv in #2720
• [r/python] Deprecation notices for used_shape by @johnkerl in #2834
• [r] Use .Deprecated for used_shape in R by @johnkerl in #2835

New features added

• [python] Optionally drop unused categories in ExperimentAxisQuery.to_anndata by @nguyenv in #2811
• [Backport release-1.13] [python] X-write performance improvement for the non-append case by @github-actions in #2848
• [Backport release-1.13] [python] Add CollectionBase.members by @github-actions in #2849
• [c++/python] Add reopen for SOMAArray and SOMAGroup by @nguyenv in #2802
• [r] Connect re-indexer to blockwise iterator by @mojaveazure in #2742
• [r] Add drop_levels to SOMAExperimentAxisQuery -> ecosystem outgestors by @mojaveazure in #2825

Bug fixes

• [Backport release-1.13] [python] Use original error message when re-raising with DoesNotExistError by @github-actions in #2844
• [python] Support tiledbsoma.io.update_obs with post-nullable non-string attributes by @johnkerl in #2719
• [python] Allow platform config to set both extent and filter by @johnkerl in #2808
• [python] Improve error message on non-constant query-condition RHS by @johnkerl in #2831
• [python] More gracefully handle invalid operators in query-condition parser by @johnkerl in #2830
• [python] Allow platform config to set extent and filter by @johnkerl in #2808
• [c++] Fix enum-of-bool leveling by @johnkerl in #2723

Minor changes

• Use core 2.25, TileDB-Py 0.31, TileDB-R 0.29 by @johnkerl in #2813
• [Backport release-1.13] [python] Debug CI mypy failures related to # type: ignore[misc] by @github-actions in #2845
• [Backport release-1.13] [python] Misc. ingest/outgest code-neaten improvements by @github-actions in #2847
• [Backport release-1.13] [python] Ingest/outgest round-trip improvements by @github-actions in #2850
• [r] Prep for TileDB-SOMA 1.13 (release-1.13 branch) by @johnkerl in #2837
• [python/r] Update lifecycle tags to "Maturing" by @johnkerl in #2807
• [python] Pin to numpy < 2.0 for 1.12 release by @johnkerl in #2745
• [python] Add pybind11 bindings for SOMAGroup 2/5 by @nguyenv in #2732
• [python] Use bindings for SOMAGroup 3/5 by @nguyenv in #2735
• [python] Replace TileDBObject with SOMAObject; Remove tiledb.TileDBError usage 4/5 by @nguyenv in #2736
• [python] Remove tiledb-py from top-level src (sans io and options) 5/5 by @nguyenv in #2740
• [python] Remove a debug statement by @johnkerl in #2766
• [python] Continue to use tiledb.Ctx's config until it is removed by @nguyenv in #2768
• [python] devtools/outgestor stats and neaten by @johnkerl in #2770
• [python] Update a confusing filename by @johnkerl in #2776
• [python] Bump the stack level of the tiledb_ctx deprecation warning by @ivirshup in #2788
• [python] Release GIL where necessary in pybind11 code by @nguyenv in #2733
• [python] Hold onto GIL for open methods by @nguyenv in #2800
• [python] Retain GIL for exist by @nguyenv in #2801
• [python] Replace py::none with py::tuple for column_names default arg by @nguyenv in #2806
• [python] Typo-fix some parameter names in some internal functions by @johnkerl in #2809
• [c++/python] Add tiledb_timestamp argument for reopen by @nguyenv in #2820
• [python] Remove reshape stub by @johnkerl in #2826
• [python] More reshape neatens by @johnkerl in #2828
• [python] pin codecov/uploader==0.7.3 in codecov-action by @ryan-williams in #2827
• [r] R versioning past release-1.12 branch by @johnkerl in #2716
• [r,c++] Support DenseNDArray write case by @eddelbuettel in #2730
• Compare core to tiledb-py/tiledb-r major+minor, main branch by @johnkerl in #2784
• [r] Add missing documentation for set_data_type by @eddelbuettel in #2818
• [r] Support SparseNDArray writes case by @eddelbuettel in #2755
• [r] Use exists() to check for (local or) remote arrays by @eddelbuettel in #2803
• [r] Remove default mode-flipping in $reopen() by @mojaveazure in #2817
• [r] Expose object-type getter by @eddelbuettel in #2833
• [c++] Fix a nightly-build issue by @johnkerl in #2718
• [c++] Add missing SOMACollection getters and setters by @nguyenv in #2508
• [c++] More SOMAGroup refinements 1/5 by @nguyenv in #2731
• [c++] Remap TileDBError to TileDBSOMAError by @nguyenv in #2753
• [c++] General CMake clean-up by @jp-dark in #2762
• [c++] Split nanoarrow into a separate object library by @jp-dark in #2763
• [c++] Improve exception-handling for query futures by @johnkerl in #2789
• [c++] Cast ArrowTable values in SOMAArray::write to match ArraySchema on-disk by @nguyenv in #2786
• Bump setuptools from 69.5.1 to 70.0.0 in /doc by @dependabot in #2796
• [c++] exists should take in SOMAContext by @nguyenv in #2794
• [c++] Add NDArray::soma_data_type by @nguyenv in #2810
• [c++] Do not call schema evolution for each column by @nguyenv in #2805
• [c++] Check index in enumeration casting before using by @nguyenv in #2816
• pre-commit update by @bkmartinjr in #2821

New Contributors

• @ivirshup made their first contribution in #2788

Full Changelog: 1.12.0rc0...1.13.0

TileDB-SOMA 1.13.0rc0

Deprecation notices

As noted in #2407, there will be new shape support in the upcoming TileDB-SOMA 1.14, with some impact to user API in the append-to-experiment use-case.

As noted in #2843, TileDBCreateOptions may no longer be passed to write methods.

• [Backport release-1.13] [python] Implement scheduled removal of TileDBCreateOptions being passed to write methods by @github-actions in #2843
• [python] Deprecate tiledb_ctx usage in TileDBSOMAContext by @nguyenv in #2693
• [python] Deprecate tiledb.Filter getters in PlatformConfig by @nguyenv in #2720
• [r/python] Deprecation notices for used_shape by @johnkerl in #2834
• [r] Use .Deprecated for used_shape in R by @johnkerl in #2835

New features added

• [python] Optionally drop unused categories in ExperimentAxisQuery.to_anndata by @nguyenv in #2811
• [Backport release-1.13] [python] X-write performance improvement for the non-append case by @github-actions in #2848
• [Backport release-1.13] [python] Add CollectionBase.members by @github-actions in #2849
• [c++/python] Add reopen for SOMAArray and SOMAGroup by @nguyenv in #2802
• [r] Connect re-indexer to blockwise iterator by @mojaveazure in #2742
• [r] Add drop_levels to SOMAExperimentAxisQuery -> ecosystem outgestors by @mojaveazure in #2825

Bug fixes

• [Backport release-1.13] [python] Use original error message when re-raising with DoesNotExistError by @github-actions in #2844
• [python] Support tiledbsoma.io.update_obs with post-nullable non-string attributes by @johnkerl in #2719
• [python] Allow platform config to set both extent and filter by @johnkerl in #2808
• [python] Improve error message on non-constant query-condition RHS by @johnkerl in #2831
• [python] More gracefully handle invalid operators in query-condition parser by @johnkerl in #2830
• [python] Allow platform config to set extent and filter by @johnkerl in #2808
• [c++] Fix enum-of-bool leveling by @johnkerl in #2723

Minor changes

• Use core 2.25, TileDB-Py 0.31, TileDB-R 0.29 by @johnkerl in #2813
• [Backport release-1.13] [python] Debug CI mypy failures related to # type: ignore[misc] by @github-actions in #2845
• [Backport release-1.13] [python] Misc. ingest/outgest code-neaten improvements by @github-actions in #2847
• [Backport release-1.13] [python] Ingest/outgest round-trip improvements by @github-actions in #2850
• [r] Prep for TileDB-SOMA 1.13 (release-1.13 branch) by @johnkerl in #2837
• [python/r] Update lifecycle tags to "Maturing" by @johnkerl in #2807
• [python] Pin to numpy < 2.0 for 1.12 release by @johnkerl in #2745
• [python] Add pybind11 bindings for SOMAGroup 2/5 by @nguyenv in #2732
• [python] Use bindings for SOMAGroup 3/5 by @nguyenv in #2735
• [python] Replace TileDBObject with SOMAObject; Remove tiledb.TileDBError usage 4/5 by @nguyenv in #2736
• [python] Remove tiledb-py from top-level src (sans io and options) 5/5 by @nguyenv in #2740
• [python] Remove a debug statement by @johnkerl in #2766
• [python] Continue to use tiledb.Ctx's config until it is removed by @nguyenv in #2768
• [python] devtools/outgestor stats and neaten by @johnkerl in #2770
• [python] Update a confusing filename by @johnkerl in #2776
• [python] Bump the stack level of the tiledb_ctx deprecation warning by @ivirshup in #2788
• [python] Release GIL where necessary in pybind11 code by @nguyenv in #2733
• [python] Hold onto GIL for open methods by @nguyenv in #2800
• [python] Retain GIL for exist by @nguyenv in #2801
• [python] Replace py::none with py::tuple for column_names default arg by @nguyenv in #2806
• [python] Typo-fix some parameter names in some internal functions by @johnkerl in #2809
• [c++/python] Add tiledb_timestamp argument for reopen by @nguyenv in #2820
• [python] Remove reshape stub by @johnkerl in #2826
• [python] More reshape neatens by @johnkerl in #2828
• [python] pin codecov/uploader==0.7.3 in codecov-action by @ryan-williams in #2827
• [r] R versioning past release-1.12 branch by @johnkerl in #2716
• [r,c++] Support DenseNDArray write case by @eddelbuettel in #2730
• Compare core to tiledb-py/tiledb-r major+minor, main branch by @johnkerl in #2784
• [r] Add missing documentation for set_data_type by @eddelbuettel in #2818
• [r] Support SparseNDArray writes case by @eddelbuettel in #2755
• [r] Use exists() to check for (local or) remote arrays by @eddelbuettel in #2803
• [r] Remove default mode-flipping in $reopen() by @mojaveazure in #2817
• [r] Expose object-type getter by @eddelbuettel in #2833
• [c++] Fix a nightly-build issue by @johnkerl in #2718
• [c++] Add missing SOMACollection getters and setters by @nguyenv in #2508
• [c++] More SOMAGroup refinements 1/5 by @nguyenv in #2731
• [c++] Remap TileDBError to TileDBSOMAError by @nguyenv in #2753
• [c++] General CMake clean-up by @jp-dark in #2762
• Split nanoarrow into a separate object library by @jp-dark in #2763
• [c++] Improve exception-handling for query futures by @johnkerl in #2789
• [c++] Cast ArrowTable values in SOMAArray::write to match ArraySchema on-disk by @nguyenv in #2786
• Bump setuptools from 69.5.1 to 70.0.0 in /doc by @dependabot in #2796
• [c++] exists should take in SOMAContext by @nguyenv in #2794
• [c++] Add NDArray::soma_data_type by @nguyenv in #2810
• [c++] Do not call schema evolution for each column by @nguyenv in #2805
• [c++] Check index in enumeration casting before using by @nguyenv in #2816
• pre-commit update by @bkmartinjr in #2821

New Contributors

• @ivirshup made their first contribution in #2788

Full Changelog: 1.12.0rc0...1.13.0

TileDB-SOMA 1.12.3

This release brings in TileDB Core 2.24.2.

• [Backport release-1.12] Use core 2.24.2 by @github-actions in #2778
• [Backport release-1.12] [python] Update a confusing filename by @github-actions in #2782
• tiledbsoma-r 1.12.3 by @johnkerl in #2780
• Compare core to tiledb-py/tiledb-r major+minor, release-1.12 branch by @johnkerl in #2783

Full Changelog: 1.12.2...1.12.3

TileDB-SOMA 1.12.2

This release contains a bugfix for the Python API.

Bug fixes

• [Backport release-1.12] [python] Continue to use tiledb.Ctx's config until it is removed by @github-actions in #2769

Minor changes

• [Backport release-1.12] [python] Remove a debug statement by @github-actions in #2767
• [Backport release-1.12] [python] devtools/outgestor stats and neaten by @github-actions in #2771
• [Backport release-1.12] [c++] General CMake clean-up by @github-actions in #2773
• [Backport release-1.12] Split nanoarrow into a separate object library by @github-actions in #2774
• [r] TileDB-SOMA 1.12.2 by @johnkerl in #2775

Full Changelog: 1.12.1...1.12.2