Skip to content

Commit

Permalink
Fixed command for indexing/search flat vectors
Browse files Browse the repository at this point in the history
  • Loading branch information
lintool committed Aug 8, 2024
1 parent 6f45516 commit 3e68abc
Show file tree
Hide file tree
Showing 153 changed files with 310 additions and 304 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building quantized flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-arguana.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat-int8.beir-v1.0.0-arguana.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-arguana.test.bge-base-en-v1.5.jsonl.gz \
-topicReader JsonStringVector \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building quantized flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-arguana.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat-int8.beir-v1.0.0-arguana.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-arguana.test.tsv.gz \
-topicReader TsvString \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-arguana.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat.beir-v1.0.0-arguana.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-arguana.test.bge-base-en-v1.5.jsonl.gz \
-topicReader JsonStringVector \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-arguana.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat.beir-v1.0.0-arguana.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-arguana.test.tsv.gz \
-topicReader TsvString \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building quantized flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-bioasq.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat-int8.beir-v1.0.0-bioasq.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-bioasq.test.bge-base-en-v1.5.jsonl.gz \
-topicReader JsonStringVector \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building quantized flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-bioasq.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat-int8.beir-v1.0.0-bioasq.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-bioasq.test.tsv.gz \
-topicReader TsvString \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-bioasq.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat.beir-v1.0.0-bioasq.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-bioasq.test.bge-base-en-v1.5.jsonl.gz \
-topicReader JsonStringVector \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-bioasq.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat.beir-v1.0.0-bioasq.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-bioasq.test.tsv.gz \
-topicReader TsvString \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building quantized flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-climate-fever.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat-int8.beir-v1.0.0-climate-fever.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-climate-fever.test.bge-base-en-v1.5.jsonl.gz \
-topicReader JsonStringVector \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building quantized flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-climate-fever.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat-int8.beir-v1.0.0-climate-fever.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-climate-fever.test.tsv.gz \
-topicReader TsvString \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-climate-fever.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat.beir-v1.0.0-climate-fever.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-climate-fever.test.bge-base-en-v1.5.jsonl.gz \
-topicReader JsonStringVector \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-climate-fever.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat.beir-v1.0.0-climate-fever.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-climate-fever.test.tsv.gz \
-topicReader TsvString \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building quantized flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-cqadupstack-android.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat-int8.beir-v1.0.0-cqadupstack-android.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-cqadupstack-android.test.bge-base-en-v1.5.jsonl.gz \
-topicReader JsonStringVector \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building quantized flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-cqadupstack-android.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat-int8.beir-v1.0.0-cqadupstack-android.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-cqadupstack-android.test.tsv.gz \
-topicReader TsvString \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-cqadupstack-android.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat.beir-v1.0.0-cqadupstack-android.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-cqadupstack-android.test.bge-base-en-v1.5.jsonl.gz \
-topicReader JsonStringVector \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-cqadupstack-android.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat.beir-v1.0.0-cqadupstack-android.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-cqadupstack-android.test.tsv.gz \
-topicReader TsvString \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building quantized flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-cqadupstack-english.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat-int8.beir-v1.0.0-cqadupstack-english.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-cqadupstack-english.test.bge-base-en-v1.5.jsonl.gz \
-topicReader JsonStringVector \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building quantized flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-cqadupstack-english.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat-int8.beir-v1.0.0-cqadupstack-english.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-cqadupstack-english.test.tsv.gz \
-topicReader TsvString \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-cqadupstack-english.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat.beir-v1.0.0-cqadupstack-english.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-cqadupstack-english.test.bge-base-en-v1.5.jsonl.gz \
-topicReader JsonStringVector \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-cqadupstack-english.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat.beir-v1.0.0-cqadupstack-english.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-cqadupstack-english.test.tsv.gz \
-topicReader TsvString \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building quantized flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-cqadupstack-gaming.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat-int8.beir-v1.0.0-cqadupstack-gaming.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-cqadupstack-gaming.test.bge-base-en-v1.5.jsonl.gz \
-topicReader JsonStringVector \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ After download and unpacking the corpora, the `run_regression.py` command above
Sample indexing command, building quantized flat indexes:

```
bin/run.sh io.anserini.index.IndexCollection \
bin/run.sh io.anserini.index.IndexFlatDenseVectors \
-collection JsonDenseVectorCollection \
-input /path/to/beir-v1.0.0-cqadupstack-gaming.bge-base-en-v1.5 \
-generator DenseVectorDocumentGenerator \
Expand All @@ -50,7 +50,7 @@ Topics and qrels are stored [here](https://github.com/castorini/anserini-tools/t
After indexing has completed, you should be able to perform retrieval as follows:

```
bin/run.sh io.anserini.search.SearchCollection \
bin/run.sh io.anserini.search.SearchFlatDenseVectors \
-index indexes/lucene-flat-int8.beir-v1.0.0-cqadupstack-gaming.bge-base-en-v1.5/ \
-topics tools/topics-and-qrels/topics.beir-v1.0.0-cqadupstack-gaming.test.tsv.gz \
-topicReader TsvString \
Expand Down
Loading

0 comments on commit 3e68abc

Please sign in to comment.