SIGIR'23
Huawei | [FinalMLP](https://arxiv.org/abs/2304.00902) {cite}`FinalMLP`
AAAI'23
Huawei | [EulerNet](https://arxiv.org/abs/2304.10711) {cite}`EulerNet`
SIGIR'23
Huawei | [GDCN](https://arxiv.org/abs/2311.04635) {cite}`GDCN`
CIKM'23
Microsoft | +| | [MemoNet](https://arxiv.org/abs/2211.01334) {cite}`MemoNet`
CIKM'23
Sina Weibo | +| **2022** | [FRNet](https://arxiv.org/abs/2204.08758) {cite}`FRNet`
SIGIR'22
Microsoft | [APG](https://arxiv.org/abs/2203.16218) {cite}`APG`
NeurIPS'22
Alibaba | [FINT](https://arxiv.org/abs/2107.01999) {cite}`FINT`
ICASSP'22
iQIYI | +| **2021** | [DCN-V2](https://arxiv.org/abs/2008.13535) {cite}`DCNv2`
WWW'21
Google | [FM2](https://arxiv.org/abs/2102.12994) {cite}`FM2`
WWW'21
Yahoo | [EDCN](https://dlp-kdd.github.io/assets/pdf/DLP-KDD_2021_paper_12.pdf) {cite}`EDCN`
CIKM'21
Huawei | [DESTINE](https://arxiv.org/abs/2101.03654) {cite}`DESTINE`
CIKM'21
Alibaba | +| | [SAM](https://arxiv.org/abs/2105.05563) {cite}`SAM`
SIGIR'21
BOSS Zhipin | [PCF-GNN](https://arxiv.org/abs/2105.07752) {cite}`PCF-GNN`
SIGIR'21
Alibaba | [xLightFM](https://dl.acm.org/doi/10.1145/3404835.3462941) {cite}`xLightFM`
SIGIR'21 | [AOANet](https://dl.acm.org/doi/10.1145/3447548.3467133) {cite}`AOANet`
KDD'21
Didi Chuxing | +| | [DCAP](https://arxiv.org/abs/2105.08649) {cite}`DCAP`
CIKM'21 | | +| **2020** | [AFN](https://ojs.aaai.org/index.php/AAAI/article/view/5768) {cite}`AFN`
AAAI'20 | [DeepIM](https://dl.acm.org/doi/10.1145/3340531.3412077) {cite}`DeepIM`
CIKM'20
Alibaba | [AutoGroup](https://dl.acm.org/doi/abs/10.1145/3397271.3401082) {cite}`AutoGroup`
SIGIR'20
Huawei | [FWL](https://arxiv.org/abs/2012.00202) {cite}`FWL`
NeurIPS'20 | +| | [ONN](https://arxiv.org/pdf/1904.12579) {cite}`ONN`
NeuralNets'20 | [DIFM](https://www.ijcai.org/Proceedings/2020/0434.pdf) {cite}`DIFM`
IJCAI'20 | [AutoFIS](https://arxiv.org/abs/2003.11235) {cite}`AutoFIS`
KDD'20
Huawei | [AutoCTR](https://arxiv.org/abs/2007.06434) {cite}`AutoCTR`
KDD'20
Facebook | +| |[GLIDER](https://arxiv.org/abs/2006.10966) {cite}`GLIDER`
ICLR'20
Facebook | +| **2019** | [AutoInt](https://arxiv.org/abs/1810.11921) {cite}`AutoInt`
CIKM'19 | [FiGNN](https://arxiv.org/abs/1910.05552) {cite}`FiGNN`
CIKM'19 | [FGCNN](https://arxiv.org/abs/1904.04447) {cite}`FGCNN`
WWW'19
Huawei | [FiBiNET](https://arxiv.org/abs/1905.09433) {cite}`FiBiNET`
RecSys'19
Sina Weibo | +| | [HFM](https://ojs.aaai.org//index.php/AAAI/article/view/4448) {cite}`HFM`
AAAI'19 | [DLRM](https://arxiv.org/abs/1906.00091) {cite}`DLRM`
Arxiv'19
Facebook | [IFM](https://www.ijcai.org/proceedings/2019/203) {cite}`IFM`
IJCAI'19 | +| **2018** | [FwFM](https://dl.acm.org/doi/10.1145/3178876.3186040) {cite}`FwFM`
WWW'18
Yahoo | [xDeepFM](https://arxiv.org/pdf/1803.05170.pdf) {cite}`xDeepFM`
KDD'18
Microsoft | +| **2017** | [NFM](https://arxiv.org/abs/1708.05027) {cite}`NFM`
SIGIR'17 | [FFM](https://arxiv.org/pdf/1701.04099.pdf) {cite}`FFM2`
WWW'17
Criteo | [DCN](https://arxiv.org/abs/1708.05123) {cite}`DCN`
ADKDD'17
Google | [DeepFM](https://arxiv.org/abs/1703.04247) {cite}`DeepFM`
IJCAI'17
Huawei | +| | [AFM](https://www.ijcai.org/proceedings/2017/0435.pdf) {cite}`AFM`
IJCAI'17 | +| **2016
&before** | [FFM](https://www.csie.ntu.edu.tw/~cjlin/papers/ffm.pdf) {cite}`FFM`
RecSys'16 | [YoutubeDNN](https://research.google.com/pubs/archive/45530.pdf) {cite}`YoutubeDNN`
RecSys'16
Google | [PNN](https://arxiv.org/pdf/1611.00144.pdf) {cite}`PNN`
ICDM'16 | [Wide&Deep](https://arxiv.org/pdf/1606.07792.pdf) {cite}`WideDeep`
DLRS'16
Google | +| | [DeepCrossing](https://www.kdd.org/kdd2016/papers/files/adf0975-shanA.pdf) {cite}`DeepCrossing`
KDD'16
Microsoft | [HOFM](https://arxiv.org/abs/1607.07195) {cite}`HOFM`
NIPS'16 | [DeepCTR](https://arxiv.org/abs/1609.06018) {cite}`DeepCTR`
MM'16 | [CCPM](https://arxiv.org/abs/1609.06018) {cite}`CCPM`
CIKM'15 | +| | [LR+GBDT](https://arxiv.org/abs/1609.06018) {cite}`LR_GBDT`
ADKDD'14
Facebook | [FTRL](https://research.google.com/pubs/archive/41159.pdf) {cite}`FTRL`
KDD'13
Google | [FM](https://www.csie.ntu.edu.tw/~b97053/paper/Rendle2010FM.pdf) {cite}`FM`
ICDM'10 | [LR](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/predictingclicks.pdf) {cite}`LR`
WWW'07
Microsoft | -````{admonition} Feature Interaction -```{table} -:align: left -:width: 94% -| | | | | | | -| :---------:|:------:|:------:|:------:|:------:|:------:| -| SIGIR'23 | [FinalNet](https://dl.acm.org/doi/10.1145/3539618.3591988) {cite}`FinalNet`
Huawei | AAAI'23 | [FinalMLP](https://arxiv.org/abs/2304.00902) {cite}`FinalMLP`
Huawei | SIGIR'23 | [EulerNet](https://arxiv.org/abs/2304.10711) {cite}`EulerNet`
Huawei | -| CIKM'23 | [GDCN](https://arxiv.org/abs/2311.04635) {cite}`GDCN`
Microsoft | CIKM'23 | [MemoNet](https://arxiv.org/abs/2211.01334) {cite}`MemoNet`
Sina Weibo | +## Behaviour Sequence Modeling -``` -```` -````{admonition} Behaviour Sequence Modeling -:class: tip -```{table} -:align: left -:width: 94% -| | | | | | | -| :---------:|:------:|:------:|:------:|:------:|:------:| -| KDD'23 | [TWIN](https://arxiv.org/abs/2302.02352) {cite}`TWIN`
Kuaishou | CIKM'23 | [DCIN](https://arxiv.org/pdf/2308.06037.pdf) {cite}`DCIN`
Meituan | -``` -```` -````{admonition} Multi-Domain Learning -:class: important -```{table} -:align: left -:width: 94% -| | | | | | | -| :---------:|:------:|:------:|:------:|:------:|:------:| -| KDD'23 | [SATrans](https://dl.acm.org/doi/10.1145/3580305.3599936) {cite}`SATrans`
Tencent | -``` -```` -````{admonition} Pretraining -:class: warning -```{table} -:align: left -:width: 94% | | | | | | | | :---------:|:------:|:------:|:------:|:------:|:------:| -| KDD'23 | [MAP](https://arxiv.org/abs/2308.01737) {cite}`MAP`
Huawei | KDD'23 | [BERT4CTR](https://arxiv.org/abs/2308.11527) {cite}`BERT4CTR`
Microsoft | -``` -```` +| **2023** | [TWIN](https://arxiv.org/abs/2302.02352) {cite}`TWIN`
KDD'23
Kuaishou | [DCIN](https://arxiv.org/pdf/2308.06037.pdf) {cite}`DCIN`
CIKM'23
Meituan | | | +| **2022** | [SDIM](https://arxiv.org/abs/2205.10249) {cite}`SDIM`
CIKM'22
Meituan | [DINMP](https://arxiv.org/abs/2104.06312) {cite}`DINMP`
SDM'22
Alibaba | +| **2021** | [CIN](https://dl.acm.org/doi/fullHtml/10.1145/3428079) {cite}`CIN`
TKDD'21 | [HyperCTR](https://arxiv.org/pdf/2109.02398) {cite}`HyperCTR`
CIKM'21 | +| **2020** | [DMIN](https://www.researchgate.net/profile/Luwei-Yang-2/publication/345125472_Deep_Multi-Interest_Network_for_Click-through_Rate_Prediction/links/5f9e1d6b458515b7cfaeffce/Deep-Multi-Interest-Network-for-Click-through-Rate-Prediction.pdf) {cite}`DMIN`
CIKM'20
Alibaba | [MARN](https://arxiv.org/abs/2003.07162) {cite}`MARN`
WWW'20
Alibaba | +| **2019** | [DIEN](https://arxiv.org/abs/1809.03672) {cite}`DIEN`
AAAI'19
Alibaba | [DSIN](https://arxiv.org/abs/1905.06482) {cite}`DSIN`
IJCAI'19
Alibaba | [DSTN](https://arxiv.org/abs/1906.03776) {cite}`DSTN`
KDD'19
Alibaba | [MIMN](https://arxiv.org/abs/1905.09248) {cite}`MIMN`
KDD'19
Alibaba | +| | [BST](https://arxiv.org/abs/1905.06874) {cite}`BST`
DLP-KDD'19
Alibaba | [GIN](https://arxiv.org/abs/2103.16164) {cite}`GIN`
SIGIR'19
Alibaba | +| **2018** | [DIN](https://www.kdd.org/kdd2018/accepted-papers/view/deep-interest-network-for-click-through-rate-prediction) {cite}`DIN`
KDD'18
Alibaba | -````` -`````{tab-item} 2022 +## Multi-Task & Multi-Domain -````{admonition} Feature Interaction -```{table} -:align: left -:width: 94% | | | | | | | | :---------:|:------:|:------:|:------:|:------:|:------:| -| SIGIR'22 | [FRNet](https://arxiv.org/abs/2204.08758) {cite}`FRNet`
Microsoft | NeurIPS'22 | [APG](https://arxiv.org/abs/2203.16218) {cite}`APG`
Alibaba | ICASSP'22 | [FINT](https://arxiv.org/abs/2107.01999) {cite}`FINT`
iQIYI | +| **2023** | [SATrans](https://dl.acm.org/doi/10.1145/3580305.3599936) {cite}`SATrans`
KDD'23
Tencent | +| **2021** | [STAR](https://arxiv.org/abs/2101.11427) {cite}`STAR`
CIKM'21
Alibaba | [DASL](https://arxiv.org/abs/2106.02768) {cite}`DASL`
KDD'21
Alibaba | +| **2019** | [DeepMCP](https://arxiv.org/abs/1906.04365) {cite}`DeepMCP`
IJCAI'19
Alibaba | -``` -```` -````{admonition} Behaviour Sequence Modeling -:class: tip -```{table} -:align: left -:width: 94% -| | | | | | | -| :---------:|:------:|:------:|:------:|:------:|:------:| -| CIKM'22 | [SDIM](https://arxiv.org/abs/2205.10249) {cite}`SDIM`
Meituan | SDM'22 | [DINMP](https://arxiv.org/abs/2104.06312) {cite}`DINMP`
Alibaba | -``` -```` +## Embedding Learning -````` -`````{tab-item} 2021 - -````{admonition} Feature Interaction -```{table} -:align: left -:width: 94% -| | | | | | | -| :---------:|:------:|:------:|:------:|:------:|:------:| -| WWW'21 | [DCN-V2](https://arxiv.org/abs/2008.13535) {cite}`DCNv2`
Google | WWW'21 | [FM2](https://arxiv.org/abs/2102.12994) {cite}`FM2`
Yahoo | CIKM'21 | [EDCN](https://dlp-kdd.github.io/assets/pdf/DLP-KDD_2021_paper_12.pdf) {cite}`EDCN`
Huawei | -| CIKM'21 | [DESTINE](https://arxiv.org/abs/2101.03654) {cite}`DESTINE`
Alibaba | SIGIR'21 | [SAM](https://arxiv.org/abs/2105.05563) {cite}`SAM`
BOSS Zhipin | SIGIR'21 | [PCF-GNN](https://arxiv.org/abs/2105.07752) {cite}`PCF-GNN`
Alibaba | -| SIGIR'21 | [xLightFM](https://dl.acm.org/doi/10.1145/3404835.3462941) {cite}`xLightFM` | KDD'21 | [AOANet](https://dl.acm.org/doi/10.1145/3447548.3467133) {cite}`AOANet`
Didi Chuxing | CIKM'21 | [DCAP](https://arxiv.org/abs/2105.08649) {cite}`DCAP` | | -``` -```` -````{admonition} Behaviour Sequence Modeling -:class: tip -```{table} -:align: left -:width: 94% -| | | | | | | -| :---------:|:------:|:------:|:------:|:------:|:------:| -| TKDD'21 | [CIN](https://dl.acm.org/doi/fullHtml/10.1145/3428079) {cite}`CIN` | CIKM'21 | [HyperCTR](https://arxiv.org/pdf/2109.02398) {cite}`HyperCTR` | -``` -```` -````{admonition} Multi-Domain/Multi-Task Learning -:class: important -```{table} -:align: left -:width: 94% | | | | | | | | :---------:|:------:|:------:|:------:|:------:|:------:| -| CIKM'21 | [STAR](https://arxiv.org/abs/2101.11427) {cite}`STAR`
Alibaba | KDD'21 | [DASL](https://arxiv.org/abs/2106.02768) {cite}`DASL`
Alibaba | CIKM'21 | [MetaCTR](https://dl.acm.org/doi/abs/10.1145/3459637.3481912) {cite}`MetaCTR`
Baidu | +| **2021** | [AutoDis](https://arxiv.org/abs/2012.08986) {cite}`AutoDis`
KDD'21
Huawei | [DG-ENN](https://arxiv.org/abs/2106.00314) {cite}`DG-ENN`
KDD'21
Huawei | [GME](https://arxiv.org/abs/2105.08909) {cite}`GME`
KDD'21
Alibaba | +| **2019** | [MetaEmbedding](https://arxiv.org/abs/1904.11547) {cite}`MetaEmbedding`
SIGIR'19 | -``` -```` -````{admonition} Embedding Learning -:class: warning -```{table} -:align: left -:width: 94% -| | | | | | | -| :---------:|:------:|:------:|:------:|:------:|:------:| -| KDD'21 | [AutoDis](https://arxiv.org/abs/2012.08986) {cite}`AutoDis`
Huawei | KDD'21 | [DG-ENN](https://arxiv.org/abs/2106.00314) {cite}`DG-ENN`
Huawei | KDD'21 | [GME](https://arxiv.org/abs/2105.08909) {cite}`GME`
Alibaba | -``` -```` -````` - -`````{tab-item} 2020 +## Pretraining -````{admonition} Feature Interaction -```{table} -:align: left -:width: 94% | | | | | | | | :---------:|:------:|:------:|:------:|:------:|:------:| -| AAAI'20 | [AFN](https://ojs.aaai.org/index.php/AAAI/article/view/5768) {cite}`AFN` | CIKM'20 | [DeepIM](https://dl.acm.org/doi/10.1145/3340531.3412077) {cite}`DeepIM`
Alibaba | SIGIR'20 | [AutoGroup](https://dl.acm.org/doi/abs/10.1145/3397271.3401082) {cite}`AutoGroup`
Huawei | -| NeurIPS'20 | [FWL](https://arxiv.org/abs/2012.00202) {cite}`FWL` | NeuralNet'20 | [ONN](https://arxiv.org/pdf/1904.12579) {cite}`ONN` | IJCAI'20 | [DIFM](https://www.ijcai.org/Proceedings/2020/0434.pdf) {cite}`DIFM` | -| KDD'20 | [AutoFIS](https://arxiv.org/abs/2003.11235) {cite}`AutoFIS`
Huawei | KDD'20 | [AutoCTR](https://arxiv.org/abs/2007.06434) {cite}`AutoCTR`
Facebook | ICLR'20 |[GLIDER](https://arxiv.org/abs/2006.10966) {cite}`GLIDER`
Facebook | +| **2023** | [MAP](https://arxiv.org/abs/2308.01737) {cite}`MAP`
KDD'23
Huawei | [BERT4CTR](https://arxiv.org/abs/2308.11527) {cite}`BERT4CTR`
KDD'23
Microsoft | -``` -```` -````{admonition} Behaviour Sequence Modeling -:class: tip -```{table} -:align: left -:width: 94% -| | | | | | | -| :---------:|:------:|:------:|:------:|:------:|:------:| -| CIKM'20 | [DMIN](https://www.researchgate.net/profile/Luwei-Yang-2/publication/345125472_Deep_Multi-Interest_Network_for_Click-through_Rate_Prediction/links/5f9e1d6b458515b7cfaeffce/Deep-Multi-Interest-Network-for-Click-through-Rate-Prediction.pdf) {cite}`DMIN`
Alibaba | WWW'20 | [MARN](https://arxiv.org/abs/2003.07162) {cite}`MARN`
Alibaba | -``` -```` -````` -`````{tab-item} 2019 - -````{admonition} Feature Interaction -```{table} -:align: left -:width: 94% -| | | | | | | -| :---------:|:------:|:------:|:------:|:------:|:------:| -| CIKM'19 | [AutoInt](https://arxiv.org/abs/1810.11921) {cite}`AutoInt` | CIKM'19 | [FiGNN](https://arxiv.org/abs/1910.05552) {cite}`FiGNN` | WWW'19 | [FGCNN](https://arxiv.org/abs/1904.04447) {cite}`FGCNN`
Huawei | -| RecSys'19 | [FiBiNET](https://arxiv.org/abs/1905.09433) {cite}`FiBiNET`
Sina Weibo | AAAI'19 | [HFM](https://ojs.aaai.org//index.php/AAAI/article/view/4448) {cite}`HFM` | Arxiv'19 | [DLRM](https://arxiv.org/abs/1906.00091) {cite}`DLRM`
Facebook | -| IJCAI'19 | [IFM](https://www.ijcai.org/proceedings/2019/203) {cite}`IFM` | -``` -```` -````{admonition} Behaviour Sequence Modeling -:class: tip -```{table} -:align: left -:width: 94% -| | | | | | | -| :---------:|:------:|:------:|:------:|:------:|:------:| -| IJCAI'19 | [DSIN](https://arxiv.org/abs/1905.06482) {cite}`DSIN`
Alibaba | AAAI'19 | [DIEN](https://arxiv.org/abs/1809.03672) {cite}`DIEN`
Alibaba | KDD'19 | [DSTN](https://arxiv.org/abs/1906.03776) {cite}`DSTN`
Alibaba | -| KDD'19 | [MIMN](https://arxiv.org/abs/1905.09248) {cite}`MIMN`
Alibaba | DLP-KDD'19 | [BST](https://arxiv.org/abs/1905.06874) {cite}`BST`
Alibaba | SIGIR'19 | [GIN](https://arxiv.org/abs/2103.16164) {cite}`GIN`
Alibaba | -``` -```` -````{admonition} Multi-Task Learning -:class: important -```{table} -:align: left -:width: 94% -| | | | | | | -| :---------:|:------:|:------:|:------:|:------:|:------:| -| IJCAI'19 | [DeepMCP](https://arxiv.org/abs/1906.04365) {cite}`DeepMCP`
Alibaba | SIGIR'19 | [MetaEmbedding](https://arxiv.org/abs/1904.11547) {cite}`MetaEmbedding` | -``` -```` -````` -`````{tab-item} 2018 -````{admonition} Feature Interaction -```{table} -:align: left -:width: 94% -| | | | | | | -| :---------:|:------:|:------:|:------:|:------:|:------:| -| WWW'18 | [FwFM](https://dl.acm.org/doi/10.1145/3178876.3186040) {cite}`FwFM`
Yahoo | KDD'18 | [xDeepFM](https://arxiv.org/pdf/1803.05170.pdf) {cite}`xDeepFM`
Microsoft | -``` -```` - -````{admonition} Behaviour Sequence Modeling -:class: tip -```{table} -:align: left -:width: 94% -| | | | | | | -| :---------:|:------:|:------:|:------:|:------:|:------:| -| KDD'18 | [DIN](https://www.kdd.org/kdd2018/accepted-papers/view/deep-interest-network-for-click-through-rate-prediction) {cite}`DIN`
Alibaba | | | -``` -```` - -````` -`````{tab-item} 2017 -````{admonition} Feature Interaction -```{table} -:align: left -:width: 94% -| | | | | | | -| :---------:|:------:|:------:|:------:|:------:|:------:| -| SIGIR'17 | [NFM](https://arxiv.org/abs/1708.05027) {cite}`NFM` | WWW'17 | [FFM](https://arxiv.org/pdf/1701.04099.pdf) {cite}`FFM2`
Criteo | ADKDD'17 | [DCN](https://arxiv.org/abs/1708.05123) {cite}`DCN`
Google | -| IJCAI'17 | [DeepFM](https://arxiv.org/abs/1703.04247) {cite}`DeepFM`
Huawei | IJCAI'17 | [AFM](https://www.ijcai.org/proceedings/2017/0435.pdf) {cite}`AFM` | - - -``` -```` -````` -`````{tab-item} 2016&Before - -````{admonition} Feature Interaction -```{table} -:align: left -:width: 94% -| | | | | -| :---------:|:------:|:------:|:------:| -| RecSys'16 | [FFM](https://www.csie.ntu.edu.tw/~cjlin/papers/ffm.pdf) {cite}`FFM` | RecSys'16 | [YoutubeDNN](https://research.google.com/pubs/archive/45530.pdf) {cite}`YoutubeDNN`
Google | -| ICDM'16| [PNN](https://arxiv.org/pdf/1611.00144.pdf) {cite}`PNN` | DLRS'16 |[Wide&Deep](https://arxiv.org/pdf/1606.07792.pdf) {cite}`WideDeep`
Google | -| KDD'16 | [DeepCrossing](https://www.kdd.org/kdd2016/papers/files/adf0975-shanA.pdf) {cite}`DeepCrossing`
Microsoft | NIPS'16 | [HOFM](https://arxiv.org/abs/1607.07195) {cite}`HOFM` | -|MM'16 | [DeepCTR](https://arxiv.org/abs/1609.06018) {cite}`DeepCTR` | CIKM'15 | [CCPM](https://arxiv.org/abs/1609.06018) {cite}`CCPM` | -| ADKDD'14 | [LR+GBDT](https://arxiv.org/abs/1609.06018) {cite}`LR_GBDT`
Facebook |KDD'13 | [FTRL](https://research.google.com/pubs/archive/41159.pdf) {cite}`FTRL`
Google | -|ICDM'10 | [FM](https://www.csie.ntu.edu.tw/~b97053/paper/Rendle2010FM.pdf) {cite}`FM` | WWW'07 |[LR](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/predictingclicks.pdf) {cite}`LR`
Microsoft | - -``` -```` -````` -`````` -### Paper List +## References ```{bibliography} :style: unsrt diff --git a/docs/CTR/references.bib b/docs/CTR/references.bib new file mode 100644 index 00000000..82e5f8c9 --- /dev/null +++ b/docs/CTR/references.bib @@ -0,0 +1,1124 @@ +--- +--- + +@inproceedings{AOANet, + author = {Lang Lang and + Zhenlong Zhu and + Xuanye Liu and + Jianxin Zhao and + Jixing Xu and + Minghui Shan}, + title = {{Architecture and Operation Adaptive Network for Online Recommendations}}, + booktitle = {The 27th {ACM} {SIGKDD} Conference on Knowledge Discovery + and Data Mining ({KDD} '21)}, + pages = {3139--3149}, + year = {2021}, +} + +@inproceedings{xLightFM, + author = {Gangwei Jiang and + Hao Wang and + Jin Chen and + Haoyu Wang and + Defu Lian and + Enhong Chen}, + title = {{xLightFM: Extremely Memory-Efficient Factorization Machine}}, + booktitle = {The 44th International {ACM} {SIGIR} Conference on Research + and Development in Information Retrieval ({SIGIR} '21)}, + pages = {337--346}, + year = {2021}, +} + +@inproceedings{DMIN, + author = {Zhibo Xiao and + Luwei Yang and + Wen Jiang and + Yi Wei and + Yi Hu and + Hao Wang}, + title = {{Deep Multi-Interest Network for Click-through Rate Prediction}}, + booktitle = {The 29th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '20)}, + pages = {2265--2268}, + year = {2020}, +} + + +@article{DLRM, + author = {Maxim Naumov and + Dheevatsa Mudigere and + Hao{-}Jun Michael Shi and + Jianyu Huang and + Narayanan Sundaraman and + Jongsoo Park and + Xiaodong Wang and + Udit Gupta and + Carole{-}Jean Wu and + Alisson G. Azzolini and + Dmytro Dzhulgakov and + Andrey Mallevich and + Ilia Cherniavskii and + Yinghai Lu and + Raghuraman Krishnamoorthi and + Ansha Yu and + Volodymyr Kondratenko and + Stephanie Pereira and + Xianjie Chen and + Wenlin Chen and + Vijay Rao and + Bill Jia and + Liang Xiong and + Misha Smelyanskiy}, + title = {{Deep Learning Recommendation Model for Personalization and Recommendation + Systems}}, + journal = {CoRR}, + volume = {abs/1906.00091}, + year = {2019}, +} + +@inproceedings{FiBiNET, + author = {Tongwen Huang and + Zhiqi Zhang and + Junlin Zhang}, + title = {{FiBiNET: combining feature importance and bilinear feature interaction + for click-through rate prediction}}, + booktitle = {Proceedings of the 13th {ACM} Conference on Recommender Systems (RecSys '19)}, + pages = {169--177}, + year = {2019}, +} + +@article{ONN, + author = {Yi Yang and + Baile Xu and + Shaofeng Shen and + Furao Shen and + Jian Zhao}, + title = {{Operation-aware Neural Networks for user response prediction}}, + journal = {Neural Networks}, + volume = {121}, + pages = {161--168}, + year = {2020}, +} + +@inproceedings{DSIN, + author = {Yufei Feng and + Fuyu Lv and + Weichen Shen and + Menghan Wang and + Fei Sun and + Yu Zhu and + Keping Yang}, + title = {{Deep Session Interest Network for Click-Through Rate Prediction}}, + booktitle = {Proceedings of the Twenty-Eighth International Joint Conference on + Artificial Intelligence ({IJCAI} '19)}, + pages = {2301--2307}, + year = {2019}, +} + +@inproceedings{DSTN, + author = {Wentao Ouyang and + Xiuwu Zhang and + Li Li and + Heng Zou and + Xin Xing and + Zhaojie Liu and + Yanlong Du}, + title = {{Deep Spatio-Temporal Neural Networks for Click-Through Rate Prediction}}, + booktitle = {Proceedings of the 25th {ACM} {SIGKDD} International Conference on + Knowledge Discovery {\&} Data Mining (KDD '19)}, + pages = {2078--2086}, + year = {2019}, +} + +@inproceedings{DeepMCP, + author = {Wentao Ouyang and + Xiuwu Zhang and + Shukui Ren and + Chao Qi and + Zhaojie Liu and + Yanlong Du}, + title = {{Representation Learning-Assisted Click-Through Rate Prediction}}, + booktitle = {Proceedings of the Twenty-Eighth International Joint Conference on + Artificial Intelligence ({IJCAI} '19)}, + pages = {4561--4567}, + year = {2019}, +} + +@inproceedings{DeepFM, + author = {Huifeng Guo and + Ruiming Tang and + Yunming Ye and + Zhenguo Li and + Xiuqiang He}, + title = {{DeepFM: {A} Factorization-Machine based Neural Network for {CTR} Prediction}}, + booktitle = {Proceedings of the Twenty-Sixth International Joint Conference on + Artificial Intelligence ({IJCAI} '17)}, + pages = {1725--1731}, + year = {2017}, +} + +@inproceedings{DCAP, + author = {Zekai Chen and + Fangtian Zhong and + Zhumin Chen and + Xiao Zhang and + Robert Pless and + Xiuzhen Cheng}, + title = {{{DCAP:} Deep Cross Attentional Product Network for User Response Prediction}}, + booktitle = {The 30th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '21)}, + pages = {221--230}, + year = {2021}, +} + +@inproceedings{GIN, + author = {Feng Li and + Zhenrui Chen and + Pengjie Wang and + Yi Ren and + Di Zhang and + Xiaoyu Zhu}, + title = {{Graph Intention Network for Click-through Rate Prediction in Sponsored + Search}}, + booktitle = {Proceedings of the 42nd International {ACM} {SIGIR} Conference on + Research and Development in Information Retrieval ({SIGIR} '19)}, + pages = {961--964}, + year = {2019}, +} + +@inproceedings{MetaEmbedding, + author = {Feiyang Pan and + Shuokai Li and + Xiang Ao and + Pingzhong Tang and + Qing He}, + title = {{Warm Up Cold-start Advertisements: Improving {CTR} Predictions via + Learning to Learn {ID} Embeddings}}, + booktitle = {Proceedings of the 42nd International {ACM} {SIGIR} Conference on + Research and Development in Information Retrieval ({SIGIR} '19)}, + pages = {695--704}, + year = {2019}, +} + +@inproceedings{FGCNN, + author = {Bin Liu and + Ruiming Tang and + Yingzhi Chen and + Jinkai Yu and + Huifeng Guo and + Yuzhou Zhang}, + title = {{Feature Generation by Convolutional Neural Network for Click-Through + Rate Prediction}}, + booktitle = {The World Wide Web Conference ({WWW} '19)}, + pages = {1119--1129}, + year = {2019}, +} + +@inproceedings{HFM, + author = {Yi Tay and + Shuai Zhang and + Anh Tuan Luu and + Siu Cheung Hui and + Lina Yao and + Tran Dang Quang Vinh}, + title = {{Holographic Factorization Machines for Recommendation}}, + booktitle = {The Thirty-Third {AAAI} Conference on Artificial Intelligence ({AAAI} '19)}, + pages = {5143--5150}, + year = {2019}, +} + +@inproceedings{AutoInt, + author = {Weiping Song and + Chence Shi and + Zhiping Xiao and + Zhijian Duan and + Yewen Xu and + Ming Zhang and + Jian Tang}, + title = {{AutoInt: Automatic Feature Interaction Learning via Self-Attentive + Neural Networks}}, + booktitle = {Proceedings of the 28th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '19)}, + pages = {1161--1170}, + year = {2019}, +} + +@inproceedings{FwFM, + author = {Junwei Pan and + Jian Xu and + Alfonso Lobos Ruiz and + Wenliang Zhao and + Shengjun Pan and + Yu Sun and + Quan Lu}, + title = {{Field-weighted Factorization Machines for Click-Through Rate Prediction + in Display Advertising}}, + booktitle = {Proceedings of the 2018 World Wide Web Conference on World Wide Web ({WWW} '18)}, + pages = {1349--1357}, + year = {2018}, +} + +@inproceedings{AFM, + author = {Jun Xiao and + Hao Ye and + Xiangnan He and + Hanwang Zhang and + Fei Wu and + Tat{-}Seng Chua}, + title = {{Attentional Factorization Machines: Learning the Weight of Feature + Interactions via Attention Networks}}, + booktitle = {Proceedings of the Twenty-Sixth International Joint Conference on + Artificial Intelligence ({IJCAI} '17)}, + pages = {3119--3125}, + year = {2017}, +} + +@inproceedings{NFM, + author = {Xiangnan He and + Tat{-}Seng Chua}, + title = {{Neural Factorization Machines for Sparse Predictive Analytics}}, + booktitle = {Proceedings of the 40th International {ACM} {SIGIR} Conference on + Research and Development in Information Retrieval (SIGIR '17)}, + pages = {355--364}, + year = {2017}, +} + +@inproceedings{DCN, + author = {Ruoxi Wang and + Bin Fu and + Gang Fu and + Mingliang Wang}, + title = {{Deep {\&} Cross Network for Ad Click Predictions}}, + booktitle = {Proceedings of the ADKDD '17}, + pages = {12:1--12:7}, + year = {2017}, +} + +@inproceedings{FFM2, + author = {Yuchin Juan and + Damien Lefortier and + Olivier Chapelle}, + title = {{Field-aware Factorization Machines in a Real-world Online Advertising + System}}, + booktitle = {Proceedings of the 26th International Conference on World Wide Web + Companion (WWW '17)}, + pages = {680--688}, + year = {2017}, +} + +@inproceedings{FFM, + author = {Yu{-}Chin Juan and + Yong Zhuang and + Wei{-}Sheng Chin and + Chih{-}Jen Lin}, + title = {{Field-aware Factorization Machines for {CTR} Prediction}}, + booktitle = {Proceedings of the 10th {ACM} Conference on Recommender Systems (RecSys '16)}, + pages = {43--50}, + year = {2016}, +} + +@inproceedings{YoutubeDNN, + author = {Paul Covington and + Jay Adams and + Emre Sargin}, + title = {{Deep Neural Networks for YouTube Recommendations}}, + booktitle = {Proceedings of the 10th {ACM} Conference on Recommender Systems (RecSys '16)}, + pages = {191--198}, + year = {2016}, +} + +@inproceedings{WideDeep, + author = {Heng{-}Tze Cheng and + Levent Koc and + Jeremiah Harmsen and + Tal Shaked and + Tushar Chandra and + Hrishi Aradhye and + Glen Anderson and + Greg Corrado and + Wei Chai and + Mustafa Ispir and + Rohan Anil and + Zakaria Haque and + Lichan Hong and + Vihan Jain and + Xiaobing Liu and + Hemal Shah}, + title = {{Wide {\&} Deep Learning for Recommender Systems}}, + booktitle = {Proceedings of the 1st Workshop on Deep Learning for Recommender Systems (DLRS@RecSys '16)}, + pages = {7--10}, + year = {2016}, +} + +@inproceedings{PNN, + author = {Yanru Qu and + Han Cai and + Kan Ren and + Weinan Zhang and + Yong Yu and + Ying Wen and + Jun Wang}, + title = {{Product-Based Neural Networks for User Response Prediction}}, + booktitle = {{IEEE} 16th International Conference on Data Mining ({ICDM} '16)}, + pages = {1149--1154}, + year = {2016}, +} + +@inproceedings{DeepCTR, + author = {Junxuan Chen and + Baigui Sun and + Hao Li and + Hongtao Lu and + Xian{-}Sheng Hua}, + title = {{Deep {CTR} Prediction in Display Advertising}}, + booktitle = {Proceedings of the 2016 {ACM} Conference on Multimedia Conference ({MM} '16)}, + pages = {811--820}, + year = {2016}, +} + +@inproceedings{CCPM, + author = {Qiang Liu and + Feng Yu and + Shu Wu and + Liang Wang}, + title = {{A Convolutional Click Prediction Model}}, + booktitle = {Proceedings of the 24th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '15)}, + pages = {1743--1746}, + year = {2015}, +} + +@inproceedings{LR_GBDT, + author = {Xinran He and + Junfeng Pan and + Ou Jin and + Tianbing Xu and + Bo Liu and + Tao Xu and + Yanxin Shi and + Antoine Atallah and + Ralf Herbrich and + Stuart Bowers and + Joaquin Qui{\~{n}}onero Candela}, + title = {{Practical Lessons from Predicting Clicks on Ads at Facebook}}, + booktitle = {Proceedings of the Eighth International Workshop on Data Mining for + Online Advertising ({ADKDD} '14)}, + pages = {5:1--5:9}, + year = {2014}, +} + +@inproceedings{FTRL, + author = {H. Brendan McMahan and + Gary Holt and + David Sculley and + Michael Young and + Dietmar Ebner and + Julian Grady and + Lan Nie and + Todd Phillips and + Eugene Davydov and + Daniel Golovin and + Sharat Chikkerur and + Dan Liu and + Martin Wattenberg and + Arnar Mar Hrafnkelsson and + Tom Boulos and + Jeremy Kubica}, + title = {{Ad Click Prediction: A View from the Trenches}}, + booktitle = {The 19th {ACM} {SIGKDD} International Conference on Knowledge Discovery + and Data Mining ({KDD} '13)}, + pages = {1222--1230}, + year = {2013}, +} + +@inproceedings{FM, + author = {Steffen Rendle}, + title = {{Factorization Machines}}, + booktitle = {The 10th {IEEE} International Conference on Data Mining (ICDM '10)}, + pages = {995--1000}, + year = {2010}, +} + +@inproceedings{LR, + author = {Matthew Richardson and + Ewa Dominowska and + Robert Ragno}, + title = {{Predicting Clicks: Estimating the Click-through Rate for New Ads}}, + booktitle = {Proceedings of the 16th International Conference on World Wide Web ({WWW} '07)}, + pages = {521--530}, + year = {2007}, +} + +@inproceedings{DIN, + author = {Guorui Zhou and + Xiaoqiang Zhu and + Chengru Song and + Ying Fan and + Han Zhu and + Xiao Ma and + Yanghui Yan and + Junqi Jin and + Han Li and + Kun Gai}, + title = {{Deep Interest Network for Click-Through Rate Prediction}}, + booktitle = {Proceedings of the 24th {ACM} {SIGKDD} International Conference on + Knowledge Discovery {\&} Data Mining ({KDD} '18)}, + pages = {1059--1068}, + year = {2018}, +} + +@inproceedings{xDeepFM, + author = {Jianxun Lian and + Xiaohuan Zhou and + Fuzheng Zhang and + Zhongxia Chen and + Xing Xie and + Guangzhong Sun}, + title = {{xDeepFM: Combining Explicit and Implicit Feature Interactions for + Recommender Systems}}, + booktitle = {Proceedings of the 24th {ACM} {SIGKDD} International Conference on + Knowledge Discovery {\&} Data Mining ({KDD} '18)}, + pages = {1754--1763}, + year = {2018}, +} + +@inproceedings{FiGNN, + author = {Zekun Li and + Zeyu Cui and + Shu Wu and + Xiaoyu Zhang and + Liang Wang}, + title = {{Fi-GNN: Modeling Feature Interactions via Graph Neural Networks for + {CTR} Prediction}}, + booktitle = {Proceedings of the 28th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '19)}, + pages = {539--548}, + year = {2019}, +} + +@inproceedings{AFN, + author = {Weiyu Cheng and + Yanyan Shen and + Linpeng Huang}, + title = {{Adaptive Factorization Network: Learning Adaptive-Order Feature Interactions}}, + booktitle = {The Thirty-Fourth {AAAI} Conference on Artificial Intelligence ({AAAI} '20)}, + pages = {3609--3616}, + year = {2020}, +} + +@inproceedings{DeepIM, + author = {Feng Yu and + Zhaocheng Liu and + Qiang Liu and + Haoli Zhang and + Shu Wu and + Liang Wang}, + title = {{Deep Interaction Machine: {A} Simple but Effective Model for High-order + Feature Interactions}}, + booktitle = {The 29th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '20)}, + pages = {2285--2288}, + year = {2020}, +} + +@inproceedings{AutoGroup, + author = {Bin Liu and + Niannan Xue and + Huifeng Guo and + Ruiming Tang and + Stefanos Zafeiriou and + Xiuqiang He and + Zhenguo Li}, + title = {{AutoGroup: Automatic Feature Grouping for Modelling Explicit High-Order + Feature Interactions in {CTR} Prediction}}, + booktitle = {Proceedings of the 43rd International {ACM} {SIGIR} conference on + research and development in Information Retrieval ({SIGIR} '20)}, + pages = {199--208}, + year = {2020}, +} + +@inproceedings{FWL, + author = {Zhibin Li and + Jian Zhang and + Yongshun Gong and + Yazhou Yao and + Qiang Wu}, + title = {{Field-wise Learning for Multi-field Categorical Data}}, + booktitle = {Annual Conference on Neural Information Processing Systems (NeurIPS '20)}, + year = {2020}, +} + +@inproceedings{DIFM, + author = {Wantong Lu and + Yantao Yu and + Yongzhe Chang and + Zhen Wang and + Chenhui Li and + Bo Yuan}, + title = {{A Dual Input-aware Factorization Machine for {CTR} Prediction}}, + booktitle = {Proceedings of the Twenty-Ninth International Joint Conference on + Artificial Intelligence ({IJCAI} '20)}, + pages = {3139--3145}, + year = {2020}, +} + +@inproceedings{AutoFIS, + author = {Bin Liu and + Chenxu Zhu and + Guilin Li and + Weinan Zhang and + Jincai Lai and + Ruiming Tang and + Xiuqiang He and + Zhenguo Li and + Yong Yu}, + title = {{AutoFIS: Automatic Feature Interaction Selection in Factorization + Models for Click-Through Rate Prediction}}, + booktitle = {The 26th {ACM} {SIGKDD} Conference on Knowledge Discovery + and Data Mining ({KDD} '20)}, + pages = {2636--2645}, + year = {2020}, +} + +@inproceedings{AutoCTR, + author = {Qingquan Song and + Dehua Cheng and + Hanning Zhou and + Jiyan Yang and + Yuandong Tian and + Xia Hu}, + title = {{Towards Automated Neural Interaction Discovery for Click-Through Rate + Prediction}}, + booktitle = {The 26th {ACM} {SIGKDD} Conference on Knowledge Discovery + and Data Mining ({KDD} '20)}, + pages = {945--955}, + year = {2020}, +} + +@inproceedings{MARN, + author = {Xiang Li and + Chao Wang and + Jiwei Tan and + Xiaoyi Zeng and + Dan Ou and + Bo Zheng}, + title = {{Adversarial Multimodal Representation Learning for Click-Through Rate + Prediction}}, + booktitle = {The Web Conference ({WWW} '20)}, + pages = {827--836}, + year = {2020}, +} + +@inproceedings{GLIDER, + author = {Michael Tsang and + Dehua Cheng and + Hanpeng Liu and + Xue Feng and + Eric Zhou and + Yan Liu}, + title = {{Feature Interaction Interpretability: {A} Case for Explaining Ad-Recommendation + Systems via Neural Interaction Detection}}, + booktitle = {The 8th International Conference on Learning Representations ({ICLR} '20)}, + year = {2020}, +} + +@inproceedings{DIEN, + author = {Guorui Zhou and + Na Mou and + Ying Fan and + Qi Pi and + Weijie Bian and + Chang Zhou and + Xiaoqiang Zhu and + Kun Gai}, + title = {{Deep Interest Evolution Network for Click-Through Rate Prediction}}, + booktitle = {The Thirty-Third {AAAI} Conference on Artificial Intelligence ({AAAI} '19)}, + pages = {5941--5948}, + year = {2019}, +} + +@inproceedings{MIMN, + author = {Qi Pi and + Weijie Bian and + Guorui Zhou and + Xiaoqiang Zhu and + Kun Gai}, + title = {{Practice on Long Sequential User Behavior Modeling for Click-Through + Rate Prediction}}, + booktitle = {Proceedings of the 25th {ACM} {SIGKDD} International Conference on + Knowledge Discovery {\&} Data Mining ({KDD} '19)}, + pages = {2671--2679}, + year = {2019}, +} + +@article{BST, + author = {Qiwei Chen and + Huan Zhao and + Wei Li and + Pipei Huang and + Wenwu Ou}, + title = {{Behavior Sequence Transformer for E-commerce Recommendation in Alibaba}}, + journal = {CoRR}, + volume = {abs/1905.06874}, + year = {2019}, +} + +@inproceedings{IFM, + author = {Yantao Yu and + Zhen Wang and + Bo Yuan}, + title = {{An Input-aware Factorization Machine for Sparse Prediction}}, + booktitle = {Proceedings of the Twenty-Eighth International Joint Conference on + Artificial Intelligence ({IJCAI} '19)}, + pages = {1466--1472}, + year = {2019}, +} + +@inproceedings{DeepCrossing, + author = {Ying Shan and + T. Ryan Hoens and + Jian Jiao and + Haijing Wang and + Dong Yu and + J. C. Mao}, + title = {{Deep Crossing: Web-Scale Modeling without Manually Crafted Combinatorial + Features}}, + booktitle = {Proceedings of the 22nd {ACM} {SIGKDD} International Conference on + Knowledge Discovery and Data Mining (KDD '16)}, + pages = {255--262}, + year = {2016}, +} + +@inproceedings{HOFM, + author = {Mathieu Blondel and + Akinori Fujino and + Naonori Ueda and + Masakazu Ishihata}, + title = {{Higher-Order Factorization Machines}}, + booktitle = {Annual Conference on Neural Information Processing Systems (NeurIPS '16)}, + pages = {3351--3359}, + year = {2016}, +} + +@article{CIN, + author = {En Xu and + Zhiwen Yu and + Bin Guo and + Helei Cui}, + title = {{Core Interest Network for Click-Through Rate Prediction}}, + journal = {{ACM} Trans. Knowl. Discov. Data}, + volume = {15}, + number = {2}, + pages = {23:1--23:16}, + year = {2021}, +} + +@inproceedings{EDCN, + author = {Bo Chen and + Yichao Wang and + Zhirong Liu and + Ruiming Tang and + Wei Guo and + Hongkun Zheng and + Weiwei Yao and + Muyu Zhang and + Xiuqiang He}, + title = {{Enhancing Explicit and Implicit Feature Interactions via Information + Sharing for Parallel Deep {CTR} Models}}, + booktitle = {The 30th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '21)}, + pages = {3757--3766}, + year = {2021}, +} + +@inproceedings{HyperCTR, + author = {Li He and + Hongxu Chen and + Dingxian Wang and + Shoaib Jameel and + Philip S. Yu and + Guandong Xu}, + title = {{Click-Through Rate Prediction with Multi-Modal Hypergraphs}}, + booktitle = {The 30th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '21)}, + pages = {690--699}, + year = {2021}, +} + +@inproceedings{DESTINE, + author = {Yichen Xu and + Yanqiao Zhu and + Feng Yu and + Qiang Liu and + Shu Wu}, + title = {{Disentangled Self-Attentive Neural Networks for Click-Through Rate + Prediction}}, + booktitle = {The 30th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '21)}, + pages = {3553--3557}, + year = {2021}, +} + +@inproceedings{PCF-GNN, + author = {Feng Li and + Bencheng Yan and + Qingqing Long and + Pengjie Wang and + Wei Lin and + Jian Xu and + Bo Zheng}, + title = {{Explicit Semantic Cross Feature Learning via Pre-trained Graph Neural + Networks for {CTR} Prediction}}, + booktitle = {The 44th International {ACM} {SIGIR} Conference on Research + and Development in Information Retrieval ({SIGIR} '21)}, + pages = {2161--2165}, + year = {2021}, +} + +@inproceedings{SAM, + author = {Yuan Cheng and + Yanbo Xue}, + title = {{Looking at {CTR} Prediction Again: Is Attention All You Need?}}, + booktitle = {The 44th International {ACM} {SIGIR} Conference on Research + and Development in Information Retrieval ({SIGIR} '21)}, + pages = {1279--1287}, + year = {2021}, +} + +@inproceedings{FM2, + author = {Yang Sun and + Junwei Pan and + Alex Zhang and + Aaron Flores}, + title = {{{FM2:} Field-matrixed Factorization Machines for Recommender Systems}}, + booktitle = {The Web Conference ({WWW} '21)}, + pages = {2828--2837}, + year = {2021}, +} + +@inproceedings{DCNv2, + author = {Ruoxi Wang and + Rakesh Shivanna and + Derek Zhiyuan Cheng and + Sagar Jain and + Dong Lin and + Lichan Hong and + Ed H. Chi}, + title = {{{DCN} {V2:} Improved Deep {\&} Cross Network and Practical Lessons + for Web-scale Learning to Rank Systems}}, + booktitle = {The Web Conference ({WWW} '21)}, + pages = {1785--1797}, + year = {2021}, +} + +@inproceedings{STAR, + author = {Xiang{-}Rong Sheng and + Liqin Zhao and + Guorui Zhou and + Xinyao Ding and + Binding Dai and + Qiang Luo and + Siran Yang and + Jingshan Lv and + Chi Zhang and + Hongbo Deng and + Xiaoqiang Zhu}, + title = {{One Model to Serve All: Star Topology Adaptive Recommender for Multi-Domain + {CTR} Prediction}}, + booktitle = {The 30th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '21)}, + pages = {4104--4113}, + year = {2021}, +} + +@inproceedings{DASL, + author = {Pan Li and + Zhichao Jiang and + Maofei Que and + Yao Hu and + Alexander Tuzhilin}, + title = {{Dual Attentive Sequential Learning for Cross-Domain Click-Through + Rate Prediction}}, + booktitle = {The 27th {ACM} {SIGKDD} Conference on Knowledge Discovery + and Data Mining ({KDD} '21)}, + pages = {3172--3180}, + year = {2021}, +} + +@inproceedings{MetaCTR, + author = {Xin Wang and + Peng Yang and + Shaopeng Chen and + Lin Liu and + Lian Zhao and + Jiacheng Guo and + Mingming Sun and + Ping Li}, + title = {{Efficient Learning to Learn a Robust {CTR} Model for Web-scale Online + Sponsored Search Advertising}}, + booktitle = {The 30th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '21)}, + pages = {4203--4213}, + year = {2021}, +} + +@inproceedings{GME, + author = {Wentao Ouyang and + Xiuwu Zhang and + Shukui Ren and + Li Li and + Kun Zhang and + Jinmei Luo and + Zhaojie Liu and + Yanlong Du}, + title = {{Learning Graph Meta Embeddings for Cold-Start Ads in Click-Through + Rate Prediction}}, + booktitle = {The 44th International {ACM} {SIGIR} Conference on Research + and Development in Information Retrieval ({SIGIR} '21)}, + pages = {1157--1166}, + year = {2021}, +} + +@inproceedings{AutoDis, + author = {Huifeng Guo and + Bo Chen and + Ruiming Tang and + Weinan Zhang and + Zhenguo Li and + Xiuqiang He}, + title = {{An Embedding Learning Framework for Numerical Features in {CTR} Prediction}}, + booktitle = {The 27th {ACM} {SIGKDD} Conference on Knowledge Discovery + and Data Mining ({KDD} '21)}, + pages = {2910--2918}, + year = {2021}, +} + +@inproceedings{DG-ENN, + author = {Wei Guo and + Rong Su and + Renhao Tan and + Huifeng Guo and + Yingxue Zhang and + Zhirong Liu and + Ruiming Tang and + Xiuqiang He}, + title = {{Dual Graph enhanced Embedding Neural Network for {CTR} Prediction}}, + booktitle = {The 27th {ACM} {SIGKDD} Conference on Knowledge Discovery + and Data Mining ({KDD} '21)}, + pages = {496--504}, + year = {2021}, +} + +@inproceedings{DINMP, + author = {Keke Zhao and + Xing Zhao and + Qi Cao and + Linjian Mo}, + title = {{A Non-sequential Approach to Deep User Interest Model for {CTR} Prediction}}, + booktitle = {Proceedings of the 2022 {SIAM} International Conference on Data Mining ({SDM} '22)}, + pages = {531--539}, + year = {2022}, +} + +@article{FINT, + author = {Zhishan Zhao and + Sen Yang and + Guohui Liu and + Dawei Feng and + Kele Xu}, + title = {{{FINT:} Field-aware INTeraction Neural Network For {CTR} Prediction}}, + journal = {CoRR}, + volume = {abs/2107.01999}, + year = {2021}, +} + +@inproceedings{SDIM, + author = {Yue Cao and + Xiaojiang Zhou and + Jiaqi Feng and + Peihao Huang and + Yao Xiao and + Dayao Chen and + Sheng Chen}, + title = {{Sampling Is All You Need on Modeling Long-Term User Behaviors for + {CTR} Prediction}}, + booktitle = {Proceedings of the 31st {ACM} International Conference on Information + {\&} Knowledge Management (CIKM '22)}, + pages = {2974--2983}, + year = {2022}, +} + +@inproceedings{APG, + author = {Bencheng Yan and + Pengjie Wang and + Kai Zhang and + Feng Li and + Hongbo Deng and + Jian Xu and + Bo Zheng}, + title = {{{APG:} Adaptive Parameter Generation Network for Click-Through Rate + Prediction}}, + booktitle = {NeurIPS}, + year = {2022}, +} + +@inproceedings{FinalMLP, + author = {Kelong Mao and + Jieming Zhu and + Liangcai Su and + Guohao Cai and + Yuru Li and + Zhenhua Dong}, + title = {{FinalMLP: An Enhanced Two-Stream {MLP} Model for {CTR} Prediction}}, + booktitle = {Thirty-Seventh {AAAI} Conference on Artificial Intelligence (AAAI '23)}, + pages = {4552--4560}, + year = {2023}, +} + +@inproceedings{FinalNet, + author = {Jieming Zhu and + Qinglin Jia and + Guohao Cai and + Quanyu Dai and + Jingjie Li and + Zhenhua Dong and + Ruiming Tang and + Rui Zhang}, + title = {{{FINAL:} Factorized Interaction Layer for {CTR} Prediction}}, + booktitle = {Proceedings of the 46th International {ACM} {SIGIR} Conference on + Research and Development in Information Retrieval ({SIGIR} '23)}, + pages = {2006--2010}, + year = {2023}, +} + +@inproceedings{GDCN, + author = {Fangye Wang and + Hansu Gu and + Dongsheng Li and + Tun Lu and + Peng Zhang and + Ning Gu}, + title = {{Towards Deeper, Lighter and Interpretable Cross Network for {CTR} + Prediction}}, + booktitle = {Proceedings of the 32nd {ACM} International Conference on Information + and Knowledge Management ({CIKM} '23)}, + pages = {2523--2533}, + year = {2023}, +} + +@inproceedings{DCIN, + author = {Xuyang Hou and + Zhe Wang and + Qi Liu and + Tan Qu and + Jia Cheng and + Jun Lei}, + title = {{Deep Context Interest Network for Click-Through Rate Prediction}}, + booktitle = {The {ACM} International Conference on Information + and Knowledge Management ({CIKM} '23)}, + year = {2023}, +} + +@inproceedings{FRNet, + author = {Fangye Wang and + Yingxu Wang and + Dongsheng Li and + Hansu Gu and + Tun Lu and + Peng Zhang and + Ning Gu}, + title = {{Enhancing {CTR} Prediction with Context-Aware Feature Representation + Learning}}, + booktitle = {The 45th International {ACM} {SIGIR} Conference on Research + and Development in Information Retrieval ({SIGIR} '22)}, + pages = {343--352}, + year = {2022}, +} + +@inproceedings{MemoNet, + author = {Pengtao Zhang and + Junlin Zhang}, + title = {{MemoNet: Memorizing All Cross Features' Representations Efficiently + via Multi-Hash Codebook Network for {CTR} Prediction}}, + booktitle = {Proceedings of the 32nd {ACM} International Conference on Information + and Knowledge Management ({CIKM} '23)}, + pages = {3154--3163}, + year = {2023}, +} + +@inproceedings{TWIN, + author = {Jianxin Chang and + Chenbin Zhang and + Zhiyi Fu and + Xiaoxue Zang and + Lin Guan and + Jing Lu and + Yiqun Hui and + Dewei Leng and + Yanan Niu and + Yang Song and + Kun Gai}, + title = {{{TWIN:} TWo-stage Interest Network for Lifelong User Behavior Modeling + in {CTR} Prediction at Kuaishou}}, + booktitle = {Proceedings of the 29th {ACM} {SIGKDD} Conference on Knowledge Discovery + and Data Mining ({KDD} '23)}, + pages = {3785--3794}, + year = {2023}, +} + +@inproceedings{MAP, + author = {Jianghao Lin and + Yanru Qu and + Wei Guo and + Xinyi Dai and + Ruiming Tang and + Yong Yu and + Weinan Zhang}, + title = {{{MAP:} {A} Model-agnostic Pretraining Framework for Click-through + Rate Prediction}}, + booktitle = {Proceedings of the 29th {ACM} {SIGKDD} Conference on Knowledge Discovery + and Data Mining ({KDD} '23)}, + pages = {1384--1395}, + year = {2023}, +} + +@inproceedings{EulerNet, + author = {Zhen Tian and + Ting Bai and + Wayne Xin Zhao and + Ji{-}Rong Wen and + Zhao Cao}, + title = {{EulerNet: Adaptive Feature Interaction Learning via Euler's Formula + for {CTR} Prediction}}, + booktitle = {Proceedings of the 46th International {ACM} {SIGIR} Conference on + Research and Development in Information Retrieval ({SIGIR} '23)}, + pages = {1376--1385}, + year = {2023}, +} + +@inproceedings{SATrans, + author = {Erxue Min and + Da Luo and + Kangyi Lin and + Chunzhen Huang and + Yang Liu}, + title = {{Scenario-Adaptive Feature Interaction for Click-Through Rate Prediction}}, + booktitle = {Proceedings of the 29th {ACM} {SIGKDD} Conference on Knowledge Discovery + and Data Mining ({KDD} '23)}, + pages = {4661--4672}, + year = {2023}, +} + +@inproceedings{BERT4CTR, + author = {Dong Wang and + Kav{\'{e}} Salamatian and + Yunqing Xia and + Weiwei Deng and + Qi Zhang}, + title = {{{BERT4CTR:} An Efficient Framework to Combine Pre-trained Language + Model with Non-textual Features for {CTR} Prediction}}, + booktitle = {Proceedings of the 29th {ACM} {SIGKDD} Conference on Knowledge Discovery + and Data Mining ({KDD} '23)}, + pages = {5039--5050}, + year = {2023}, +} diff --git a/docs/Matching/index.md b/docs/Matching/index.md new file mode 100644 index 00000000..1a8da37f --- /dev/null +++ b/docs/Matching/index.md @@ -0,0 +1,13 @@ +# BARS-Match Overview + +BARS-Match: An Open Benchmark for Candidate Item Matching https://openbenchmark.github.io/BARS/Matching + +Recommender systems generally comprise two main stages, matching and ranking. As the first-stage task, candidate item matching is designed to efficiently retrieve hundreds of item candidates out of the entire item corpus. Representative methods of candidate item matching include collaborative filtering, two-tower models, autoencoder-based models, sequential models, graph-based models, etc. To drive research in this direction, the BARS project aims to build an open benchmark for candidate item matching, which consists of: + ++ [A curated list of papers on candidate item matching](./papers.md), which have been tagged into different categories, such as CF, autoencoders, two-tower models, GNNs, and so on. + ++ [A collection of open datasets](https://github.com/reczoo/Datasets?tab=readme-ov-file#matching) for research on candidate item matching, and unique dataset IDs to track specific data splits for each dataset. + ++ [An open-source library for candidate item matching](https://github.com/reczoo/RecBox) with key features in configurability, tunability, and reproduciblity. + ++ Most importantly, [the most comprehensive benchmarking results](./leaderboard/index.md) on various models and datasets. For each result, the detailed reproducing step is recorded along with the open-source benchmarking scripts. diff --git a/docs/Matching/leaderboard/amazonbooks_m1.csv b/docs/Matching/leaderboard/amazonbooks_m1.csv new file mode 100644 index 00000000..c71edac4 --- /dev/null +++ b/docs/Matching/leaderboard/amazonbooks_m1.csv @@ -0,0 +1,30 @@ +Year,Publication,Model,Paper URL,Recall@20,Recall@50,NDCG@20,NDCG@50,HitRate@20,HitRate@50,Running Steps,Contributor +2001,WWW'01,ItemCF,https://files.grouplens.org/papers/www10_sarwar.pdf,0.0736,0.1175,0.0606,0.0771,0.3765,0.5234,https://github.com/reczoo/BARS/tree/main/matching/item2item/ItemCF/ItemCF_amazonbooks_m1,"Zhu et al." +2009,UAI'09,MF-BPR,https://arxiv.org/abs/1205.2618,0.0338,0.066,0.0261,0.038,0.2103,0.353,https://github.com/reczoo/BARS/tree/main/matching/cf/MF_BPR/MF_BPR_amazonbooks_m1,"Zhu et al." +2011,ICDM'11,SLIM,https://ieeexplore.ieee.org/document/6137254,0.0755,0.1257,0.0602,0.0791,0.3873,0.5472,https://github.com/reczoo/BARS/tree/main/matching/cf/SLIM/SLIM_amazonbooks_m1,"Zhu et al." +2017,CIKM'17,GRMF,https://dl.acm.org/doi/10.1145/3132847.3133036,0.0354,,0.027,,,,,"He et al." +2016,MLSP'16,Item2Vec,https://arxiv.org/abs/1603.04259,0.0326,0.0623,0.0252,0.0361,0.1897,0.3192,https://github.com/reczoo/BARS/tree/main/matching/item2item/Item2Vec/Item2Vec_amazonbooks_m1,"Zhu et al." +2016,RecSys'16,YoutubeDNN,https://research.google.com/pubs/archive/45530.pdf,0.0502,0.0924,0.0388,0.0545,0.2757,0.4354,https://github.com/reczoo/BARS/tree/main/matching/two_tower/YoutubeDNN/YoutubeDNN_amazonbooks_m1,"Zhu et al." +2017,WWW'17,NeuMF,https://arxiv.org/abs/1708.05031,0.0258,,0.02,,,,,"Wang et al." +2017,WWW'07,CML,https://scholar.archive.org/work/f3326mghgzcsjagczw6hr5lmce/access/wayback/http://pdfs.semanticscholar.org/61a9/f78bd184c0f95ffcbc10997d6b88a50be2c3.pdf,0.0522,0.0953,0.0428,0.0591,0.284,0.441,https://github.com/reczoo/BARS/tree/main/matching/cf/CML/CML_amazonbooks_m1,"Zhu et al." +2018,SIGIR'18,CMN,https://arxiv.org/abs/1804.10862,0.0267,,0.0218,,,,,"Wang et al." +2018,KDDW'18,GC-MC,https://arxiv.org/abs/1706.02263,0.0288,,0.0224,,,,,"Wang et al." +2018,KDD'18,PinSage,https://arxiv.org/abs/1806.01973,0.0282,,0.0219,,,,,"Wang et al." +2018,RecSys'18,HOP-Rec,https://dl.acm.org/doi/10.1145/3240323.3240381,0.0309,,0.0232,,,,,"Wang et al." +2018,ICLR'18,GAT,https://arxiv.org/abs/1710.10903,0.0326,,0.0235,,,,,"Song et al." +2018,WWW'18,MultVAE,https://dl.acm.org/doi/10.1145/3178876.3186150,0.0407,,0.0315,,,,,"He et al." +2019,ICML'19,DisenGCN,https://proceedings.mlr.press/v97/ma19a.html,0.0329,,0.0254,,,,,"Wang et al." +2019,SIGIR'19,NGCF,https://arxiv.org/abs/1905.08108,0.0344,,0.0263,,,,,"Wang et al." +2019,NeurIPS'19,MacridVAE,https://papers.nips.cc/paper/2019/file/a2186aa7c086b46ad4e8bf81e2a3a19b-Paper.pdf,0.0383,,0.0295,,,,,"Wang et al." +2019,WWW'19,EASE^R,https://arxiv.org/abs/1905.03375,0.071,0.1177,0.0567,0.0744,0.371,0.5293,https://github.com/reczoo/BARS/tree/main/matching/cf/EASE_r/EASE_amazonbooks_m1,"Zhu et al." +2020,TOIS'20,ENMF,http://www.thuir.cn/group/~mzhang/publications/TOIS2020Chen.pdf,0.0359,0.0691,0.0281,0.0404,0.2187,0.3649,https://github.com/reczoo/BARS/tree/main/matching/cf/ENMF/ENMF_amazonbooks_m1,"Zhu et al." +2020,AAAI'20,LR-GCCF,https://ojs.aaai.org/index.php/AAAI/article/view/5330,0.0335,,0.0265,,0.0349,,https://github.com/reczoo/BARS/tree/main/matching/gnn/LR_GCCF/LR_GCCF_amazonbooks_m1,"Zhu et al." +2020,SIGIR'20,NIA-GCN,https://dl.acm.org/doi/10.1145/3397271.3401123,0.0369,,0.0287,,,,,"Song et al." +2020,SIGIR'20,LightGCN,https://arxiv.org/abs/2002.02126,0.0411,0.0799,0.0318,0.0461,0.2423,0.4019,https://github.com/reczoo/BARS/tree/main/matching/gnn/LightGCN/LightGCN_amazonbooks_m1,"Zhu et al." +2020,SIGIR'20,DGCF,http://staff.ustc.edu.cn/~hexn/papers/sigir20-DGCF.pdf,0.0422,,0.0324,,,,,"Wang et al." +2020,Arixv'20,NGAT4Rec,https://arxiv.org/abs/2010.12256,0.0457,,0.0358,,,,,"Song et al." +2021,SIGIR'21,SGL-ED,http://staff.ustc.edu.cn/~hexn/papers/sigir21-SGL.pdf,0.0478,,0.0379,,,,,"Wu et al." +2021,CIKM'21,SimpleX,https://arxiv.org/abs/2109.12613,0.0583,0.1005,0.0468,0.0624,0.303,0.4557,https://github.com/reczoo/BARS/tree/main/matching/cf/SimpleX/SimpleX_amazonbooks_m1,"Zhu et al." +2021,CIKM'21,UltraGCN,https://arxiv.org/abs/2110.15114,0.0681,,0.0556,,,,https://github.com/reczoo/RecZoo/tree/main/matching/gnn/UltraGCN,"Zhu et al." +2021,CIKM'21,MF-CCL,https://arxiv.org/abs/2109.12613,0.0559,0.0961,0.0447,0.0596,0.2943,0.4427,https://github.com/reczoo/BARS/tree/main/matching/cf/MF_CCL/MF_CCL_amazonbooks_m1,"Zhu et al." +2021,CIKM'21,GF-CF,https://arxiv.org/abs/2108.07567,0.071,,0.0584,,,,,"Shen et al." diff --git a/docs/Matching/leaderboard/amazonbooks_m1.md b/docs/Matching/leaderboard/amazonbooks_m1.md new file mode 100644 index 00000000..5cd735e1 --- /dev/null +++ b/docs/Matching/leaderboard/amazonbooks_m1.md @@ -0,0 +1,27 @@ +--- +kernelspec: + name: python +jupytext: + formats: md:myst + text_representation: + extension: .md + format_name: myst + format_version: 0.13 + jupytext_version: 1.14.5 +--- + +# AmazonBooks_m1 + +```{note} +Please use the following evaluation settings for this benchmark: ++ Dataset split: [AmazonBooks_m1](https://github.com/reczoo/Datasets/tree/main/Amazon/AmazonBooks_m1) ++ Embedding size: 64 +``` + +🔥 **See the benchmarking results**: + +```{code-cell} +from plots import show_table, show_plot +show_plot("amazonbooks_m1.csv") +show_table("amazonbooks_m1.csv") +``` diff --git a/docs/Matching/leaderboard/gowalla_m1.csv b/docs/Matching/leaderboard/gowalla_m1.csv new file mode 100644 index 00000000..fe290e89 --- /dev/null +++ b/docs/Matching/leaderboard/gowalla_m1.csv @@ -0,0 +1,28 @@ +Year,Publication,Model,Paper URL,Recall@20,Recall@50,NDCG@20,NDCG@50,HitRate@20,HitRate@50,Running Steps,Contributor +2001,WWW'01,ItemCF,https://files.grouplens.org/papers/www10_sarwar.pdf,0.157,0.2549,0.1214,0.1527,0.5094,0.665,https://github.com/reczoo/BARS/tree/main/matching/item2item/ItemCF/ItemCF_gowalla_m1,"Zhu et al." +2009,UAI'09,MF-BPR,https://arxiv.org/abs/1205.2618,0.1627,0.2533,0.1378,0.1662,0.5544,0.6936,https://github.com/reczoo/BARS/tree/main/matching/cf/MF_BPR/MF_BPR_gowalla_m1,"Zhu et al." +2011,ICMD'11,SLIM,https://ieeexplore.ieee.org/document/6137254,0.1699,0.2658,0.1382,0.1687,0.5564,0.696,https://github.com/reczoo/BARS/tree/main/matching/cf/SLIM/SLIM_gowalla_m1,"Zhu et al." +2016,MLSP'16,Item2Vec,https://arxiv.org/abs/1603.04259,0.1326,0.2158,0.1057,0.132,0.4743,0.6188,https://github.com/reczoo/BARS/tree/main/matching/item2item/Item2Vec/Item2Vec_gowalla_m1,"Zhu et al." +2017,WWW'17,CML,https://scholar.archive.org/work/f3326mghgzcsjagczw6hr5lmce/access/wayback/http://pdfs.semanticscholar.org/61a9/f78bd184c0f95ffcbc10997d6b88a50be2c3.pdf,0.167,0.2602,0.1292,0.1587,0.541,0.675,https://github.com/reczoo/BARS/tree/main/matching/cf/benchmarks/CML/CML_gowalla_m1,"Zhu et al." +2019,WWW'19,EASE^R,https://arxiv.org/abs/1905.03375,0.1765,0.2701,0.1467,0.176,0.5727,0.7081,https://github.com/reczoo/BARS/tree/main/matching/cf/EASE_R/EASE_gowalla_m1,"Zhu et al." +2020,TOIS'20,ENMF,http://www.thuir.cn/group/~mzhang/publications/TOIS2020Chen.pdf,0.1523,0.2379,0.1315,0.1583,0.5336,0.6701,https://github.com/reczoo/BARS/tree/main/matching/cf/ENMF/ENMF_gowalla_m1,"Zhu et al." +2020,AAAI'20,LR-GCCF,https://ojs.aaai.org/index.php/AAAI/article/view/5330,0.1519,,0.1285,,0.1555,,https://github.com/reczoo/BARS/tree/main/matching/gnn/LR_GCCF/LR_GCCF_gowalla_m1,"Zhu et al." +2020,SIGIR'20,LightGCN,https://arxiv.org/abs/2002.02126,0.182,0.2821,0.1547,0.1859,0.5924,0.7295,https://github.com/reczoo/BARS/tree/main/matching/gnn/LightGCN/LightGCN_gowalla_m1,"Zhu et al." +2020,SIGIR'20,DGCF,http://staff.ustc.edu.cn/~hexn/papers/sigir20-DGCF.pdf,0.1794,,0.1521,,,,,"Wang et al." +2016,RecSys'16,YoutubeDNN,https://research.google.com/pubs/archive/45530.pdf,0.1754,0.2711,0.1473,0.1772,0.5628,0.6978,https://github.com/reczoo/BARS/tree/main/matching/two_tower/YoutubeDNN/YoutubeDNN_gowalla_m1,"Zhu et al." +2017,CIKM'17,GRMF,https://dl.acm.org/doi/10.1145/3132847.3133036,0.1477,,0.1205,,,,,"He et al." +2017,WWW'17,NeuMF,https://arxiv.org/abs/1708.05031,0.1399,,0.1212,,,,,"Wang et al." +2018,SIGIR'18,CMN,https://arxiv.org/abs/1804.10862,0.1405,,0.1221,,,,,"Wang et al." +2018,KDDW'18,GC-MC,https://arxiv.org/abs/1706.02263,0.1395,,0.1204,,,,,"Wang et al." +2018,KDD'18,PinSage,https://arxiv.org/abs/1806.01973,0.138,,0.1196,,,,,"Wang et al." +2018,RecSys'18,HOP-Rec,https://dl.acm.org/doi/10.1145/3240323.3240381,0.1399,,0.1214,,,,,"Wang et al." +2018,WWW'18,MultVAE,https://dl.acm.org/doi/10.1145/3178876.3186150,0.1641,,0.1335,,,,,"He et al." +2019,ICML'19,DisenGCN,https://proceedings.mlr.press/v97/ma19a.html,0.1356,,0.1174,,,,,"Wang et al." +2019,SIGIR'19,NGCF,https://arxiv.org/abs/1905.08108,0.157,,0.1327,,,,,"Wang et al." +2019,NeurIPS'19,MacridVAE,https://papers.nips.cc/paper/2019/file/a2186aa7c086b46ad4e8bf81e2a3a19b-Paper.pdf,0.1618,,0.1202,,,,,"Wang et al." +2020,SIGIR'20,NIA-GCN,https://dl.acm.org/doi/10.1145/3397271.3401123,0.1726,,0.1358,,,,,"Sun et al." +2021,CIKM'21,SimpleX,https://arxiv.org/abs/2109.12613,0.1872,0.2874,0.1557,0.1868,0.5927,0.7272,https://github.com/reczoo/BARS/tree/main/matching/cf/SimpleX/SimpleX_gowalla_m1,"Zhu et al." +2021,CIKM'21,UltraGCN,https://arxiv.org/abs/2110.15114,0.1862,,0.158,,,,https://github.com/reczoo/RecZoo/tree/main/matching/gnn/UltraGCN,"Zhu et al." +2021,CIKM'21,MF-CCL,https://arxiv.org/abs/2109.12613,0.1837,0.2871,0.1493,0.1816,0.584,0.7243,https://github.com/reczoo/BARS/tree/main/matching/cf/MF_CCL/MF_CCL_gowalla_m1,"Zhu et al." +2018,ICLR'18,GAT,https://arxiv.org/abs/1710.10903,0.1501,,0.1233,,,,,"Song et al." +2021,CIKM'21,GF-CF,https://arxiv.org/abs/2108.07567,0.1849,,0.1518,,,,,"Shen et al." diff --git a/docs/Matching/leaderboard/gowalla_m1.md b/docs/Matching/leaderboard/gowalla_m1.md new file mode 100644 index 00000000..c69ab3d2 --- /dev/null +++ b/docs/Matching/leaderboard/gowalla_m1.md @@ -0,0 +1,27 @@ +--- +kernelspec: + name: python +jupytext: + formats: md:myst + text_representation: + extension: .md + format_name: myst + format_version: 0.13 + jupytext_version: 1.14.5 +--- + +# Gowalla_m1 + +```{note} +Please use the following evaluation settings for this benchmark: ++ Dataset split: [Gowalla_m1](https://github.com/reczoo/Datasets/tree/main/Gowalla/Gowalla_m1) ++ Embedding size: 64 +``` + +🔥 **See the benchmarking results**: + +```{code-cell} +from plots import show_table, show_plot +show_plot("gowalla_m1.csv") +show_table("gowalla_m1.csv") +``` diff --git a/docs/Matching/leaderboard/index.md b/docs/Matching/leaderboard/index.md new file mode 100644 index 00000000..8e1009e6 --- /dev/null +++ b/docs/Matching/leaderboard/index.md @@ -0,0 +1,5 @@ +# Benchmark Leaderboard + + +```{tableofcontents} +``` diff --git a/docs/Matching/leaderboard/plots.py b/docs/Matching/leaderboard/plots.py new file mode 100644 index 00000000..c2f7a07d --- /dev/null +++ b/docs/Matching/leaderboard/plots.py @@ -0,0 +1,54 @@ +import pandas as pd +from itables import show +import plotly.graph_objects as go +from plotly.subplots import make_subplots +import itables.options as opt +opt.columnDefs = [{"className": "dt-center", "targets": "_all"}] + +def show_table(csv_path): + df = pd.read_csv(csv_path) + df = df.fillna("") + df["Model"] = df.apply(lambda x: f"{x['Model']}", axis=1) + del df["Paper URL"], df["Recall@50"], df["NDCG@50"], df["HitRate@50"] + df["Running Steps"] = df["Running Steps"].map(lambda x: f"🔗") + df = df.sort_values(by=["Recall@20"], ascending=False).reset_index(drop=True) + df.insert(0, "Rank", range(1, len(df) + 1)) + show(df, lengthMenu=[10, 20, 50, 100], classes="display") + +def show_plot(csv_path): + df = pd.read_csv(csv_path).sort_values(by="Recall@20", ascending=True) + fig = make_subplots(specs=[[{"secondary_y": True}]]) + fig.add_trace( + go.Scatter(x=df["Model"], y=df["Recall@20"], name="Recall@20", mode='lines+markers', + line=dict(color="#0071a7"), marker=dict(size=7)), + secondary_y=False, + ) + fig.add_trace( + go.Scatter(x=df["Model"], y=df["NDCG@20"], name="NDCG@20", mode='lines+markers', + line=dict(color="#ff404e"), marker=dict(size=7)), + secondary_y=True, + ) + + fig.update_layout( + title="Sorted benchmarking results by Recall@20", + title_x=0.5, + plot_bgcolor='white', + autosize=True, + width=890, + height=450, + legend=dict(orientation="h", x=0.35, y=-0.4) + ) + fig.update_xaxes(showgrid=False) + fig.update_yaxes( + showgrid=True, + gridcolor='lightgrey', + secondary_y=False, + title_text="Recall@20" + ) + fig.update_yaxes( + showgrid=False, + gridcolor='lightgrey', + secondary_y=True, + title_text="NDCG@20" + ) + fig.show() diff --git a/docs/Matching/leaderboard/yelp18_m1.csv b/docs/Matching/leaderboard/yelp18_m1.csv new file mode 100644 index 00000000..c33c1e39 --- /dev/null +++ b/docs/Matching/leaderboard/yelp18_m1.csv @@ -0,0 +1,29 @@ +Year,Publication,Model,Paper URL,Recall@20,Recall@50,NDCG@20,NDCG@50,HitRate@20,HitRate@50,Running Steps,Contributor +2001,WWW'01,ItemCF,https://files.grouplens.org/papers/www10_sarwar.pdf,0.0639,0.1219,0.0531,0.0746,0.3876,0.5753,https://github.com/reczoo/BARS/tree/main/matching/item2item/ItemCF/ItemCF_yelp18_m1,"Zhu et al." +2009,UAI'09,MF-BPR,https://arxiv.org/abs/1205.2618,0.0576,0.1123,0.0468,0.0671,0.3624,0.5577,https://github.com/reczoo/BARS/tree/main/matching/cf/MF_BPR/MF_BPR_yelp18_m1,"Zhu et al." +2011,ICMD'11,SLIM,https://ieeexplore.ieee.org/document/6137254,0.0646,0.1213,0.0541,0.0751,0.3912,0.5799,https://github.com/reczoo/BARS/tree/main/matching/cf/SLIM/SLIM_yelp18_m1,"Zhu et al." +2016,MLSP'16,Item2Vec,https://arxiv.org/abs/1603.04259,0.0503,0.0971,0.0411,0.0585,0.3267,0.5114,https://github.com/reczoo/BARS/tree/main/matching/item2item/Item2Vec/Item2Vec_yelp18_m1,"Zhu et al." +2017,WWW'17,CML,https://scholar.archive.org/work/f3326mghgzcsjagczw6hr5lmce/access/wayback/http://pdfs.semanticscholar.org/61a9/f78bd184c0f95ffcbc10997d6b88a50be2c3.pdf,0.0622,0.1181,0.0536,0.0738,0.381,0.551,https://github.com/reczoo/BARS/tree/main/matching/cf/CML/CML_yelp18_m1,"Zhu et al." +2019,WWW'19,EASE^R,https://arxiv.org/abs/1905.03375,0.0657,0.1225,0.0552,0.0762,0.3966,0.5839,https://github.com/reczoo/BARS/tree/main/matching/cf/EASE_r/EASE_yelp18_m1,"Zhu et al." +2020,TOIS'20,ENMF,http://www.thuir.cn/group/~mzhang/publications/TOIS2020Chen.pdf,0.0624,0.1189,0.0515,0.0723,0.3848,0.5792,https://github.com/reczoo/BARS/tree/main/matching/cf/ENMF/ENMF_yelp18_m1,"Zhu et al." +2020,AAAI'20,LR-GCCF,https://ojs.aaai.org/index.php/AAAI/article/view/5330,0.0558,,0.0343,,0.0561,,https://github.com/reczoo/BARS/tree/main/matching/gnn/LR_GCCF/LR_GCCF_yelp18_m1,"Zhu et al." +2020,SIGIR'20,LightGCN,https://arxiv.org/abs/2002.02126,0.0653,0.1254,0.0532,0.0756,0.3974,0.5922,https://github.com/reczoo/BARS/tree/main/matching/gnn/LightGCN/LightGCN_yelp18_m1,"Zhu et al." +2016,RecSys'16,YoutubeDNN,https://research.google.com/pubs/archive/45530.pdf,0.0686,0.1312,0.0567,0.0798,0.41,0.606,https://github.com/reczoo/BARS/tree/main/matching/two_tower/YoutubeDNN/YoutubeDNN_yelp18_m1,"Zhu et al." +2017,CIKM'17,GRMF,https://dl.acm.org/doi/10.1145/3132847.3133036,0.0571,,0.0462,,,,,"He et al." +2017,WWW'17,NeuMF,https://arxiv.org/abs/1708.05031,0.0451,,0.0363,,,,,"Wang et al." +2018,SIGIR'18,CMN,https://arxiv.org/abs/1804.10862,0.0457,,0.0369,,,,,"Wang et al." +2018,KDDW'18,GC-MC,https://arxiv.org/abs/1706.02263,0.0462,,0.0379,,,,,"Wang et al." +2018,KDD'18,PinSage,https://arxiv.org/abs/1806.01973,0.0471,,0.0393,,,,,"Wang et al." +2018,RecSys'18,HOP-Rec,https://dl.acm.org/doi/10.1145/3240323.3240381,0.0517,,0.0428,,,,,"Wang et al." +2018,ICLR'18,GAT,https://arxiv.org/abs/1710.10903,0.0543,,0.0431,,,,,"Song et al." +2018,WWW'18,MultVAE,https://dl.acm.org/doi/10.1145/3178876.3186150,0.0584,,0.045,,,,,"Reported by SGL-ED" +2019,ICML'19,DisenGCN,https://proceedings.mlr.press/v97/ma19a.html,0.0558,,0.0454,,,,,"Wang et al." +2019,SIGIR'19,NGCF,https://arxiv.org/abs/1905.08108,0.0579,,0.0477,,,,,"Wang et al." +2019,NeurIPS'19,MacridVAE,https://papers.nips.cc/paper/2019/file/a2186aa7c086b46ad4e8bf81e2a3a19b-Paper.pdf,0.0612,,0.0495,,,,,"Wang et al." +2020,SIGIR'20,NIA-GCN,https://dl.acm.org/doi/10.1145/3397271.3401123,0.0599,,0.0491,,,,,"Song et al." +2020,Arixv'20,NGAT4Rec,https://arxiv.org/abs/2010.12256,0.0675,,0.0554,,,,,"Song et al." +2021,SIGIR'21,SGL-ED,http://staff.ustc.edu.cn/~hexn/papers/sigir21-SGL.pdf,0.0675,,0.0555,,,,,"Wu et al." +2021,CIKM'21,SimpleX,https://arxiv.org/abs/2109.12613,0.0701,0.1322,0.0575,0.0805,0.4152,0.6088,https://github.com/reczoo/BARS/tree/main/matching/cf/SimpleX/SimpleX_yelp18_m1,"Zhu et al." +2021,CIKM'21,UltraGCN,https://arxiv.org/abs/2110.15114,0.0683,,0.0561,,,,https://github.com/reczoo/RecZoo/tree/main/matching/gnn/UltraGCN,"Zhu et al." +2021,CIKM'21,MF-CCL,https://arxiv.org/abs/2109.12613,0.0698,0.1322,0.0572,0.0803,0.4133,0.6106,https://github.com/reczoo/BARS/tree/main/matching/cf/MF_CCL/MF_CCL_yelp18_m1,"Zhu et al." +2021,CIKM'21,GF-CF,https://arxiv.org/abs/2108.07567,0.0697,,0.0571,,,,,"Shen et al." diff --git a/docs/Matching/leaderboard/yelp18_m1.md b/docs/Matching/leaderboard/yelp18_m1.md new file mode 100644 index 00000000..29e56156 --- /dev/null +++ b/docs/Matching/leaderboard/yelp18_m1.md @@ -0,0 +1,27 @@ +--- +kernelspec: + name: python +jupytext: + formats: md:myst + text_representation: + extension: .md + format_name: myst + format_version: 0.13 + jupytext_version: 1.14.5 +--- + +# Yelp18_m1 + +```{note} +Please use the following evaluation settings for this benchmark: ++ Dataset split: [Yelp18_m1](https://github.com/reczoo/Datasets/tree/main/Yelp/Yelp18_m1) ++ Embedding size: 64 +``` + +🔥 **See the benchmarking results**: + +```{code-cell} +from plots import show_table, show_plot +show_plot("yelp18_m1.csv") +show_table("yelp18_m1.csv") +``` diff --git a/docs/Matching/papers.md b/docs/Matching/papers.md new file mode 100644 index 00000000..7b78a15d --- /dev/null +++ b/docs/Matching/papers.md @@ -0,0 +1,87 @@ +# Candidate Matching + +A curated list of candidate item matching models + + +## Collaborative Filtering + +| | | | | | +| :---------:|:------:|:------:|:------:|:------:| +| **2021** | [SimpleX](https://arxiv.org/abs/2109.12613) {cite}`SimpleX`
CIKM'21
Huawei | +| **2020** | [ENMF](http://www.thuir.cn/group/~mzhang/publications/TOIS2020Chen.pdf) {cite}`ENMF`
TOIS'20 | +| **2019** | [EASE^R](https://arxiv.org/abs/1905.03375) {cite}`EASE_R`
WWW'19
Netflix | [MacridVAE](https://papers.nips.cc/paper/2019/file/a2186aa7c086b46ad4e8bf81e2a3a19b-Paper.pdf) {cite}`MacridVAE`
NeurIPS'19
Alibaba | +| **2018** | [MultVAE](https://dl.acm.org/doi/10.1145/3178876.3186150) {cite}`MultVAE`
WWW'18
Netflix, Google | [CMN](https://arxiv.org/abs/1804.10862) {cite}`CMN`
SIGIR'18
Google | +| **2017** | [NeuMF](https://arxiv.org/abs/1708.05031) {cite}`NCF`
WWW'17 | [CML](https://scholar.archive.org/work/f3326mghgzcsjagczw6hr5lmce/access/wayback/http://pdfs.semanticscholar.org/61a9/f78bd184c0f95ffcbc10997d6b88a50be2c3.pdf) {cite}`CML`
WWW'17 | +| **2011** | [SLIM](https://ieeexplore.ieee.org/document/6137254) {cite}`SLIM`
ICDM'11 | +| **2009** | [BPR](https://arxiv.org/abs/1205.2618) {cite}`BPR`
UAI'09 | + + +## Two-Tower Matching + +| | | | | | +| :---------:|:------:|:------:|:------:|:------:| +| **2020** | [EBR](https://arxiv.org/abs/2006.11632) {cite}`EBR`
KDD'20
Facebook | +| **2016** | [YoutubeDNN](https://research.google.com/pubs/archive/45530.pdf) {cite}`YoutubeDNN`
RecSys'16
Google | +| **2014** | [DSSM](https://www.iro.umontreal.ca/~lisa/pointeurs/WWW2014.pdf) {cite}`DSSM`
WWW'14
Microsoft | + +## Multi-Interest Matching + +| | | | | | +| :---------:|:------:|:------:|:------:|:------:| +| **2021** | [SINE](https://arxiv.org/abs/2102.09267) {cite}`SINE`
WSDM'21
Alibaba | +| **2020** | [ComiRec](https://arxiv.org/abs/2005.09347) {cite}`ComiRec`
KDD'20
Alibaba | [PinnerSage](https://arxiv.org/abs/2007.03634) {cite}`PinnerSage`
KDD'20
Pinterest | +| **2019** | [MIND](https://arxiv.org/abs/1904.08030) {cite}`MIND`
CIKM'19
Alibaba | + +## Sequential Recommendation + +| | | | | | +| :---------:|:------:|:------:|:------:|:------:| +| **2019** | [BERT4Rec](https://arxiv.org/abs/1904.06690) {cite}`BERT4Rec`
CIKM'19
Alibaba | [SDM](https://arxiv.org/abs/1909.00385) {cite}`SDM`
CIKM'19
Alibaba | +| **2018** | [SASRec](https://arxiv.org/abs/1808.09781) {cite}`SASRec`
ICDM'18 | +| **2016** | [GRU4Rec](https://arxiv.org/abs/1511.06939) {cite}`GRU4Rec`
ICLR'16
Gravity | + +## Graph-based Matching + +| | | | | | +| :---------:|:------:|:------:|:------:|:------:| +| **2021** | [UltraGCN](https://arxiv.org/abs/2110.15114) {cite}`UltraGCN`
CIKM'21
Huawei | [SGL](https://arxiv.org/abs/2010.10783) {cite}`SGL`
SIGIR'21 | [GF-CF](https://arxiv.org/abs/2108.07567) {cite}`GFCF`
CIKM'21 | +| **2020** | [LightGCN](https://arxiv.org/abs/2002.02126) {cite}`LightGCN`
SIGIR'20 | +| **2019** | [NGCF](https://arxiv.org/abs/1905.08108) {cite}`NGCF`
SIGIR'19 | +| **2018** | [PinSage](https://arxiv.org/abs/1806.01973) {cite}`PinSage`
KDD'18
Pinterest | [EGES](https://arxiv.org/abs/1803.02349) {cite}`EGES`
KDD'18
Alibaba | + + +## Item-to-Item Matching + +| | | | | | +| :---------:|:------:|:------:|:------:|:------:| +| **2016** | [Item2Vec](https://arxiv.org/abs/1603.04259) {cite}`Item2Vec`
MLSP'16
Microsoft | +| **2001** | [ItemCF](https://files.grouplens.org/papers/www10_sarwar.pdf) {cite}`ItemCF`
WWW'01
GroupLens | + +## Generative Retrieval + +| | | | | | +| :---------:|:------:|:------:|:------:|:------:| +| **2019** | [JTM](https://arxiv.org/abs/1902.07565) {cite}`JTM`
NeurIPS'19
Alibaba | +| **2018** | [TDM](https://arxiv.org/abs/1801.02294) {cite}`TDM`
KDD'18
Alibaba | + +## Lookalike Models + +| | | | | | +| :---------:|:------:|:------:|:------:|:------:| +| **2019** | [RALM](https://arxiv.org/abs/1902.07565) {cite}`RALM`
KDD'19
Tencent | + + +## Pretraining + +| | | | | | +| :---------:|:------:|:------:|:------:|:------:| +| **2023** | [VQRec](https://arxiv.org/abs/2210.12316) {cite}`VQRec`
WWW'23 | +| **2022** | [UniSRec](https://arxiv.org/abs/2206.05941) {cite}`UniSRec`
KDD'22 | + + +## References + +```{bibliography} +:style: unsrt +:filter: docname in docnames +``` diff --git a/docs/Matching/references.bib b/docs/Matching/references.bib new file mode 100644 index 00000000..4b73739a --- /dev/null +++ b/docs/Matching/references.bib @@ -0,0 +1,453 @@ +@inproceedings{EBR, + author = {Jui{-}Ting Huang and + Ashish Sharma and + Shuying Sun and + Li Xia and + David Zhang and + Philip Pronin and + Janani Padmanabhan and + Giuseppe Ottaviano and + Linjun Yang}, + title = {{Embedding-based Retrieval in Facebook Search}}, + booktitle = {The 26th {ACM} {SIGKDD} Conference on Knowledge Discovery + and Data Mining ({KDD} '20)}, + pages = {2553--2561}, + year = {2020}, +} + +@inproceedings{PinnerSage, + author = {Aditya Pal and + Chantat Eksombatchai and + Yitong Zhou and + Bo Zhao and + Charles Rosenberg and + Jure Leskovec}, + title = {{PinnerSage: Multi-Modal User Embedding Framework for Recommendations + at Pinterest}}, + booktitle = {The 26th {ACM} {SIGKDD} Conference on Knowledge Discovery + and Data Mining ({KDD} '20)}, + pages = {2311--2320}, + year = {2020}, +} + +@inproceedings{ComiRec, + author = {Yukuo Cen and + Jianwei Zhang and + Xu Zou and + Chang Zhou and + Hongxia Yang and + Jie Tang}, + title = {{Controllable Multi-Interest Framework for Recommendation}}, + booktitle = {The 26th {ACM} {SIGKDD} Conference on Knowledge Discovery + and Data Mining ({KDD} '20)}, + pages = {2942--2951}, + year = {2020}, +} + +@inproceedings{MIND, + author = {Chao Li and + Zhiyuan Liu and + Mengmeng Wu and + Yuchi Xu and + Huan Zhao and + Pipei Huang and + Guoliang Kang and + Qiwei Chen and + Wei Li and + Dik Lun Lee}, + title = {{Multi-Interest Network with Dynamic Routing for Recommendation at + Tmall}}, + booktitle = {Proceedings of the 28th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '19)}, + pages = {2615--2623}, + year = {2019}, +} + +@inproceedings{BERT4Rec, + author = {Fei Sun and + Jun Liu and + Jian Wu and + Changhua Pei and + Xiao Lin and + Wenwu Ou and + Peng Jiang}, + title = {{BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations + from Transformer}}, + booktitle = {Proceedings of the 28th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '19)}, + pages = {1441--1450}, + year = {2019}, +} + +@inproceedings{TDM, + author = {Han Zhu and + Xiang Li and + Pengye Zhang and + Guozheng Li and + Jie He and + Han Li and + Kun Gai}, + title = {{Learning Tree-based Deep Model for Recommender Systems}}, + booktitle = {Proceedings of the 24th {ACM} {SIGKDD} International Conference on + Knowledge Discovery {\&} Data Mining ({KDD} '18)}, + pages = {1079--1088}, + year = {2018}, +} + +@inproceedings{JTM, + author = {Han Zhu and + Daqing Chang and + Ziru Xu and + Pengye Zhang and + Xiang Li and + Jie He and + Han Li and + Jian Xu and + Kun Gai}, + title = {{Joint Optimization of Tree-based Index and Deep Model for Recommender + Systems}}, + booktitle = {Annual Conference on Neural Information Processing Systems (NeurIPS '19)}, + pages = {3973--3982}, + year = {2019}, +} + +@inproceedings{ItemCF, + author = {Badrul Munir Sarwar and + George Karypis and + Joseph A. Konstan and + John Riedl}, + title = {{Item-based Collaborative Filtering Recommendation Algorithms}}, + booktitle = {Proceedings of the Tenth International World Wide Web Conference ({WWW} '01)}, + pages = {285--295}, + year = {2001}, +} + +@inproceedings{BPR, + author = {Steffen Rendle and + Christoph Freudenthaler and + Zeno Gantner and + Lars Schmidt{-}Thieme}, + title = {{{BPR:} Bayesian Personalized Ranking from Implicit Feedback}}, + booktitle = {Proceedings of the Twenty-Fifth Conference on Uncertainty + in Artificial Intelligence ({UAI} '09)}, + pages = {452--461}, + year = {2009}, +} + +@inproceedings{SLIM, + author = {Xia Ning and + George Karypis}, + title = {{{SLIM:} Sparse Linear Methods for Top-N Recommender Systems}}, + booktitle = {Proceedings of the 11th {IEEE} International Conference on Data Mining ({ICDM} '11)}, + pages = {497--506}, + year = {2011}, +} + +@inproceedings{NCF, + author = {Xiangnan He and + Lizi Liao and + Hanwang Zhang and + Liqiang Nie and + Xia Hu and + Tat{-}Seng Chua}, + title = {{Neural Collaborative Filtering}}, + booktitle = {Proceedings of the 26th International Conference on World Wide Web ({WWW} '17)}, + pages = {173--182}, + year = {2017}, +} + +@inproceedings{NGCF, + author = {Xiang Wang and + Xiangnan He and + Meng Wang and + Fuli Feng and + Tat{-}Seng Chua}, + title = {{Neural Graph Collaborative Filtering}}, + booktitle = {Proceedings of the 42nd International {ACM} {SIGIR} Conference on + Research and Development in Information Retrieval ({SIGIR} '19)}, + pages = {165--174}, + year = {2019}, +} + +@inproceedings{EASE_R, + author = {Harald Steck}, + title = {{Embarrassingly Shallow Autoencoders for Sparse Data}}, + booktitle = {The World Wide Web Conference ({WWW} '19)}, + pages = {3251--3257}, + year = {2019}, +} + +@inproceedings{LightGCN, + author = {Xiangnan He and + Kuan Deng and + Xiang Wang and + Yan Li and + Yong{-}Dong Zhang and + Meng Wang}, + title = {{LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation}}, + booktitle = {Proceedings of the 43rd International {ACM} {SIGIR} Conference on + Research and Development in Information Retrieval ({SIGIR} '20)}, + pages = {639--648}, + year = {2020}, +} + +@inproceedings{SimpleX, + author = {Kelong Mao and + Jieming Zhu and + Jinpeng Wang and + Quanyu Dai and + Zhenhua Dong and + Xi Xiao and + Xiuqiang He}, + title = {{SimpleX: {A} Simple and Strong Baseline for Collaborative Filtering}}, + booktitle = {The 30th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '21)}, + pages = {1243--1252}, + year = {2021}, +} + +@inproceedings{UltraGCN, + author = {Kelong Mao and + Jieming Zhu and + Xi Xiao and + Biao Lu and + Zhaowei Wang and + Xiuqiang He}, + title = {{UltraGCN: Ultra Simplification of Graph Convolutional Networks for + Recommendation}}, + booktitle = {The 30th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '21)}, + pages = {1253--1262}, + year = {2021}, +} + +@inproceedings{SGL, + author = {Jiancan Wu and + Xiang Wang and + Fuli Feng and + Xiangnan He and + Liang Chen and + Jianxun Lian and + Xing Xie}, + title = {{Self-supervised Graph Learning for Recommendation}}, + booktitle = {The 44th International {ACM} {SIGIR} Conference on Research + and Development in Information Retrieval ({SIGIR} '21)}, + pages = {726--735}, + year = {2021}, +} + +@inproceedings{PinSage, + author = {Rex Ying and + Ruining He and + Kaifeng Chen and + Pong Eksombatchai and + William L. Hamilton and + Jure Leskovec}, + title = {{Graph Convolutional Neural Networks for Web-Scale Recommender Systems}}, + booktitle = {Proceedings of the 24th {ACM} {SIGKDD} International Conference on + Knowledge Discovery {\&} Data Mining ({KDD} '18)}, + pages = {974--983}, + year = {2018}, +} + +@inproceedings{EGES, + author = {Jizhe Wang and + Pipei Huang and + Huan Zhao and + Zhibo Zhang and + Binqiang Zhao and + Dik Lun Lee}, + title = {{Billion-scale Commodity Embedding for E-commerce Recommendation in + Alibaba}}, + booktitle = {Proceedings of the 24th {ACM} {SIGKDD} International Conference on + Knowledge Discovery {\&} Data Mining ({KDD} '18)}, + pages = {839--848}, + year = {2018}, +} + +@inproceedings{GRU4Rec, + author = {Bal{\'{a}}zs Hidasi and + Alexandros Karatzoglou and + Linas Baltrunas and + Domonkos Tikk}, + title = {{Session-based Recommendations with Recurrent Neural Networks}}, + booktitle = {Proceedings of the 4th International Conference on Learning Representations ({ICLR} '16)}, + year = {2016}, +} + +@inproceedings{SDM, + author = {Fuyu Lv and + Taiwei Jin and + Changlong Yu and + Fei Sun and + Quan Lin and + Keping Yang and + Wilfred Ng}, + title = {{{SDM:} Sequential Deep Matching Model for Online Large-scale Recommender + System}}, + booktitle = {Proceedings of the 28th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '19)}, + pages = {2635--2643}, + year = {2019}, +} + +@inproceedings{SINE, + author = {Qiaoyu Tan and + Jianwei Zhang and + Jiangchao Yao and + Ninghao Liu and + Jingren Zhou and + Hongxia Yang and + Xia Hu}, + title = {{Sparse-Interest Network for Sequential Recommendation}}, + booktitle = {The Fourteenth {ACM} International Conference on Web Search + and Data Mining ({WSDM} '21)}, + pages = {598--606}, + year = {2021}, +} + +@inproceedings{SASRec, + author = {Wang{-}Cheng Kang and + Julian J. McAuley}, + title = {{Self-Attentive Sequential Recommendation}}, + booktitle = {{IEEE} International Conference on Data Mining ({ICDM} '18)}, + pages = {197--206}, + year = {2018}, +} + +@inproceedings{Item2Vec, + author = {Oren Barkan and + Noam Koenigstein}, + title = {{Item2Vec: Neural Item Embedding for Collaborative Filtering}}, + booktitle = {The 26th {IEEE} International Workshop on Machine Learning for Signal + Processing ({MLSP} '16)}, + pages = {1--6}, + year = {2016}, +} + +@inproceedings{GFCF, + author = {Yifei Shen and + Yongji Wu and + Yao Zhang and + Caihua Shan and + Jun Zhang and + Khaled B. Letaief and + Dongsheng Li}, + title = {{How Powerful is Graph Convolution for Recommendation?}}, + booktitle = {The 30th {ACM} International Conference on Information + and Knowledge Management ({CIKM} '21)}, + pages = {1619--1629}, + year = {2021}, +} + +@inproceedings{DSSM, + author = {Yelong Shen and + Xiaodong He and + Jianfeng Gao and + Li Deng and + Gr{\'{e}}goire Mesnil}, + title = {{Learning Semantic Representations Using Convolutional Neural Networks for Web Search}}, + booktitle = {Proceedings of the 23rd International World Wide Web Conference ({WWW} '14)}, + pages = {373--374}, + year = {2014}, +} + +@inproceedings{MacridVAE, + author = {Jianxin Ma and + Chang Zhou and + Peng Cui and + Hongxia Yang and + Wenwu Zhu}, + title = {{Learning Disentangled Representations for Recommendation}}, + booktitle = {Annual Conference on Neural Information Processing Systems (NeurIPS '19)}, + pages = {5712--5723}, + year = {2019}, +} + +@inproceedings{MultVAE, + author = {Dawen Liang and + Rahul G. Krishnan and + Matthew D. Hoffman and + Tony Jebara}, + title = {{Variational Autoencoders for Collaborative Filtering}}, + booktitle = {Proceedings of the 2018 World Wide Web Conference on World Wide Web ({WWW} '18)}, + pages = {689--698}, + year = {2018}, +} + +@inproceedings{CMN, + author = {Travis Ebesu and + Bin Shen and + Yi Fang}, + title = {{Collaborative Memory Network for Recommendation Systems}}, + booktitle = {The 41st International {ACM} {SIGIR} Conference on Research {\&} + Development in Information Retrieval ({SIGIR} '18)}, + pages = {515--524}, + year = {2018}, +} + +@inproceedings{CML, + author = {Cheng{-}Kang Hsieh and + Longqi Yang and + Yin Cui and + Tsung{-}Yi Lin and + Serge J. Belongie and + Deborah Estrin}, + title = {{Collaborative Metric Learning}}, + booktitle = {Proceedings of the 26th International Conference on World Wide Web ({WWW} '17)}, + pages = {193--201}, + year = {2017}, +} + +@article{ENMF, + author = {Chong Chen and + Min Zhang and + Yongfeng Zhang and + Yiqun Liu and + Shaoping Ma}, + title = {{Efficient Neural Matrix Factorization without Sampling for Recommendation}}, + journal = {{ACM} Trans. Inf. Syst.}, + volume = {38}, + number = {2}, + pages = {14:1--14:28}, + year = {2020}, +} + +@inproceedings{RALM, + author = {Yudan Liu and + Kaikai Ge and + Xu Zhang and + Leyu Lin}, + title = {{Real-time Attention Based Look-alike Model for Recommender System}}, + booktitle = {Proceedings of the 25th {ACM} {SIGKDD} International Conference on + Knowledge Discovery {\&} Data Mining ({KDD} '19)}, + pages = {2765--2773}, + year = {2019}, +} + +@inproceedings{UniSRec, + author = {Yupeng Hou and + Shanlei Mu and + Wayne Xin Zhao and + Yaliang Li and + Bolin Ding and + Ji{-}Rong Wen}, + title = {{Towards Universal Sequence Representation Learning for Recommender + Systems}}, + booktitle = {The 28th {ACM} {SIGKDD} Conference on Knowledge Discovery + and Data Mining ({KDD} '22)}, + pages = {585--593}, + year = {2022}, +} + +@inproceedings{VQRec, + author = {Yupeng Hou and + Zhankui He and + Julian J. McAuley and + Wayne Xin Zhao}, + title = {{Learning Vector-Quantized Item Representation for Transferable Sequential + Recommenders}}, + booktitle = {Proceedings of the {ACM} Web Conference ({WWW} '23)}, + pages = {1162--1171}, + year = {2023}, +} diff --git a/docs/Reranking/papers.md b/docs/Reranking/papers.md new file mode 100644 index 00000000..d26dd276 --- /dev/null +++ b/docs/Reranking/papers.md @@ -0,0 +1,24 @@ +# Multi-Task Ranking + +A curated list of multi-task ranking models + +### Multi-Task Modeling + +| | | | | | +| :---------:|:------:|:------:|:------:|:------:| +| **2023** | KDD'23 | + + +### Multi-Domain Modeling + +| | | | | | | +| :---------:|:------:|:------:|:------:|:------:|:------:| +| **2023** | + + +### References + +```{bibliography} +:style: unsrt +:filter: docname in docnames +``` diff --git a/_config.yml b/docs/_config.yml similarity index 70% rename from _config.yml rename to docs/_config.yml index ea58ccc9..db196898 100644 --- a/_config.yml +++ b/docs/_config.yml @@ -1,25 +1,26 @@ -title: BARS Benchmark -author: The BARS Community -copyright: 2020-2023 -# logo: ./img/logo.png +title: BARS BENCHMARK +author: RECZOO Community +copyright: 2020-2024 +logo: "" exclude_patterns: [_build, Thumbs.db, .DS_Store, "**.ipynb_checkpoints"] only_build_toc_files: true # Force re-execution of notebooks on each build. # See https://jupyterbook.org/content/execute.html execute: - execute_notebooks: off + execute_notebooks: force allow_errors: false # If `False`, when a code cell raises an error the execution is stopped, otherwise all cells are always run. stderr_output: show # One of 'show', 'remove', 'remove-warn', 'warn', 'error', 'severe' # Define the name of the latex output file for PDF builds latex: latex_documents: - targetname: bars.tex + targetname: BARS.tex # Add a bibtex file so that we can create citations bibtex_bibfiles: - - docs/citations.bib + - ./CTR/references.bib + - ./Matching/references.bib parse: myst_url_schemes: [mailto, http, https] # URI schemes that will be recognised as external URLs in Markdown links @@ -27,7 +28,7 @@ parse: # Information about where the book exists on the web repository: - url: https://github.com/openbenchmark/BARS # Online location of your book + url: https://github.com/reczoo/BARS # Online location of your book # path_to_book: docs # Optional path to your book, relative to the repository root branch: main # Which branch of the repository should be used when creating links (optional) @@ -39,7 +40,7 @@ html: use_repository_button : true # Whether to add a link to your repository button use_issues_button : false # Whether to add an "open an issue" button use_multitoc_numbering : true # Continuous numbering across parts/chapters - extra_navbar :
Join Wechat Group # Will be displayed underneath the left navbar. + extra_navbar: '
Join Wechat Group' # Will be displayed underneath the left navbar. extra_footer : "" # Will be displayed underneath the footer. google_analytics_id : "" # A GA id that can be used to track book views. home_page_in_navbar : true # Whether to include your home page in the left Navigation Bar @@ -49,13 +50,16 @@ html: utterances : false announcement : "" # A banner announcement at the top of the site. -# sphinx: -# config: -# html_js_files: -# - https://cdn.datatables.net/r/zf-5.5.2/jqc-1.11.3,dt-1.10.8/datatables.min.js -# - https://code.highcharts.com/highcharts.js -# - https://code.highcharts.com/modules/no-data-to-display.js -# - https://openbenchmark.github.io/BarsCTR/js/tablecharts.js -# html_css_files: -# - https://stackpath.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css +sphinx: + config: + bibtex_reference_style: label + html_js_files: + - https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js + # html_js_files: + # - https://cdn.datatables.net/r/zf-5.5.2/jqc-1.11.3,dt-1.10.8/datatables.min.js + # - https://code.highcharts.com/highcharts.js + # - https://code.highcharts.com/modules/no-data-to-display.js + # # - https://openbenchmark.github.io/BarsCTR/js/tablecharts.js + # html_css_files: + # # - https://stackpath.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css # - https://cdn.datatables.net/r/zf-5.5.2/jqc-1.11.3,dt-1.10.8/datatables.min.css diff --git a/docs/_toc.yml b/docs/_toc.yml new file mode 100644 index 00000000..bd4a4d5a --- /dev/null +++ b/docs/_toc.yml @@ -0,0 +1,51 @@ +# Table of contents +# Learn more at https://jupyterbook.org/customize/toc.html + +format: jb-book +root: index.md +parts: +- caption: PAPER LIST + chapters: + - file: CTR/papers.md + - file: Matching/papers.md + +- caption: "CTR PREDICTION" + chapters: + - file: CTR/index.md + - url: https://reczoo.github.io/datasets/CTR + title: Benchmark Datasets + - url: https://github.com/openbenchmark/BARS/tree/main/ranking/ctr + title: Benchmark Code + - file: CTR/leaderboard/index.md + sections: + - file: CTR/leaderboard/avazu_x1.md + - file: CTR/leaderboard/avazu_x4.md + - file: CTR/leaderboard/criteo_x1.md + - file: CTR/leaderboard/criteo_x4.md + - file: CTR/leaderboard/frappe_x1.md + - file: CTR/leaderboard/kkbox_x1.md + - file: CTR/leaderboard/kuaivideo_x1.md + - file: CTR/leaderboard/microvideo1.7m_x1.md + - file: CTR/leaderboard/movielenslatest_x1.md + - file: CTR/leaderboard/taobaoad_x1.md + +- caption: "CANDIDATE MATCHING" + chapters: + - file: Matching/index.md + - url: https://reczoo.github.io/datasets/Matching + title: Benchmark Datasets + - url: https://github.com/openbenchmark/BARS/tree/main/matching + title: Benchmark Code + - file: Matching/leaderboard/index.md + sections: + - file: Matching/leaderboard/amazonbooks_m1.md + - file: Matching/leaderboard/gowalla_m1.md + - file: Matching/leaderboard/yelp18_m1.md + +# - caption: RE-RANKING BENCHMARK + +# - caption: PRE-TRAINING BENCHMARK + +- caption: CONTRIBUTION + chapters: + - file: contribution.md diff --git a/docs/citations.bib b/docs/citations.bib deleted file mode 100644 index 783ec6aa..00000000 --- a/docs/citations.bib +++ /dev/null @@ -1,56 +0,0 @@ ---- ---- - -@inproceedings{holdgraf_evidence_2014, - address = {Brisbane, Australia, Australia}, - title = {Evidence for {Predictive} {Coding} in {Human} {Auditory} {Cortex}}, - booktitle = {International {Conference} on {Cognitive} {Neuroscience}}, - publisher = {Frontiers in Neuroscience}, - author = {Holdgraf, Christopher Ramsay and de Heer, Wendy and Pasley, Brian N. and Knight, Robert T.}, - year = {2014} -} - -@article{holdgraf_rapid_2016, - title = {Rapid tuning shifts in human auditory cortex enhance speech intelligibility}, - volume = {7}, - issn = {2041-1723}, - url = {http://www.nature.com/doifinder/10.1038/ncomms13654}, - doi = {10.1038/ncomms13654}, - number = {May}, - journal = {Nature Communications}, - author = {Holdgraf, Christopher Ramsay and de Heer, Wendy and Pasley, Brian N. and Rieger, Jochem W. and Crone, Nathan and Lin, Jack J. and Knight, Robert T. and Theunissen, Frédéric E.}, - year = {2016}, - pages = {13654}, - file = {Holdgraf et al. - 2016 - Rapid tuning shifts in human auditory cortex enhance speech intelligibility.pdf:C\:\\Users\\chold\\Zotero\\storage\\MDQP3JWE\\Holdgraf et al. - 2016 - Rapid tuning shifts in human auditory cortex enhance speech intelligibility.pdf:application/pdf} -} - -@inproceedings{holdgraf_portable_2017, - title = {Portable learning environments for hands-on computational instruction using container-and cloud-based technology to teach data science}, - volume = {Part F1287}, - isbn = {978-1-4503-5272-7}, - doi = {10.1145/3093338.3093370}, - abstract = {© 2017 ACM. There is an increasing interest in learning outside of the traditional classroom setting. This is especially true for topics covering computational tools and data science, as both are challenging to incorporate in the standard curriculum. These atypical learning environments offer new opportunities for teaching, particularly when it comes to combining conceptual knowledge with hands-on experience/expertise with methods and skills. Advances in cloud computing and containerized environments provide an attractive opportunity to improve the effciency and ease with which students can learn. This manuscript details recent advances towards using commonly-Available cloud computing services and advanced cyberinfrastructure support for improving the learning experience in bootcamp-style events. We cover the benets (and challenges) of using a server hosted remotely instead of relying on student laptops, discuss the technology that was used in order to make this possible, and give suggestions for how others could implement and improve upon this model for pedagogy and reproducibility.}, - booktitle = {{ACM} {International} {Conference} {Proceeding} {Series}}, - author = {Holdgraf, Christopher Ramsay and Culich, A. and Rokem, A. and Deniz, F. and Alegro, M. and Ushizima, D.}, - year = {2017}, - keywords = {Teaching, Bootcamps, Cloud computing, Data science, Docker, Pedagogy} -} - -@article{holdgraf_encoding_2017, - title = {Encoding and decoding models in cognitive electrophysiology}, - volume = {11}, - issn = {16625137}, - doi = {10.3389/fnsys.2017.00061}, - abstract = {© 2017 Holdgraf, Rieger, Micheli, Martin, Knight and Theunissen. Cognitive neuroscience has seen rapid growth in the size and complexity of data recorded from the human brain as well as in the computational tools available to analyze this data. This data explosion has resulted in an increased use of multivariate, model-based methods for asking neuroscience questions, allowing scientists to investigate multiple hypotheses with a single dataset, to use complex, time-varying stimuli, and to study the human brain under more naturalistic conditions. These tools come in the form of “Encoding” models, in which stimulus features are used to model brain activity, and “Decoding” models, in which neural features are used to generated a stimulus output. Here we review the current state of encoding and decoding models in cognitive electrophysiology and provide a practical guide toward conducting experiments and analyses in this emerging field. Our examples focus on using linear models in the study of human language and audition. We show how to calculate auditory receptive fields from natural sounds as well as how to decode neural recordings to predict speech. The paper aims to be a useful tutorial to these approaches, and a practical introduction to using machine learning and applied statistics to build models of neural activity. The data analytic approaches we discuss may also be applied to other sensory modalities, motor systems, and cognitive systems, and we cover some examples in these areas. In addition, a collection of Jupyter notebooks is publicly available as a complement to the material covered in this paper, providing code examples and tutorials for predictive modeling in python. The aimis to provide a practical understanding of predictivemodeling of human brain data and to propose best-practices in conducting these analyses.}, - journal = {Frontiers in Systems Neuroscience}, - author = {Holdgraf, Christopher Ramsay and Rieger, J.W. and Micheli, C. and Martin, S. and Knight, R.T. and Theunissen, F.E.}, - year = {2017}, - keywords = {Decoding models, Encoding models, Electrocorticography (ECoG), Electrophysiology/evoked potentials, Machine learning applied to neuroscience, Natural stimuli, Predictive modeling, Tutorials} -} - -@book{ruby, - title = {The Ruby Programming Language}, - author = {Flanagan, David and Matsumoto, Yukihiro}, - year = {2008}, - publisher = {O'Reilly Media} -} diff --git a/contribution.md b/docs/contribution.md similarity index 100% rename from contribution.md rename to docs/contribution.md diff --git a/index.md b/docs/index.md similarity index 73% rename from index.md rename to docs/index.md index 7140ec9b..47e710a2 100644 --- a/index.md +++ b/docs/index.md @@ -1,6 +1,6 @@ # Welcome to BARS -BARS is a large, open benchmark for recommender systems: https://openbenchmark.github.io/BARS +BARS is a project aimed for open BenchmArking for Recommender Systems https://openbenchmark.github.io/BARS Despite the significant progress made in both research and practice of recommender systems over the past two decades, there is a lack of a widely-recognized benchmark in this field. This not only increases the difficulty in reproducing existing studies, but also incurs inconsistent experimental results among them, which largely limit the practical value and potential impact of research in this field. @@ -22,8 +22,7 @@ In summary, BARS is built with the following key features: + **Strong maintainability**: BARS is open to the community. Anyone can contribute new datasets, implement new models, update new benchmarking results, or revise existing numbers through a pull request on Github. The contributor will be marked honorably along with the contributions accordingly. - -```{figure} docs/img/pipeline.jpg +```{figure} img/pipeline.jpg --- scale: 10% alt : "pipeline.jpg" @@ -38,26 +37,26 @@ By establishing an open benchmarking standard, together with the freely availabl The BARS benchmark currently covers the following two main recommendation tasks. -+ [BarsCTR: A Benchmark for CTR Prediction](https://openbenchmark.github.io/BARS/ctr_prediction) -+ [BarsMatch: A Benchmark for Candidate Item Matching](https://openbenchmark.github.io/BARS/candidate_matching) ++ [BARS-CTR: An Open Benchmark for CTR Prediction](https://openbenchmark.github.io/BARS/CTR) ++ [BARS-Match: An Open Benchmark for Candidate Item Matching](https://openbenchmark.github.io/BARS/Matching) Ongoing projects: -+ BarsRank: A Benchmark for Listwise Re-ranking -+ BarsMTL: A Benchmark for Multi-Task Recommendation ++ BARS-Rerank: An Open Benchmark for Listwise Re-ranking ++ BARS-MTL: An Open Benchmark for Multi-Task Ranking ## Citation -*:bell: If you find our benchmarks helpful in your research, please kindly cite the following papers.* +If you find our benchmarks helpful in your research, please kindly cite the following papers. -> Jieming Zhu, Quanyu Dai, Liangcai Su, Rong Ma, Jinyang Liu, Guohao Cai, Xi Xiao, Rui Zhang. [BARS: Towards Open Benchmarking for Recommender Systems](https://arxiv.org/abs/2205.09626). *The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)*, 2022. [[Bibtex](https://dblp.org/rec/conf/sigir/ZhuDSMLCXZ22.html?view=bibtex)] ++ Jieming Zhu, Quanyu Dai, Liangcai Su, Rong Ma, Jinyang Liu, Guohao Cai, Xi Xiao, Rui Zhang. [BARS: Towards Open Benchmarking for Recommender Systems](https://arxiv.org/abs/2205.09626). The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2022. [[Bibtex](https://dblp.org/rec/conf/sigir/ZhuDSMLCXZ22.html?view=bibtex)] ## Discussion -If you have any questions or feedback about the BARS benchamrk, please [start a discussion here](https://github.com/openbenchmark/BARS/discussions/new), or join our WeChat group. +If you have any questions or feedback about the BARS benchamrk, please [start a discussion here](https://github.com/reczoo/BARS/discussions/new), or join our WeChat group. - + diff --git a/matching/README.md b/matching/README.md new file mode 100644 index 00000000..05ec1aad --- /dev/null +++ b/matching/README.md @@ -0,0 +1,24 @@ +# BARS-Match + +BARS-Match: An Open Benchmark for Candidate Item Matching https://openbenchmark.github.io/BARS/Matching + +Recommender systems generally comprise two main stages, matching and ranking. As the first-stage task, candidate item matching is designed to efficiently retrieve hundreds of item candidates out of the entire item corpus. Representative methods of candidate item matching include collaborative filtering, two-tower models, autoencoder-based models, sequential models, graph-based models, etc. To drive research in this direction, the BARS project aims to build an open benchmark for candidate item matching. + +## Model List + +The following models have been benchmarked with open-source code and detailed reproducing steps. + +| Publication | Model | Paper Title | +| ----:|:----------:|:--------------------------------------------------------------------------------------------| +| WWW'01 | ItemCF | Item-Based Collaborative Filtering Recommendation Algorithms | +| UAI'09 | MF-BPR | BPR: Bayesian Personalized Ranking from Implicit Feedback | +| ICDM'11 | SLIM | SLIM: Sparse Linear Methods for Top-N Recommender Systems | +| RecSys'16 | YoutubeNet | Deep Neural Networks for YouTube Recommendations | +| WWW'17 | NeuMF | Neural Collaborative Filtering | +| WWW'17 | CML | Collaborative Metric Learning | +| SIGIR'19 | NGCF | Neural Graph Collaborative Filtering | +| WWW'19 | EASE^R | Embarrassingly Shallow Autoencoders for Sparse Data | +| AAAI'20 | LR-GCCF | Revisiting Graph based Collaborative Filtering: A Linear Residual Graph Convolutional Network Approach | +| SIGIR'20 | LightGCN | LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation | +| TOIS'20 | ENMF | Efficient Neural Matrix Factorization without Sampling for Recommendation | +| CIKM'21 | SimpleX | SimpleX: A Simple and Strong Baseline for Collaborative Filtering | diff --git a/candidate_matching/benchmarks/CML/CML_amazonbooks_x1/README.md b/matching/cf/CML/CML_amazonbooks_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/CML/CML_amazonbooks_x1/README.md rename to matching/cf/CML/CML_amazonbooks_m1/README.md diff --git a/candidate_matching/benchmarks/CML/CML_gowalla_x1/README.md b/matching/cf/CML/CML_gowalla_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/CML/CML_gowalla_x1/README.md rename to matching/cf/CML/CML_gowalla_m1/README.md diff --git a/candidate_matching/benchmarks/CML/CML_yelp18_x1/README.md b/matching/cf/CML/CML_yelp18_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/CML/CML_yelp18_x1/README.md rename to matching/cf/CML/CML_yelp18_m1/README.md diff --git a/candidate_matching/benchmarks/CML/main.py b/matching/cf/CML/main.py similarity index 100% rename from candidate_matching/benchmarks/CML/main.py rename to matching/cf/CML/main.py diff --git a/candidate_matching/benchmarks/EASE_r/EASE_amazonbooks_x1/README.md b/matching/cf/EASE_R/EASE_amazonbooks_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/EASE_r/EASE_amazonbooks_x1/README.md rename to matching/cf/EASE_R/EASE_amazonbooks_m1/README.md diff --git a/candidate_matching/benchmarks/EASE_r/EASE_gowalla_x1/README.md b/matching/cf/EASE_R/EASE_gowalla_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/EASE_r/EASE_gowalla_x1/README.md rename to matching/cf/EASE_R/EASE_gowalla_m1/README.md diff --git a/candidate_matching/benchmarks/EASE_r/EASE_yelp18_x1/README.md b/matching/cf/EASE_R/EASE_yelp18_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/EASE_r/EASE_yelp18_x1/README.md rename to matching/cf/EASE_R/EASE_yelp18_m1/README.md diff --git a/candidate_matching/benchmarks/EASE_r/main.py b/matching/cf/EASE_R/main.py similarity index 100% rename from candidate_matching/benchmarks/EASE_r/main.py rename to matching/cf/EASE_R/main.py diff --git a/candidate_matching/benchmarks/EASE_r/src/__init__.py b/matching/cf/EASE_R/src/__init__.py similarity index 100% rename from candidate_matching/benchmarks/EASE_r/src/__init__.py rename to matching/cf/EASE_R/src/__init__.py diff --git a/candidate_matching/benchmarks/EASE_r/src/ease_rec.py b/matching/cf/EASE_R/src/ease_rec.py similarity index 100% rename from candidate_matching/benchmarks/EASE_r/src/ease_rec.py rename to matching/cf/EASE_R/src/ease_rec.py diff --git a/candidate_matching/benchmarks/EASE_r/src/utils.py b/matching/cf/EASE_R/src/utils.py similarity index 100% rename from candidate_matching/benchmarks/EASE_r/src/utils.py rename to matching/cf/EASE_R/src/utils.py diff --git a/candidate_matching/benchmarks/ENMF/ENMF_amazonbooks_x1/README.md b/matching/cf/ENMF/ENMF_amazonbooks_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/ENMF/ENMF_amazonbooks_x1/README.md rename to matching/cf/ENMF/ENMF_amazonbooks_m1/README.md diff --git a/candidate_matching/benchmarks/ENMF/ENMF_gowalla_x1/README.md b/matching/cf/ENMF/ENMF_gowalla_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/ENMF/ENMF_gowalla_x1/README.md rename to matching/cf/ENMF/ENMF_gowalla_m1/README.md diff --git a/candidate_matching/benchmarks/ENMF/ENMF_yelp18_x1/README.md b/matching/cf/ENMF/ENMF_yelp18_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/ENMF/ENMF_yelp18_x1/README.md rename to matching/cf/ENMF/ENMF_yelp18_m1/README.md diff --git a/candidate_matching/benchmarks/ENMF/src/ENMF.py b/matching/cf/ENMF/src/ENMF.py similarity index 100% rename from candidate_matching/benchmarks/ENMF/src/ENMF.py rename to matching/cf/ENMF/src/ENMF.py diff --git a/candidate_matching/benchmarks/ENMF/src/ENMF_utils.py b/matching/cf/ENMF/src/ENMF_utils.py similarity index 100% rename from candidate_matching/benchmarks/ENMF/src/ENMF_utils.py rename to matching/cf/ENMF/src/ENMF_utils.py diff --git a/candidate_matching/benchmarks/ENMF/src/README.md b/matching/cf/ENMF/src/README.md similarity index 100% rename from candidate_matching/benchmarks/ENMF/src/README.md rename to matching/cf/ENMF/src/README.md diff --git a/candidate_matching/benchmarks/MF_BPR/MF_BPR_amazonbooks_x1/README.md b/matching/cf/MF_BPR/MF_BPR_amazonbooks_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/MF_BPR/MF_BPR_amazonbooks_x1/README.md rename to matching/cf/MF_BPR/MF_BPR_amazonbooks_m1/README.md diff --git a/candidate_matching/benchmarks/MF_BPR/MF_BPR_gowalla_x1/README.md b/matching/cf/MF_BPR/MF_BPR_gowalla_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/MF_BPR/MF_BPR_gowalla_x1/README.md rename to matching/cf/MF_BPR/MF_BPR_gowalla_m1/README.md diff --git a/candidate_matching/benchmarks/MF_BPR/MF_BPR_yelp18_x1/README.md b/matching/cf/MF_BPR/MF_BPR_yelp18_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/MF_BPR/MF_BPR_yelp18_x1/README.md rename to matching/cf/MF_BPR/MF_BPR_yelp18_m1/README.md diff --git a/candidate_matching/benchmarks/MF_CCL/MF_CCL_amazonbooks_x1/README.md b/matching/cf/MF_CCL/MF_CCL_amazonbooks_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/MF_CCL/MF_CCL_amazonbooks_x1/README.md rename to matching/cf/MF_CCL/MF_CCL_amazonbooks_m1/README.md diff --git a/candidate_matching/benchmarks/MF_CCL/MF_CCL_gowalla_x1/README.md b/matching/cf/MF_CCL/MF_CCL_gowalla_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/MF_CCL/MF_CCL_gowalla_x1/README.md rename to matching/cf/MF_CCL/MF_CCL_gowalla_m1/README.md diff --git a/candidate_matching/benchmarks/MF_CCL/MF_CCL_yelp18_x1/README.md b/matching/cf/MF_CCL/MF_CCL_yelp18_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/MF_CCL/MF_CCL_yelp18_x1/README.md rename to matching/cf/MF_CCL/MF_CCL_yelp18_m1/README.md diff --git a/candidate_matching/benchmarks/SLIM/SLIM.py b/matching/cf/SLIM/SLIM.py similarity index 100% rename from candidate_matching/benchmarks/SLIM/SLIM.py rename to matching/cf/SLIM/SLIM.py diff --git a/candidate_matching/benchmarks/SLIM/SLIM_amazonbooks_x1/README.md b/matching/cf/SLIM/SLIM_amazonbooks_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/SLIM/SLIM_amazonbooks_x1/README.md rename to matching/cf/SLIM/SLIM_amazonbooks_m1/README.md diff --git a/candidate_matching/benchmarks/SLIM/SLIM_gowalla_x1/README.md b/matching/cf/SLIM/SLIM_gowalla_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/SLIM/SLIM_gowalla_x1/README.md rename to matching/cf/SLIM/SLIM_gowalla_m1/README.md diff --git a/candidate_matching/benchmarks/SLIM/SLIM_yelp18_x1/README.md b/matching/cf/SLIM/SLIM_yelp18_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/SLIM/SLIM_yelp18_x1/README.md rename to matching/cf/SLIM/SLIM_yelp18_m1/README.md diff --git a/candidate_matching/benchmarks/SimpleX/SimpleX_amazonbooks_x1/README.md b/matching/cf/SimpleX/SimpleX_amazonbooks_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/SimpleX/SimpleX_amazonbooks_x1/README.md rename to matching/cf/SimpleX/SimpleX_amazonbooks_m1/README.md diff --git a/candidate_matching/benchmarks/SimpleX/SimpleX_gowalla_x1/README.md b/matching/cf/SimpleX/SimpleX_gowalla_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/SimpleX/SimpleX_gowalla_x1/README.md rename to matching/cf/SimpleX/SimpleX_gowalla_m1/README.md diff --git a/candidate_matching/benchmarks/SimpleX/SimpleX_yelp18_x1/README.md b/matching/cf/SimpleX/SimpleX_yelp18_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/SimpleX/SimpleX_yelp18_x1/README.md rename to matching/cf/SimpleX/SimpleX_yelp18_m1/README.md diff --git a/candidate_matching/benchmarks/LR_GCCF/LR_GCCF.py b/matching/gnn/LR_GCCF/LR_GCCF.py similarity index 100% rename from candidate_matching/benchmarks/LR_GCCF/LR_GCCF.py rename to matching/gnn/LR_GCCF/LR_GCCF.py diff --git a/candidate_matching/benchmarks/LR_GCCF/LR_GCCF_amazonbooks_x1/README.md b/matching/gnn/LR_GCCF/LR_GCCF_amazonbooks_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/LR_GCCF/LR_GCCF_amazonbooks_x1/README.md rename to matching/gnn/LR_GCCF/LR_GCCF_amazonbooks_m1/README.md diff --git a/candidate_matching/benchmarks/LR_GCCF/LR_GCCF_gowalla_x1/README.md b/matching/gnn/LR_GCCF/LR_GCCF_gowalla_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/LR_GCCF/LR_GCCF_gowalla_x1/README.md rename to matching/gnn/LR_GCCF/LR_GCCF_gowalla_m1/README.md diff --git a/candidate_matching/benchmarks/LR_GCCF/LR_GCCF_yelp18_x1/README.md b/matching/gnn/LR_GCCF/LR_GCCF_yelp18_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/LR_GCCF/LR_GCCF_yelp18_x1/README.md rename to matching/gnn/LR_GCCF/LR_GCCF_yelp18_m1/README.md diff --git a/candidate_matching/benchmarks/LightGCN/LightGCN.py b/matching/gnn/LightGCN/LightGCN.py similarity index 100% rename from candidate_matching/benchmarks/LightGCN/LightGCN.py rename to matching/gnn/LightGCN/LightGCN.py diff --git a/candidate_matching/benchmarks/LightGCN/LightGCN_amazonbooks_x1/README.md b/matching/gnn/LightGCN/LightGCN_amazonbooks_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/LightGCN/LightGCN_amazonbooks_x1/README.md rename to matching/gnn/LightGCN/LightGCN_amazonbooks_m1/README.md diff --git a/candidate_matching/benchmarks/LightGCN/LightGCN_gowalla_x1/README.md b/matching/gnn/LightGCN/LightGCN_gowalla_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/LightGCN/LightGCN_gowalla_x1/README.md rename to matching/gnn/LightGCN/LightGCN_gowalla_m1/README.md diff --git a/candidate_matching/benchmarks/LightGCN/LightGCN_yelp18_x1/README.md b/matching/gnn/LightGCN/LightGCN_yelp18_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/LightGCN/LightGCN_yelp18_x1/README.md rename to matching/gnn/LightGCN/LightGCN_yelp18_m1/README.md diff --git a/candidate_matching/benchmarks/Item2Vec/Item2Vec.py b/matching/item2item/Item2Vec/Item2Vec.py similarity index 100% rename from candidate_matching/benchmarks/Item2Vec/Item2Vec.py rename to matching/item2item/Item2Vec/Item2Vec.py diff --git a/candidate_matching/benchmarks/Item2Vec/Item2Vec_amazonbooks_x1/README.md b/matching/item2item/Item2Vec/Item2Vec_amazonbooks_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/Item2Vec/Item2Vec_amazonbooks_x1/README.md rename to matching/item2item/Item2Vec/Item2Vec_amazonbooks_m1/README.md diff --git a/candidate_matching/benchmarks/Item2Vec/Item2Vec_gowalla_x1/README.md b/matching/item2item/Item2Vec/Item2Vec_gowalla_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/Item2Vec/Item2Vec_gowalla_x1/README.md rename to matching/item2item/Item2Vec/Item2Vec_gowalla_m1/README.md diff --git a/candidate_matching/benchmarks/Item2Vec/Item2Vec_yelp18_x1/README.md b/matching/item2item/Item2Vec/Item2Vec_yelp18_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/Item2Vec/Item2Vec_yelp18_x1/README.md rename to matching/item2item/Item2Vec/Item2Vec_yelp18_m1/README.md diff --git a/candidate_matching/benchmarks/ItemCF/ItemCF_amazonbooks_x1/README.md b/matching/item2item/ItemCF/ItemCF_amazonbooks_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/ItemCF/ItemCF_amazonbooks_x1/README.md rename to matching/item2item/ItemCF/ItemCF_amazonbooks_m1/README.md diff --git a/candidate_matching/benchmarks/ItemCF/ItemCF_gowalla_x1/README.md b/matching/item2item/ItemCF/ItemCF_gowalla_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/ItemCF/ItemCF_gowalla_x1/README.md rename to matching/item2item/ItemCF/ItemCF_gowalla_m1/README.md diff --git a/candidate_matching/benchmarks/ItemCF/ItemCF_yelp18_x1/README.md b/matching/item2item/ItemCF/ItemCF_yelp18_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/ItemCF/ItemCF_yelp18_x1/README.md rename to matching/item2item/ItemCF/ItemCF_yelp18_m1/README.md diff --git a/candidate_matching/benchmarks/ItemCF/ItemKNN.py b/matching/item2item/ItemCF/ItemKNN.py similarity index 100% rename from candidate_matching/benchmarks/ItemCF/ItemKNN.py rename to matching/item2item/ItemCF/ItemKNN.py diff --git a/candidate_matching/benchmarks/ItemCF/ItemKNN_utils.py b/matching/item2item/ItemCF/ItemKNN_utils.py similarity index 100% rename from candidate_matching/benchmarks/ItemCF/ItemKNN_utils.py rename to matching/item2item/ItemCF/ItemKNN_utils.py diff --git a/candidate_matching/benchmarks/ItemCF/README.md b/matching/item2item/ItemCF/README.md similarity index 100% rename from candidate_matching/benchmarks/ItemCF/README.md rename to matching/item2item/ItemCF/README.md diff --git a/candidate_matching/libs/CollMetric/CML.py b/matching/libs/CollMetric/CML.py similarity index 100% rename from candidate_matching/libs/CollMetric/CML.py rename to matching/libs/CollMetric/CML.py diff --git a/candidate_matching/libs/CollMetric/CML_utils.py b/matching/libs/CollMetric/CML_utils.py similarity index 100% rename from candidate_matching/libs/CollMetric/CML_utils.py rename to matching/libs/CollMetric/CML_utils.py diff --git a/candidate_matching/libs/CollMetric/LICENSE b/matching/libs/CollMetric/LICENSE similarity index 100% rename from candidate_matching/libs/CollMetric/LICENSE rename to matching/libs/CollMetric/LICENSE diff --git a/candidate_matching/libs/CollMetric/README.md b/matching/libs/CollMetric/README.md similarity index 100% rename from candidate_matching/libs/CollMetric/README.md rename to matching/libs/CollMetric/README.md diff --git a/candidate_matching/libs/CollMetric/evaluator.py b/matching/libs/CollMetric/evaluator.py similarity index 100% rename from candidate_matching/libs/CollMetric/evaluator.py rename to matching/libs/CollMetric/evaluator.py diff --git a/candidate_matching/libs/CollMetric/requirements.txt b/matching/libs/CollMetric/requirements.txt similarity index 100% rename from candidate_matching/libs/CollMetric/requirements.txt rename to matching/libs/CollMetric/requirements.txt diff --git a/candidate_matching/libs/CollMetric/sampler.py b/matching/libs/CollMetric/sampler.py similarity index 100% rename from candidate_matching/libs/CollMetric/sampler.py rename to matching/libs/CollMetric/sampler.py diff --git a/candidate_matching/libs/CollMetric/utils.py b/matching/libs/CollMetric/utils.py similarity index 100% rename from candidate_matching/libs/CollMetric/utils.py rename to matching/libs/CollMetric/utils.py diff --git a/candidate_matching/libs/LR-GCCF/data_utils.py b/matching/libs/LR-GCCF/data_utils.py similarity index 100% rename from candidate_matching/libs/LR-GCCF/data_utils.py rename to matching/libs/LR-GCCF/data_utils.py diff --git a/candidate_matching/libs/LR-GCCF/evaluate.py b/matching/libs/LR-GCCF/evaluate.py similarity index 100% rename from candidate_matching/libs/LR-GCCF/evaluate.py rename to matching/libs/LR-GCCF/evaluate.py diff --git a/candidate_matching/libs/LR-GCCF/parser_my.py b/matching/libs/LR-GCCF/parser_my.py similarity index 100% rename from candidate_matching/libs/LR-GCCF/parser_my.py rename to matching/libs/LR-GCCF/parser_my.py diff --git a/candidate_matching/libs/LR-GCCF/test_amazons.py b/matching/libs/LR-GCCF/test_amazons.py similarity index 100% rename from candidate_matching/libs/LR-GCCF/test_amazons.py rename to matching/libs/LR-GCCF/test_amazons.py diff --git a/candidate_matching/libs/LR-GCCF/test_gowalla.py b/matching/libs/LR-GCCF/test_gowalla.py similarity index 100% rename from candidate_matching/libs/LR-GCCF/test_gowalla.py rename to matching/libs/LR-GCCF/test_gowalla.py diff --git a/candidate_matching/libs/LR-GCCF/test_yelp.py b/matching/libs/LR-GCCF/test_yelp.py similarity index 100% rename from candidate_matching/libs/LR-GCCF/test_yelp.py rename to matching/libs/LR-GCCF/test_yelp.py diff --git a/candidate_matching/libs/LR-GCCF/train_amazons.py b/matching/libs/LR-GCCF/train_amazons.py similarity index 100% rename from candidate_matching/libs/LR-GCCF/train_amazons.py rename to matching/libs/LR-GCCF/train_amazons.py diff --git a/candidate_matching/libs/LR-GCCF/train_gowalla.py b/matching/libs/LR-GCCF/train_gowalla.py similarity index 100% rename from candidate_matching/libs/LR-GCCF/train_gowalla.py rename to matching/libs/LR-GCCF/train_gowalla.py diff --git a/candidate_matching/libs/LR-GCCF/train_yelp.py b/matching/libs/LR-GCCF/train_yelp.py similarity index 100% rename from candidate_matching/libs/LR-GCCF/train_yelp.py rename to matching/libs/LR-GCCF/train_yelp.py diff --git a/candidate_matching/libs/LightGCN_TF/README.md b/matching/libs/LightGCN_TF/README.md similarity index 100% rename from candidate_matching/libs/LightGCN_TF/README.md rename to matching/libs/LightGCN_TF/README.md diff --git a/candidate_matching/libs/LightGCN_TF/evaluator/__init__.py b/matching/libs/LightGCN_TF/evaluator/__init__.py similarity index 100% rename from candidate_matching/libs/LightGCN_TF/evaluator/__init__.py rename to matching/libs/LightGCN_TF/evaluator/__init__.py diff --git a/candidate_matching/libs/LightGCN_TF/evaluator/cpp/apt_evaluate_foldout.pyx b/matching/libs/LightGCN_TF/evaluator/cpp/apt_evaluate_foldout.pyx similarity index 100% rename from candidate_matching/libs/LightGCN_TF/evaluator/cpp/apt_evaluate_foldout.pyx rename to matching/libs/LightGCN_TF/evaluator/cpp/apt_evaluate_foldout.pyx diff --git a/candidate_matching/libs/LightGCN_TF/evaluator/cpp/apt_tools.pyx b/matching/libs/LightGCN_TF/evaluator/cpp/apt_tools.pyx similarity index 100% rename from candidate_matching/libs/LightGCN_TF/evaluator/cpp/apt_tools.pyx rename to matching/libs/LightGCN_TF/evaluator/cpp/apt_tools.pyx diff --git a/candidate_matching/libs/LightGCN_TF/evaluator/cpp/evaluate_foldout.py b/matching/libs/LightGCN_TF/evaluator/cpp/evaluate_foldout.py similarity index 100% rename from candidate_matching/libs/LightGCN_TF/evaluator/cpp/evaluate_foldout.py rename to matching/libs/LightGCN_TF/evaluator/cpp/evaluate_foldout.py diff --git a/candidate_matching/libs/LightGCN_TF/evaluator/cpp/include/evaluate_foldout.h b/matching/libs/LightGCN_TF/evaluator/cpp/include/evaluate_foldout.h similarity index 100% rename from candidate_matching/libs/LightGCN_TF/evaluator/cpp/include/evaluate_foldout.h rename to matching/libs/LightGCN_TF/evaluator/cpp/include/evaluate_foldout.h diff --git a/candidate_matching/libs/LightGCN_TF/evaluator/cpp/include/thread_pool.h b/matching/libs/LightGCN_TF/evaluator/cpp/include/thread_pool.h similarity index 100% rename from candidate_matching/libs/LightGCN_TF/evaluator/cpp/include/thread_pool.h rename to matching/libs/LightGCN_TF/evaluator/cpp/include/thread_pool.h diff --git a/candidate_matching/libs/LightGCN_TF/evaluator/cpp/include/tools.h b/matching/libs/LightGCN_TF/evaluator/cpp/include/tools.h similarity index 100% rename from candidate_matching/libs/LightGCN_TF/evaluator/cpp/include/tools.h rename to matching/libs/LightGCN_TF/evaluator/cpp/include/tools.h diff --git a/candidate_matching/libs/LightGCN_TF/evaluator/python/evaluate_foldout.py b/matching/libs/LightGCN_TF/evaluator/python/evaluate_foldout.py similarity index 100% rename from candidate_matching/libs/LightGCN_TF/evaluator/python/evaluate_foldout.py rename to matching/libs/LightGCN_TF/evaluator/python/evaluate_foldout.py diff --git a/candidate_matching/libs/LightGCN_TF/evaluator/python/evaluate_loo.py b/matching/libs/LightGCN_TF/evaluator/python/evaluate_loo.py similarity index 100% rename from candidate_matching/libs/LightGCN_TF/evaluator/python/evaluate_loo.py rename to matching/libs/LightGCN_TF/evaluator/python/evaluate_loo.py diff --git a/candidate_matching/libs/LightGCN_TF/lightgcn_tf.py b/matching/libs/LightGCN_TF/lightgcn_tf.py similarity index 100% rename from candidate_matching/libs/LightGCN_TF/lightgcn_tf.py rename to matching/libs/LightGCN_TF/lightgcn_tf.py diff --git a/candidate_matching/libs/LightGCN_TF/setup.py b/matching/libs/LightGCN_TF/setup.py similarity index 100% rename from candidate_matching/libs/LightGCN_TF/setup.py rename to matching/libs/LightGCN_TF/setup.py diff --git a/candidate_matching/libs/LightGCN_TF/utility/README.md b/matching/libs/LightGCN_TF/utility/README.md similarity index 100% rename from candidate_matching/libs/LightGCN_TF/utility/README.md rename to matching/libs/LightGCN_TF/utility/README.md diff --git a/candidate_matching/libs/LightGCN_TF/utility/batch_test.py b/matching/libs/LightGCN_TF/utility/batch_test.py similarity index 100% rename from candidate_matching/libs/LightGCN_TF/utility/batch_test.py rename to matching/libs/LightGCN_TF/utility/batch_test.py diff --git a/candidate_matching/libs/LightGCN_TF/utility/helper.py b/matching/libs/LightGCN_TF/utility/helper.py similarity index 100% rename from candidate_matching/libs/LightGCN_TF/utility/helper.py rename to matching/libs/LightGCN_TF/utility/helper.py diff --git a/candidate_matching/libs/LightGCN_TF/utility/load_data.py b/matching/libs/LightGCN_TF/utility/load_data.py similarity index 100% rename from candidate_matching/libs/LightGCN_TF/utility/load_data.py rename to matching/libs/LightGCN_TF/utility/load_data.py diff --git a/candidate_matching/libs/LightGCN_TF/utility/parser.py b/matching/libs/LightGCN_TF/utility/parser.py similarity index 100% rename from candidate_matching/libs/LightGCN_TF/utility/parser.py rename to matching/libs/LightGCN_TF/utility/parser.py diff --git a/candidate_matching/libs/README.md b/matching/libs/README.md similarity index 100% rename from candidate_matching/libs/README.md rename to matching/libs/README.md diff --git a/candidate_matching/libs/daisyRec/LICENSE b/matching/libs/daisyRec/LICENSE similarity index 100% rename from candidate_matching/libs/daisyRec/LICENSE rename to matching/libs/daisyRec/LICENSE diff --git a/candidate_matching/libs/daisyRec/daisy/model/CDAERecommender.py b/matching/libs/daisyRec/daisy/model/CDAERecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/CDAERecommender.py rename to matching/libs/daisyRec/daisy/model/CDAERecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/Item2VecRecommender.py b/matching/libs/daisyRec/daisy/model/Item2VecRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/Item2VecRecommender.py rename to matching/libs/daisyRec/daisy/model/Item2VecRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/KNNCFRecommender.py b/matching/libs/daisyRec/daisy/model/KNNCFRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/KNNCFRecommender.py rename to matching/libs/daisyRec/daisy/model/KNNCFRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/PopRecommender.py b/matching/libs/daisyRec/daisy/model/PopRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/PopRecommender.py rename to matching/libs/daisyRec/daisy/model/PopRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/PureSVDRecommender.py b/matching/libs/daisyRec/daisy/model/PureSVDRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/PureSVDRecommender.py rename to matching/libs/daisyRec/daisy/model/PureSVDRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/SLiMRecommender.py b/matching/libs/daisyRec/daisy/model/SLiMRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/SLiMRecommender.py rename to matching/libs/daisyRec/daisy/model/SLiMRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/VAERecommender.py b/matching/libs/daisyRec/daisy/model/VAERecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/VAERecommender.py rename to matching/libs/daisyRec/daisy/model/VAERecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/WRMFRecommender.py b/matching/libs/daisyRec/daisy/model/WRMFRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/WRMFRecommender.py rename to matching/libs/daisyRec/daisy/model/WRMFRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/extensions/matrix_factorization.pyx b/matching/libs/daisyRec/daisy/model/extensions/matrix_factorization.pyx similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/extensions/matrix_factorization.pyx rename to matching/libs/daisyRec/daisy/model/extensions/matrix_factorization.pyx diff --git a/candidate_matching/libs/daisyRec/daisy/model/extensions/simlib_cython.pyx b/matching/libs/daisyRec/daisy/model/extensions/simlib_cython.pyx similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/extensions/simlib_cython.pyx rename to matching/libs/daisyRec/daisy/model/extensions/simlib_cython.pyx diff --git a/candidate_matching/libs/daisyRec/daisy/model/extensions/simlib_python.py b/matching/libs/daisyRec/daisy/model/extensions/simlib_python.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/extensions/simlib_python.py rename to matching/libs/daisyRec/daisy/model/extensions/simlib_python.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/pair/AFMRecommender.py b/matching/libs/daisyRec/daisy/model/pair/AFMRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/pair/AFMRecommender.py rename to matching/libs/daisyRec/daisy/model/pair/AFMRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/pair/DeepFMRecommender.py b/matching/libs/daisyRec/daisy/model/pair/DeepFMRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/pair/DeepFMRecommender.py rename to matching/libs/daisyRec/daisy/model/pair/DeepFMRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/pair/FMRecommender.py b/matching/libs/daisyRec/daisy/model/pair/FMRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/pair/FMRecommender.py rename to matching/libs/daisyRec/daisy/model/pair/FMRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/pair/MFRecommender.py b/matching/libs/daisyRec/daisy/model/pair/MFRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/pair/MFRecommender.py rename to matching/libs/daisyRec/daisy/model/pair/MFRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/pair/NFMRecommender.py b/matching/libs/daisyRec/daisy/model/pair/NFMRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/pair/NFMRecommender.py rename to matching/libs/daisyRec/daisy/model/pair/NFMRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/pair/NeuMFRecommender.py b/matching/libs/daisyRec/daisy/model/pair/NeuMFRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/pair/NeuMFRecommender.py rename to matching/libs/daisyRec/daisy/model/pair/NeuMFRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/point/AFMRecommender.py b/matching/libs/daisyRec/daisy/model/point/AFMRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/point/AFMRecommender.py rename to matching/libs/daisyRec/daisy/model/point/AFMRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/point/DeepFMRecommender.py b/matching/libs/daisyRec/daisy/model/point/DeepFMRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/point/DeepFMRecommender.py rename to matching/libs/daisyRec/daisy/model/point/DeepFMRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/point/FMRecommender.py b/matching/libs/daisyRec/daisy/model/point/FMRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/point/FMRecommender.py rename to matching/libs/daisyRec/daisy/model/point/FMRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/point/MFRecommender.py b/matching/libs/daisyRec/daisy/model/point/MFRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/point/MFRecommender.py rename to matching/libs/daisyRec/daisy/model/point/MFRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/point/NFMRecommender.py b/matching/libs/daisyRec/daisy/model/point/NFMRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/point/NFMRecommender.py rename to matching/libs/daisyRec/daisy/model/point/NFMRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/model/point/NeuMFRecommender.py b/matching/libs/daisyRec/daisy/model/point/NeuMFRecommender.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/model/point/NeuMFRecommender.py rename to matching/libs/daisyRec/daisy/model/point/NeuMFRecommender.py diff --git a/candidate_matching/libs/daisyRec/daisy/utils/data.py b/matching/libs/daisyRec/daisy/utils/data.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/utils/data.py rename to matching/libs/daisyRec/daisy/utils/data.py diff --git a/candidate_matching/libs/daisyRec/daisy/utils/generator.py b/matching/libs/daisyRec/daisy/utils/generator.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/utils/generator.py rename to matching/libs/daisyRec/daisy/utils/generator.py diff --git a/candidate_matching/libs/daisyRec/daisy/utils/loader.py b/matching/libs/daisyRec/daisy/utils/loader.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/utils/loader.py rename to matching/libs/daisyRec/daisy/utils/loader.py diff --git a/candidate_matching/libs/daisyRec/daisy/utils/metrics.py b/matching/libs/daisyRec/daisy/utils/metrics.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/utils/metrics.py rename to matching/libs/daisyRec/daisy/utils/metrics.py diff --git a/candidate_matching/libs/daisyRec/daisy/utils/parser.py b/matching/libs/daisyRec/daisy/utils/parser.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/utils/parser.py rename to matching/libs/daisyRec/daisy/utils/parser.py diff --git a/candidate_matching/libs/daisyRec/daisy/utils/sampler.py b/matching/libs/daisyRec/daisy/utils/sampler.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/utils/sampler.py rename to matching/libs/daisyRec/daisy/utils/sampler.py diff --git a/candidate_matching/libs/daisyRec/daisy/utils/splitter.py b/matching/libs/daisyRec/daisy/utils/splitter.py similarity index 100% rename from candidate_matching/libs/daisyRec/daisy/utils/splitter.py rename to matching/libs/daisyRec/daisy/utils/splitter.py diff --git a/candidate_matching/benchmarks/YoutubeDNN/YoutubeDNN_amazonbooks_x1/README.md b/matching/two_tower/YoutubeDNN/YoutubeDNN_amazonbooks_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/YoutubeDNN/YoutubeDNN_amazonbooks_x1/README.md rename to matching/two_tower/YoutubeDNN/YoutubeDNN_amazonbooks_m1/README.md diff --git a/candidate_matching/benchmarks/YoutubeDNN/YoutubeDNN_gowalla_x1/README.md b/matching/two_tower/YoutubeDNN/YoutubeDNN_gowalla_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/YoutubeDNN/YoutubeDNN_gowalla_x1/README.md rename to matching/two_tower/YoutubeDNN/YoutubeDNN_gowalla_m1/README.md diff --git a/candidate_matching/benchmarks/YoutubeDNN/YoutubeDNN_yelp18_x1/README.md b/matching/two_tower/YoutubeDNN/YoutubeDNN_yelp18_m1/README.md similarity index 100% rename from candidate_matching/benchmarks/YoutubeDNN/YoutubeDNN_yelp18_x1/README.md rename to matching/two_tower/YoutubeDNN/YoutubeDNN_yelp18_m1/README.md diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x1/AFM_avazu_x1_002_4a58edb9.log b/ranking/ctr/AFM/AFM_avazu_x1/AFM_avazu_x1_002_4a58edb9.log similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x1/AFM_avazu_x1_002_4a58edb9.log rename to ranking/ctr/AFM/AFM_avazu_x1/AFM_avazu_x1_002_4a58edb9.log diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x1/AFM_avazu_x1_tuner_config_03.csv b/ranking/ctr/AFM/AFM_avazu_x1/AFM_avazu_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x1/AFM_avazu_x1_tuner_config_03.csv rename to ranking/ctr/AFM/AFM_avazu_x1/AFM_avazu_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x1/AFM_avazu_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/AFM/AFM_avazu_x1/AFM_avazu_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x1/AFM_avazu_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/AFM/AFM_avazu_x1/AFM_avazu_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x1/AFM_avazu_x1_tuner_config_03/model_config.yaml b/ranking/ctr/AFM/AFM_avazu_x1/AFM_avazu_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x1/AFM_avazu_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/AFM/AFM_avazu_x1/AFM_avazu_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x1/README.md b/ranking/ctr/AFM/AFM_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x1/README.md rename to ranking/ctr/AFM/AFM_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x1/environments.txt b/ranking/ctr/AFM/AFM_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x1/environments.txt rename to ranking/ctr/AFM/AFM_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x1/results.csv b/ranking/ctr/AFM/AFM_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x1/results.csv rename to ranking/ctr/AFM/AFM_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x1/run_expid.py b/ranking/ctr/AFM/AFM_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x1/run_expid.py rename to ranking/ctr/AFM/AFM_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x4_001/AFM_avazu_x4_3bbbc4c9_008_8bd19e2a.log b/ranking/ctr/AFM/AFM_avazu_x4_001/AFM_avazu_x4_3bbbc4c9_008_8bd19e2a.log similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x4_001/AFM_avazu_x4_3bbbc4c9_008_8bd19e2a.log rename to ranking/ctr/AFM/AFM_avazu_x4_001/AFM_avazu_x4_3bbbc4c9_008_8bd19e2a.log diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x4_001/AFM_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/AFM/AFM_avazu_x4_001/AFM_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x4_001/AFM_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/AFM/AFM_avazu_x4_001/AFM_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x4_001/AFM_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/AFM/AFM_avazu_x4_001/AFM_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x4_001/AFM_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/AFM/AFM_avazu_x4_001/AFM_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x4_001/README.md b/ranking/ctr/AFM/AFM_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x4_001/README.md rename to ranking/ctr/AFM/AFM_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x4_001/environments.txt b/ranking/ctr/AFM/AFM_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x4_001/environments.txt rename to ranking/ctr/AFM/AFM_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x4_001/results.csv b/ranking/ctr/AFM/AFM_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x4_001/results.csv rename to ranking/ctr/AFM/AFM_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x4_001/run_expid.py b/ranking/ctr/AFM/AFM_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x4_001/run_expid.py rename to ranking/ctr/AFM/AFM_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x4_002/AFM_avazu_x4_009_50fcacf1.log b/ranking/ctr/AFM/AFM_avazu_x4_002/AFM_avazu_x4_009_50fcacf1.log similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x4_002/AFM_avazu_x4_009_50fcacf1.log rename to ranking/ctr/AFM/AFM_avazu_x4_002/AFM_avazu_x4_009_50fcacf1.log diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x4_002/AFM_avazu_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/AFM/AFM_avazu_x4_002/AFM_avazu_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x4_002/AFM_avazu_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/AFM/AFM_avazu_x4_002/AFM_avazu_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x4_002/AFM_avazu_x4_tuner_config_02/model_config.yaml b/ranking/ctr/AFM/AFM_avazu_x4_002/AFM_avazu_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x4_002/AFM_avazu_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/AFM/AFM_avazu_x4_002/AFM_avazu_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x4_002/README.md b/ranking/ctr/AFM/AFM_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x4_002/README.md rename to ranking/ctr/AFM/AFM_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x4_002/environments.txt b/ranking/ctr/AFM/AFM_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x4_002/environments.txt rename to ranking/ctr/AFM/AFM_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x4_002/results.csv b/ranking/ctr/AFM/AFM_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x4_002/results.csv rename to ranking/ctr/AFM/AFM_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/AFM/AFM_avazu_x4_002/run_expid.py b/ranking/ctr/AFM/AFM_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_avazu_x4_002/run_expid.py rename to ranking/ctr/AFM/AFM_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x1/AFM_criteo_x1_004_954c0ecc.log b/ranking/ctr/AFM/AFM_criteo_x1/AFM_criteo_x1_004_954c0ecc.log similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x1/AFM_criteo_x1_004_954c0ecc.log rename to ranking/ctr/AFM/AFM_criteo_x1/AFM_criteo_x1_004_954c0ecc.log diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x1/AFM_criteo_x1_tuner_config_02.csv b/ranking/ctr/AFM/AFM_criteo_x1/AFM_criteo_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x1/AFM_criteo_x1_tuner_config_02.csv rename to ranking/ctr/AFM/AFM_criteo_x1/AFM_criteo_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x1/AFM_criteo_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/AFM/AFM_criteo_x1/AFM_criteo_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x1/AFM_criteo_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/AFM/AFM_criteo_x1/AFM_criteo_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x1/AFM_criteo_x1_tuner_config_02/model_config.yaml b/ranking/ctr/AFM/AFM_criteo_x1/AFM_criteo_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x1/AFM_criteo_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/AFM/AFM_criteo_x1/AFM_criteo_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x1/README.md b/ranking/ctr/AFM/AFM_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x1/README.md rename to ranking/ctr/AFM/AFM_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x1/environments.txt b/ranking/ctr/AFM/AFM_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x1/environments.txt rename to ranking/ctr/AFM/AFM_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x1/results.csv b/ranking/ctr/AFM/AFM_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x1/results.csv rename to ranking/ctr/AFM/AFM_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x1/run_expid.py b/ranking/ctr/AFM/AFM_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x1/run_expid.py rename to ranking/ctr/AFM/AFM_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x4_001/AFM_criteo_x4_5c863b0f_010_f040edb0.log b/ranking/ctr/AFM/AFM_criteo_x4_001/AFM_criteo_x4_5c863b0f_010_f040edb0.log similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x4_001/AFM_criteo_x4_5c863b0f_010_f040edb0.log rename to ranking/ctr/AFM/AFM_criteo_x4_001/AFM_criteo_x4_5c863b0f_010_f040edb0.log diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x4_001/AFM_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/AFM/AFM_criteo_x4_001/AFM_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x4_001/AFM_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/AFM/AFM_criteo_x4_001/AFM_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x4_001/AFM_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/AFM/AFM_criteo_x4_001/AFM_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x4_001/AFM_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/AFM/AFM_criteo_x4_001/AFM_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x4_001/README.md b/ranking/ctr/AFM/AFM_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x4_001/README.md rename to ranking/ctr/AFM/AFM_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x4_001/environments.txt b/ranking/ctr/AFM/AFM_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x4_001/environments.txt rename to ranking/ctr/AFM/AFM_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x4_001/results.csv b/ranking/ctr/AFM/AFM_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x4_001/results.csv rename to ranking/ctr/AFM/AFM_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x4_001/run_expid.py b/ranking/ctr/AFM/AFM_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x4_001/run_expid.py rename to ranking/ctr/AFM/AFM_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x4_002/AFM_criteo_x4_011_4bd97d26.log b/ranking/ctr/AFM/AFM_criteo_x4_002/AFM_criteo_x4_011_4bd97d26.log similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x4_002/AFM_criteo_x4_011_4bd97d26.log rename to ranking/ctr/AFM/AFM_criteo_x4_002/AFM_criteo_x4_011_4bd97d26.log diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x4_002/AFM_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/AFM/AFM_criteo_x4_002/AFM_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x4_002/AFM_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/AFM/AFM_criteo_x4_002/AFM_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x4_002/AFM_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/AFM/AFM_criteo_x4_002/AFM_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x4_002/AFM_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/AFM/AFM_criteo_x4_002/AFM_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x4_002/README.md b/ranking/ctr/AFM/AFM_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x4_002/README.md rename to ranking/ctr/AFM/AFM_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x4_002/environments.txt b/ranking/ctr/AFM/AFM_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x4_002/environments.txt rename to ranking/ctr/AFM/AFM_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x4_002/results.csv b/ranking/ctr/AFM/AFM_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x4_002/results.csv rename to ranking/ctr/AFM/AFM_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/AFM/AFM_criteo_x4_002/run_expid.py b/ranking/ctr/AFM/AFM_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_criteo_x4_002/run_expid.py rename to ranking/ctr/AFM/AFM_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/AFM/AFM_frappe_x1/AFM_frappe_x1_011_2d591f68.log b/ranking/ctr/AFM/AFM_frappe_x1/AFM_frappe_x1_011_2d591f68.log similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_frappe_x1/AFM_frappe_x1_011_2d591f68.log rename to ranking/ctr/AFM/AFM_frappe_x1/AFM_frappe_x1_011_2d591f68.log diff --git a/ctr_prediction/benchmarks/AFM/AFM_frappe_x1/AFM_frappe_x1_tuner_config_02.csv b/ranking/ctr/AFM/AFM_frappe_x1/AFM_frappe_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_frappe_x1/AFM_frappe_x1_tuner_config_02.csv rename to ranking/ctr/AFM/AFM_frappe_x1/AFM_frappe_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/AFM/AFM_frappe_x1/AFM_frappe_x1_tuner_config_02/dataset_config (5).yaml b/ranking/ctr/AFM/AFM_frappe_x1/AFM_frappe_x1_tuner_config_02/dataset_config (5).yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_frappe_x1/AFM_frappe_x1_tuner_config_02/dataset_config (5).yaml rename to ranking/ctr/AFM/AFM_frappe_x1/AFM_frappe_x1_tuner_config_02/dataset_config (5).yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_frappe_x1/AFM_frappe_x1_tuner_config_02/model_config (10).yaml b/ranking/ctr/AFM/AFM_frappe_x1/AFM_frappe_x1_tuner_config_02/model_config (10).yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_frappe_x1/AFM_frappe_x1_tuner_config_02/model_config (10).yaml rename to ranking/ctr/AFM/AFM_frappe_x1/AFM_frappe_x1_tuner_config_02/model_config (10).yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_frappe_x1/README.md b/ranking/ctr/AFM/AFM_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_frappe_x1/README.md rename to ranking/ctr/AFM/AFM_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/AFM/AFM_frappe_x1/environments.txt b/ranking/ctr/AFM/AFM_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_frappe_x1/environments.txt rename to ranking/ctr/AFM/AFM_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AFM/AFM_frappe_x1/results.csv b/ranking/ctr/AFM/AFM_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_frappe_x1/results.csv rename to ranking/ctr/AFM/AFM_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/AFM/AFM_frappe_x1/run_expid.py b/ranking/ctr/AFM/AFM_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_frappe_x1/run_expid.py rename to ranking/ctr/AFM/AFM_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AFM/AFM_kkbox_x1/AFM_kkbox_x1_007_7d582ff2.log b/ranking/ctr/AFM/AFM_kkbox_x1/AFM_kkbox_x1_007_7d582ff2.log similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_kkbox_x1/AFM_kkbox_x1_007_7d582ff2.log rename to ranking/ctr/AFM/AFM_kkbox_x1/AFM_kkbox_x1_007_7d582ff2.log diff --git a/ctr_prediction/benchmarks/AFM/AFM_kkbox_x1/AFM_kkbox_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/AFM/AFM_kkbox_x1/AFM_kkbox_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_kkbox_x1/AFM_kkbox_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/AFM/AFM_kkbox_x1/AFM_kkbox_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_kkbox_x1/AFM_kkbox_x1_tuner_config_01/model_config.yaml b/ranking/ctr/AFM/AFM_kkbox_x1/AFM_kkbox_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_kkbox_x1/AFM_kkbox_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/AFM/AFM_kkbox_x1/AFM_kkbox_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_kkbox_x1/README.md b/ranking/ctr/AFM/AFM_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_kkbox_x1/README.md rename to ranking/ctr/AFM/AFM_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/AFM/AFM_kkbox_x1/environments.txt b/ranking/ctr/AFM/AFM_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_kkbox_x1/environments.txt rename to ranking/ctr/AFM/AFM_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AFM/AFM_kkbox_x1/results.csv b/ranking/ctr/AFM/AFM_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_kkbox_x1/results.csv rename to ranking/ctr/AFM/AFM_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/AFM/AFM_kkbox_x1/run_expid.py b/ranking/ctr/AFM/AFM_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_kkbox_x1/run_expid.py rename to ranking/ctr/AFM/AFM_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AFM/AFM_movielenslatest_x1/AFM_movielenslatest_x1_002_76325415.log b/ranking/ctr/AFM/AFM_movielenslatest_x1/AFM_movielenslatest_x1_002_76325415.log similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_movielenslatest_x1/AFM_movielenslatest_x1_002_76325415.log rename to ranking/ctr/AFM/AFM_movielenslatest_x1/AFM_movielenslatest_x1_002_76325415.log diff --git a/ctr_prediction/benchmarks/AFM/AFM_movielenslatest_x1/AFM_movielenslatest_x1_tuner_config_04.csv b/ranking/ctr/AFM/AFM_movielenslatest_x1/AFM_movielenslatest_x1_tuner_config_04.csv similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_movielenslatest_x1/AFM_movielenslatest_x1_tuner_config_04.csv rename to ranking/ctr/AFM/AFM_movielenslatest_x1/AFM_movielenslatest_x1_tuner_config_04.csv diff --git a/ctr_prediction/benchmarks/AFM/AFM_movielenslatest_x1/AFM_movielenslatest_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/AFM/AFM_movielenslatest_x1/AFM_movielenslatest_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_movielenslatest_x1/AFM_movielenslatest_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/AFM/AFM_movielenslatest_x1/AFM_movielenslatest_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_movielenslatest_x1/AFM_movielenslatest_x1_tuner_config_04/model_config.yaml b/ranking/ctr/AFM/AFM_movielenslatest_x1/AFM_movielenslatest_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_movielenslatest_x1/AFM_movielenslatest_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/AFM/AFM_movielenslatest_x1/AFM_movielenslatest_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFM/AFM_movielenslatest_x1/README.md b/ranking/ctr/AFM/AFM_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_movielenslatest_x1/README.md rename to ranking/ctr/AFM/AFM_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/AFM/AFM_movielenslatest_x1/environments.txt b/ranking/ctr/AFM/AFM_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_movielenslatest_x1/environments.txt rename to ranking/ctr/AFM/AFM_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AFM/AFM_movielenslatest_x1/results.csv b/ranking/ctr/AFM/AFM_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_movielenslatest_x1/results.csv rename to ranking/ctr/AFM/AFM_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/AFM/AFM_movielenslatest_x1/run_expid.py b/ranking/ctr/AFM/AFM_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFM/AFM_movielenslatest_x1/run_expid.py rename to ranking/ctr/AFM/AFM_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x1/AFN+_avazu_x1_tuner_config_01.csv b/ranking/ctr/AFN/AFN+_avazu_x1/AFN+_avazu_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x1/AFN+_avazu_x1_tuner_config_01.csv rename to ranking/ctr/AFN/AFN+_avazu_x1/AFN+_avazu_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x1/AFN+_avazu_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/AFN/AFN+_avazu_x1/AFN+_avazu_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x1/AFN+_avazu_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/AFN/AFN+_avazu_x1/AFN+_avazu_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x1/AFN+_avazu_x1_tuner_config_01/model_config.yaml b/ranking/ctr/AFN/AFN+_avazu_x1/AFN+_avazu_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x1/AFN+_avazu_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/AFN/AFN+_avazu_x1/AFN+_avazu_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x1/AFN_avazu_x1_013_9015d34c.log b/ranking/ctr/AFN/AFN+_avazu_x1/AFN_avazu_x1_013_9015d34c.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x1/AFN_avazu_x1_013_9015d34c.log rename to ranking/ctr/AFN/AFN+_avazu_x1/AFN_avazu_x1_013_9015d34c.log diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x1/README.md b/ranking/ctr/AFN/AFN+_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x1/README.md rename to ranking/ctr/AFN/AFN+_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x1/environments.txt b/ranking/ctr/AFN/AFN+_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x1/environments.txt rename to ranking/ctr/AFN/AFN+_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x1/results.csv b/ranking/ctr/AFN/AFN+_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x1/results.csv rename to ranking/ctr/AFN/AFN+_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x1/run_expid.py b/ranking/ctr/AFN/AFN+_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x1/run_expid.py rename to ranking/ctr/AFN/AFN+_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_001/AFN_avazu_x4_003_9466021e.log b/ranking/ctr/AFN/AFN+_avazu_x4_001/AFN_avazu_x4_003_9466021e.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_001/AFN_avazu_x4_003_9466021e.log rename to ranking/ctr/AFN/AFN+_avazu_x4_001/AFN_avazu_x4_003_9466021e.log diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_001/AFN_avazu_x4_tuner_config_11/dataset_config.yaml b/ranking/ctr/AFN/AFN+_avazu_x4_001/AFN_avazu_x4_tuner_config_11/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_001/AFN_avazu_x4_tuner_config_11/dataset_config.yaml rename to ranking/ctr/AFN/AFN+_avazu_x4_001/AFN_avazu_x4_tuner_config_11/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_001/AFN_avazu_x4_tuner_config_11/model_config.yaml b/ranking/ctr/AFN/AFN+_avazu_x4_001/AFN_avazu_x4_tuner_config_11/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_001/AFN_avazu_x4_tuner_config_11/model_config.yaml rename to ranking/ctr/AFN/AFN+_avazu_x4_001/AFN_avazu_x4_tuner_config_11/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_001/README.md b/ranking/ctr/AFN/AFN+_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_001/README.md rename to ranking/ctr/AFN/AFN+_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_001/environments.txt b/ranking/ctr/AFN/AFN+_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_001/environments.txt rename to ranking/ctr/AFN/AFN+_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_001/results.csv b/ranking/ctr/AFN/AFN+_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_001/results.csv rename to ranking/ctr/AFN/AFN+_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_001/run_expid.py b/ranking/ctr/AFN/AFN+_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_001/run_expid.py rename to ranking/ctr/AFN/AFN+_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_002/AFN_avazu_x4_003_24bfb45b.log b/ranking/ctr/AFN/AFN+_avazu_x4_002/AFN_avazu_x4_003_24bfb45b.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_002/AFN_avazu_x4_003_24bfb45b.log rename to ranking/ctr/AFN/AFN+_avazu_x4_002/AFN_avazu_x4_003_24bfb45b.log diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_002/AFN_avazu_x4_tuner_config_07/dataset_config.yaml b/ranking/ctr/AFN/AFN+_avazu_x4_002/AFN_avazu_x4_tuner_config_07/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_002/AFN_avazu_x4_tuner_config_07/dataset_config.yaml rename to ranking/ctr/AFN/AFN+_avazu_x4_002/AFN_avazu_x4_tuner_config_07/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_002/AFN_avazu_x4_tuner_config_07/model_config.yaml b/ranking/ctr/AFN/AFN+_avazu_x4_002/AFN_avazu_x4_tuner_config_07/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_002/AFN_avazu_x4_tuner_config_07/model_config.yaml rename to ranking/ctr/AFN/AFN+_avazu_x4_002/AFN_avazu_x4_tuner_config_07/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_002/README.md b/ranking/ctr/AFN/AFN+_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_002/README.md rename to ranking/ctr/AFN/AFN+_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_002/environments.txt b/ranking/ctr/AFN/AFN+_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_002/environments.txt rename to ranking/ctr/AFN/AFN+_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_002/results.csv b/ranking/ctr/AFN/AFN+_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_002/results.csv rename to ranking/ctr/AFN/AFN+_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_002/run_expid.py b/ranking/ctr/AFN/AFN+_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_avazu_x4_002/run_expid.py rename to ranking/ctr/AFN/AFN+_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x1/AFN+_criteo_x1_tuner_config_04.csv b/ranking/ctr/AFN/AFN+_criteo_x1/AFN+_criteo_x1_tuner_config_04.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x1/AFN+_criteo_x1_tuner_config_04.csv rename to ranking/ctr/AFN/AFN+_criteo_x1/AFN+_criteo_x1_tuner_config_04.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x1/AFN+_criteo_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/AFN/AFN+_criteo_x1/AFN+_criteo_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x1/AFN+_criteo_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/AFN/AFN+_criteo_x1/AFN+_criteo_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x1/AFN+_criteo_x1_tuner_config_04/model_config.yaml b/ranking/ctr/AFN/AFN+_criteo_x1/AFN+_criteo_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x1/AFN+_criteo_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/AFN/AFN+_criteo_x1/AFN+_criteo_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x1/AFN_criteo_x1_002_32798d82.log b/ranking/ctr/AFN/AFN+_criteo_x1/AFN_criteo_x1_002_32798d82.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x1/AFN_criteo_x1_002_32798d82.log rename to ranking/ctr/AFN/AFN+_criteo_x1/AFN_criteo_x1_002_32798d82.log diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x1/README.md b/ranking/ctr/AFN/AFN+_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x1/README.md rename to ranking/ctr/AFN/AFN+_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x1/environments.txt b/ranking/ctr/AFN/AFN+_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x1/environments.txt rename to ranking/ctr/AFN/AFN+_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x1/results.csv b/ranking/ctr/AFN/AFN+_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x1/results.csv rename to ranking/ctr/AFN/AFN+_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x1/run_expid.py b/ranking/ctr/AFN/AFN+_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x1/run_expid.py rename to ranking/ctr/AFN/AFN+_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_001/AFN_criteo_x4_004_8fcb4074.log b/ranking/ctr/AFN/AFN+_criteo_x4_001/AFN_criteo_x4_004_8fcb4074.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_001/AFN_criteo_x4_004_8fcb4074.log rename to ranking/ctr/AFN/AFN+_criteo_x4_001/AFN_criteo_x4_004_8fcb4074.log diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_001/AFN_criteo_x4_tuner_config_12/dataset_config.yaml b/ranking/ctr/AFN/AFN+_criteo_x4_001/AFN_criteo_x4_tuner_config_12/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_001/AFN_criteo_x4_tuner_config_12/dataset_config.yaml rename to ranking/ctr/AFN/AFN+_criteo_x4_001/AFN_criteo_x4_tuner_config_12/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_001/AFN_criteo_x4_tuner_config_12/model_config.yaml b/ranking/ctr/AFN/AFN+_criteo_x4_001/AFN_criteo_x4_tuner_config_12/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_001/AFN_criteo_x4_tuner_config_12/model_config.yaml rename to ranking/ctr/AFN/AFN+_criteo_x4_001/AFN_criteo_x4_tuner_config_12/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_001/README.md b/ranking/ctr/AFN/AFN+_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_001/README.md rename to ranking/ctr/AFN/AFN+_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_001/environments.txt b/ranking/ctr/AFN/AFN+_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_001/environments.txt rename to ranking/ctr/AFN/AFN+_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_001/results.csv b/ranking/ctr/AFN/AFN+_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_001/results.csv rename to ranking/ctr/AFN/AFN+_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_001/run_expid.py b/ranking/ctr/AFN/AFN+_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_001/run_expid.py rename to ranking/ctr/AFN/AFN+_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_002/AFN_criteo_x4_005_82d192ec.log b/ranking/ctr/AFN/AFN+_criteo_x4_002/AFN_criteo_x4_005_82d192ec.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_002/AFN_criteo_x4_005_82d192ec.log rename to ranking/ctr/AFN/AFN+_criteo_x4_002/AFN_criteo_x4_005_82d192ec.log diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_002/AFN_criteo_x4_tuner_config_03/dataset_config.yaml b/ranking/ctr/AFN/AFN+_criteo_x4_002/AFN_criteo_x4_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_002/AFN_criteo_x4_tuner_config_03/dataset_config.yaml rename to ranking/ctr/AFN/AFN+_criteo_x4_002/AFN_criteo_x4_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_002/AFN_criteo_x4_tuner_config_03/model_config.yaml b/ranking/ctr/AFN/AFN+_criteo_x4_002/AFN_criteo_x4_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_002/AFN_criteo_x4_tuner_config_03/model_config.yaml rename to ranking/ctr/AFN/AFN+_criteo_x4_002/AFN_criteo_x4_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_002/README.md b/ranking/ctr/AFN/AFN+_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_002/README.md rename to ranking/ctr/AFN/AFN+_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_002/environments.txt b/ranking/ctr/AFN/AFN+_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_002/environments.txt rename to ranking/ctr/AFN/AFN+_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_002/results.csv b/ranking/ctr/AFN/AFN+_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_002/results.csv rename to ranking/ctr/AFN/AFN+_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_002/run_expid.py b/ranking/ctr/AFN/AFN+_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_criteo_x4_002/run_expid.py rename to ranking/ctr/AFN/AFN+_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN+_frappe_x1/AFN+_frappe_x1_tuner_config_02.csv b/ranking/ctr/AFN/AFN+_frappe_x1/AFN+_frappe_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_frappe_x1/AFN+_frappe_x1_tuner_config_02.csv rename to ranking/ctr/AFN/AFN+_frappe_x1/AFN+_frappe_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN+_frappe_x1/AFN+_frappe_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/AFN/AFN+_frappe_x1/AFN+_frappe_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_frappe_x1/AFN+_frappe_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/AFN/AFN+_frappe_x1/AFN+_frappe_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_frappe_x1/AFN+_frappe_x1_tuner_config_02/model_config.yaml b/ranking/ctr/AFN/AFN+_frappe_x1/AFN+_frappe_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_frappe_x1/AFN+_frappe_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/AFN/AFN+_frappe_x1/AFN+_frappe_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_frappe_x1/AFN_frappe_x1_004_d2ea60c3.log b/ranking/ctr/AFN/AFN+_frappe_x1/AFN_frappe_x1_004_d2ea60c3.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_frappe_x1/AFN_frappe_x1_004_d2ea60c3.log rename to ranking/ctr/AFN/AFN+_frappe_x1/AFN_frappe_x1_004_d2ea60c3.log diff --git a/ctr_prediction/benchmarks/AFN/AFN+_frappe_x1/README.md b/ranking/ctr/AFN/AFN+_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_frappe_x1/README.md rename to ranking/ctr/AFN/AFN+_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN+_frappe_x1/environments.txt b/ranking/ctr/AFN/AFN+_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_frappe_x1/environments.txt rename to ranking/ctr/AFN/AFN+_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN+_frappe_x1/results.csv b/ranking/ctr/AFN/AFN+_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_frappe_x1/results.csv rename to ranking/ctr/AFN/AFN+_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN+_frappe_x1/run_expid.py b/ranking/ctr/AFN/AFN+_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_frappe_x1/run_expid.py rename to ranking/ctr/AFN/AFN+_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN+_kkbox_x1/AFN+_kkbox_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/AFN/AFN+_kkbox_x1/AFN+_kkbox_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_kkbox_x1/AFN+_kkbox_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/AFN/AFN+_kkbox_x1/AFN+_kkbox_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_kkbox_x1/AFN+_kkbox_x1_tuner_config_04/model_config.yaml b/ranking/ctr/AFN/AFN+_kkbox_x1/AFN+_kkbox_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_kkbox_x1/AFN+_kkbox_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/AFN/AFN+_kkbox_x1/AFN+_kkbox_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_kkbox_x1/AFN_kkbox_x1_014_a8ea82ca.log b/ranking/ctr/AFN/AFN+_kkbox_x1/AFN_kkbox_x1_014_a8ea82ca.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_kkbox_x1/AFN_kkbox_x1_014_a8ea82ca.log rename to ranking/ctr/AFN/AFN+_kkbox_x1/AFN_kkbox_x1_014_a8ea82ca.log diff --git a/ctr_prediction/benchmarks/AFN/AFN+_kkbox_x1/README.md b/ranking/ctr/AFN/AFN+_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_kkbox_x1/README.md rename to ranking/ctr/AFN/AFN+_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN+_kkbox_x1/environments.txt b/ranking/ctr/AFN/AFN+_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_kkbox_x1/environments.txt rename to ranking/ctr/AFN/AFN+_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN+_kkbox_x1/results.csv b/ranking/ctr/AFN/AFN+_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_kkbox_x1/results.csv rename to ranking/ctr/AFN/AFN+_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN+_kkbox_x1/run_expid.py b/ranking/ctr/AFN/AFN+_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_kkbox_x1/run_expid.py rename to ranking/ctr/AFN/AFN+_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN+_movielenslatest_x1/AFN+_movielenslatest_x1_tuner_config_01.csv b/ranking/ctr/AFN/AFN+_movielenslatest_x1/AFN+_movielenslatest_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_movielenslatest_x1/AFN+_movielenslatest_x1_tuner_config_01.csv rename to ranking/ctr/AFN/AFN+_movielenslatest_x1/AFN+_movielenslatest_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN+_movielenslatest_x1/AFN+_movielenslatest_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/AFN/AFN+_movielenslatest_x1/AFN+_movielenslatest_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_movielenslatest_x1/AFN+_movielenslatest_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/AFN/AFN+_movielenslatest_x1/AFN+_movielenslatest_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_movielenslatest_x1/AFN+_movielenslatest_x1_tuner_config_01/model_config.yaml b/ranking/ctr/AFN/AFN+_movielenslatest_x1/AFN+_movielenslatest_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_movielenslatest_x1/AFN+_movielenslatest_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/AFN/AFN+_movielenslatest_x1/AFN+_movielenslatest_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN+_movielenslatest_x1/AFN_movielenslatest_x1_003_cc30c477.log b/ranking/ctr/AFN/AFN+_movielenslatest_x1/AFN_movielenslatest_x1_003_cc30c477.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_movielenslatest_x1/AFN_movielenslatest_x1_003_cc30c477.log rename to ranking/ctr/AFN/AFN+_movielenslatest_x1/AFN_movielenslatest_x1_003_cc30c477.log diff --git a/ctr_prediction/benchmarks/AFN/AFN+_movielenslatest_x1/README.md b/ranking/ctr/AFN/AFN+_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_movielenslatest_x1/README.md rename to ranking/ctr/AFN/AFN+_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN+_movielenslatest_x1/environments.txt b/ranking/ctr/AFN/AFN+_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_movielenslatest_x1/environments.txt rename to ranking/ctr/AFN/AFN+_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN+_movielenslatest_x1/results.csv b/ranking/ctr/AFN/AFN+_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_movielenslatest_x1/results.csv rename to ranking/ctr/AFN/AFN+_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN+_movielenslatest_x1/run_expid.py b/ranking/ctr/AFN/AFN+_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN+_movielenslatest_x1/run_expid.py rename to ranking/ctr/AFN/AFN+_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x1/AFN_avazu_x1_014_af89ab38.log b/ranking/ctr/AFN/AFN_avazu_x1/AFN_avazu_x1_014_af89ab38.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x1/AFN_avazu_x1_014_af89ab38.log rename to ranking/ctr/AFN/AFN_avazu_x1/AFN_avazu_x1_014_af89ab38.log diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x1/AFN_avazu_x1_tuner_config_01.csv b/ranking/ctr/AFN/AFN_avazu_x1/AFN_avazu_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x1/AFN_avazu_x1_tuner_config_01.csv rename to ranking/ctr/AFN/AFN_avazu_x1/AFN_avazu_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x1/AFN_avazu_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/AFN/AFN_avazu_x1/AFN_avazu_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x1/AFN_avazu_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/AFN/AFN_avazu_x1/AFN_avazu_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x1/AFN_avazu_x1_tuner_config_01/model_config.yaml b/ranking/ctr/AFN/AFN_avazu_x1/AFN_avazu_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x1/AFN_avazu_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/AFN/AFN_avazu_x1/AFN_avazu_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x1/README.md b/ranking/ctr/AFN/AFN_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x1/README.md rename to ranking/ctr/AFN/AFN_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x1/environments.txt b/ranking/ctr/AFN/AFN_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x1/environments.txt rename to ranking/ctr/AFN/AFN_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x1/results.csv b/ranking/ctr/AFN/AFN_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x1/results.csv rename to ranking/ctr/AFN/AFN_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x1/run_expid.py b/ranking/ctr/AFN/AFN_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x1/run_expid.py rename to ranking/ctr/AFN/AFN_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x4_001/AFN_avazu_x4_003_3c4fe87a.log b/ranking/ctr/AFN/AFN_avazu_x4_001/AFN_avazu_x4_003_3c4fe87a.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x4_001/AFN_avazu_x4_003_3c4fe87a.log rename to ranking/ctr/AFN/AFN_avazu_x4_001/AFN_avazu_x4_003_3c4fe87a.log diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x4_001/AFN_avazu_x4_tuner_config_04/dataset_config.yaml b/ranking/ctr/AFN/AFN_avazu_x4_001/AFN_avazu_x4_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x4_001/AFN_avazu_x4_tuner_config_04/dataset_config.yaml rename to ranking/ctr/AFN/AFN_avazu_x4_001/AFN_avazu_x4_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x4_001/AFN_avazu_x4_tuner_config_04/model_config.yaml b/ranking/ctr/AFN/AFN_avazu_x4_001/AFN_avazu_x4_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x4_001/AFN_avazu_x4_tuner_config_04/model_config.yaml rename to ranking/ctr/AFN/AFN_avazu_x4_001/AFN_avazu_x4_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x4_001/README.md b/ranking/ctr/AFN/AFN_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x4_001/README.md rename to ranking/ctr/AFN/AFN_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x4_001/environments.txt b/ranking/ctr/AFN/AFN_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x4_001/environments.txt rename to ranking/ctr/AFN/AFN_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x4_001/results.csv b/ranking/ctr/AFN/AFN_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x4_001/results.csv rename to ranking/ctr/AFN/AFN_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x4_001/run_expid.py b/ranking/ctr/AFN/AFN_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x4_001/run_expid.py rename to ranking/ctr/AFN/AFN_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x4_002/AFN_avazu_x4_012_00ce5d6c.log b/ranking/ctr/AFN/AFN_avazu_x4_002/AFN_avazu_x4_012_00ce5d6c.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x4_002/AFN_avazu_x4_012_00ce5d6c.log rename to ranking/ctr/AFN/AFN_avazu_x4_002/AFN_avazu_x4_012_00ce5d6c.log diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x4_002/AFN_avazu_x4_tuner_config_03/dataset_config.yaml b/ranking/ctr/AFN/AFN_avazu_x4_002/AFN_avazu_x4_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x4_002/AFN_avazu_x4_tuner_config_03/dataset_config.yaml rename to ranking/ctr/AFN/AFN_avazu_x4_002/AFN_avazu_x4_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x4_002/AFN_avazu_x4_tuner_config_03/model_config.yaml b/ranking/ctr/AFN/AFN_avazu_x4_002/AFN_avazu_x4_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x4_002/AFN_avazu_x4_tuner_config_03/model_config.yaml rename to ranking/ctr/AFN/AFN_avazu_x4_002/AFN_avazu_x4_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x4_002/README.md b/ranking/ctr/AFN/AFN_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x4_002/README.md rename to ranking/ctr/AFN/AFN_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x4_002/environments.txt b/ranking/ctr/AFN/AFN_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x4_002/environments.txt rename to ranking/ctr/AFN/AFN_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x4_002/results.csv b/ranking/ctr/AFN/AFN_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x4_002/results.csv rename to ranking/ctr/AFN/AFN_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN_avazu_x4_002/run_expid.py b/ranking/ctr/AFN/AFN_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_avazu_x4_002/run_expid.py rename to ranking/ctr/AFN/AFN_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x1/AFN_criteo_x1_002_d1191676.log b/ranking/ctr/AFN/AFN_criteo_x1/AFN_criteo_x1_002_d1191676.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x1/AFN_criteo_x1_002_d1191676.log rename to ranking/ctr/AFN/AFN_criteo_x1/AFN_criteo_x1_002_d1191676.log diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x1/AFN_criteo_x1_tuner_config_03.csv b/ranking/ctr/AFN/AFN_criteo_x1/AFN_criteo_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x1/AFN_criteo_x1_tuner_config_03.csv rename to ranking/ctr/AFN/AFN_criteo_x1/AFN_criteo_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x1/AFN_criteo_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/AFN/AFN_criteo_x1/AFN_criteo_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x1/AFN_criteo_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/AFN/AFN_criteo_x1/AFN_criteo_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x1/AFN_criteo_x1_tuner_config_03/model_config.yaml b/ranking/ctr/AFN/AFN_criteo_x1/AFN_criteo_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x1/AFN_criteo_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/AFN/AFN_criteo_x1/AFN_criteo_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x1/README.md b/ranking/ctr/AFN/AFN_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x1/README.md rename to ranking/ctr/AFN/AFN_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x1/environments.txt b/ranking/ctr/AFN/AFN_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x1/environments.txt rename to ranking/ctr/AFN/AFN_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x1/results.csv b/ranking/ctr/AFN/AFN_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x1/results.csv rename to ranking/ctr/AFN/AFN_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x1/run_expid.py b/ranking/ctr/AFN/AFN_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x1/run_expid.py rename to ranking/ctr/AFN/AFN_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x4_001/AFN_criteo_x4_012_d479cb4e.log b/ranking/ctr/AFN/AFN_criteo_x4_001/AFN_criteo_x4_012_d479cb4e.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x4_001/AFN_criteo_x4_012_d479cb4e.log rename to ranking/ctr/AFN/AFN_criteo_x4_001/AFN_criteo_x4_012_d479cb4e.log diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x4_001/AFN_criteo_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/AFN/AFN_criteo_x4_001/AFN_criteo_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x4_001/AFN_criteo_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/AFN/AFN_criteo_x4_001/AFN_criteo_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x4_001/AFN_criteo_x4_tuner_config_02/model_config.yaml b/ranking/ctr/AFN/AFN_criteo_x4_001/AFN_criteo_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x4_001/AFN_criteo_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/AFN/AFN_criteo_x4_001/AFN_criteo_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x4_001/README.md b/ranking/ctr/AFN/AFN_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x4_001/README.md rename to ranking/ctr/AFN/AFN_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x4_001/environments.txt b/ranking/ctr/AFN/AFN_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x4_001/environments.txt rename to ranking/ctr/AFN/AFN_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x4_001/results.csv b/ranking/ctr/AFN/AFN_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x4_001/results.csv rename to ranking/ctr/AFN/AFN_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x4_001/run_expid.py b/ranking/ctr/AFN/AFN_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x4_001/run_expid.py rename to ranking/ctr/AFN/AFN_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x4_002/AFN_criteo_x4_080_cf447dc2.log b/ranking/ctr/AFN/AFN_criteo_x4_002/AFN_criteo_x4_080_cf447dc2.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x4_002/AFN_criteo_x4_080_cf447dc2.log rename to ranking/ctr/AFN/AFN_criteo_x4_002/AFN_criteo_x4_080_cf447dc2.log diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x4_002/AFN_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/AFN/AFN_criteo_x4_002/AFN_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x4_002/AFN_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/AFN/AFN_criteo_x4_002/AFN_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x4_002/AFN_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/AFN/AFN_criteo_x4_002/AFN_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x4_002/AFN_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/AFN/AFN_criteo_x4_002/AFN_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x4_002/README.md b/ranking/ctr/AFN/AFN_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x4_002/README.md rename to ranking/ctr/AFN/AFN_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x4_002/environments.txt b/ranking/ctr/AFN/AFN_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x4_002/environments.txt rename to ranking/ctr/AFN/AFN_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x4_002/results.csv b/ranking/ctr/AFN/AFN_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x4_002/results.csv rename to ranking/ctr/AFN/AFN_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN_criteo_x4_002/run_expid.py b/ranking/ctr/AFN/AFN_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_criteo_x4_002/run_expid.py rename to ranking/ctr/AFN/AFN_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN_frappe_x1/AFN_frappe_x1_008_f15b0bf0.log b/ranking/ctr/AFN/AFN_frappe_x1/AFN_frappe_x1_008_f15b0bf0.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_frappe_x1/AFN_frappe_x1_008_f15b0bf0.log rename to ranking/ctr/AFN/AFN_frappe_x1/AFN_frappe_x1_008_f15b0bf0.log diff --git a/ctr_prediction/benchmarks/AFN/AFN_frappe_x1/AFN_frappe_x1_tuner_config_02.csv b/ranking/ctr/AFN/AFN_frappe_x1/AFN_frappe_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_frappe_x1/AFN_frappe_x1_tuner_config_02.csv rename to ranking/ctr/AFN/AFN_frappe_x1/AFN_frappe_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN_frappe_x1/AFN_frappe_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/AFN/AFN_frappe_x1/AFN_frappe_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_frappe_x1/AFN_frappe_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/AFN/AFN_frappe_x1/AFN_frappe_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_frappe_x1/AFN_frappe_x1_tuner_config_02/model_config.yaml b/ranking/ctr/AFN/AFN_frappe_x1/AFN_frappe_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_frappe_x1/AFN_frappe_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/AFN/AFN_frappe_x1/AFN_frappe_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_frappe_x1/README.md b/ranking/ctr/AFN/AFN_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_frappe_x1/README.md rename to ranking/ctr/AFN/AFN_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN_frappe_x1/environments.txt b/ranking/ctr/AFN/AFN_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_frappe_x1/environments.txt rename to ranking/ctr/AFN/AFN_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN_frappe_x1/results.csv b/ranking/ctr/AFN/AFN_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_frappe_x1/results.csv rename to ranking/ctr/AFN/AFN_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN_frappe_x1/run_expid.py b/ranking/ctr/AFN/AFN_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_frappe_x1/run_expid.py rename to ranking/ctr/AFN/AFN_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN_kkbox_x1/AFN_kkbox_x1_006_f1f7fac0.log b/ranking/ctr/AFN/AFN_kkbox_x1/AFN_kkbox_x1_006_f1f7fac0.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_kkbox_x1/AFN_kkbox_x1_006_f1f7fac0.log rename to ranking/ctr/AFN/AFN_kkbox_x1/AFN_kkbox_x1_006_f1f7fac0.log diff --git a/ctr_prediction/benchmarks/AFN/AFN_kkbox_x1/AFN_kkbox_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/AFN/AFN_kkbox_x1/AFN_kkbox_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_kkbox_x1/AFN_kkbox_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/AFN/AFN_kkbox_x1/AFN_kkbox_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_kkbox_x1/AFN_kkbox_x1_tuner_config_03/model_config.yaml b/ranking/ctr/AFN/AFN_kkbox_x1/AFN_kkbox_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_kkbox_x1/AFN_kkbox_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/AFN/AFN_kkbox_x1/AFN_kkbox_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_kkbox_x1/README.md b/ranking/ctr/AFN/AFN_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_kkbox_x1/README.md rename to ranking/ctr/AFN/AFN_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN_kkbox_x1/environments.txt b/ranking/ctr/AFN/AFN_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_kkbox_x1/environments.txt rename to ranking/ctr/AFN/AFN_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN_kkbox_x1/results.csv b/ranking/ctr/AFN/AFN_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_kkbox_x1/results.csv rename to ranking/ctr/AFN/AFN_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN_kkbox_x1/run_expid.py b/ranking/ctr/AFN/AFN_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_kkbox_x1/run_expid.py rename to ranking/ctr/AFN/AFN_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AFN/AFN_movielenslatest_x1/AFN_movielenslatest_x1_015_9d6fa874.log b/ranking/ctr/AFN/AFN_movielenslatest_x1/AFN_movielenslatest_x1_015_9d6fa874.log similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_movielenslatest_x1/AFN_movielenslatest_x1_015_9d6fa874.log rename to ranking/ctr/AFN/AFN_movielenslatest_x1/AFN_movielenslatest_x1_015_9d6fa874.log diff --git a/ctr_prediction/benchmarks/AFN/AFN_movielenslatest_x1/AFN_movielenslatest_x1_tuner_config_02.csv b/ranking/ctr/AFN/AFN_movielenslatest_x1/AFN_movielenslatest_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_movielenslatest_x1/AFN_movielenslatest_x1_tuner_config_02.csv rename to ranking/ctr/AFN/AFN_movielenslatest_x1/AFN_movielenslatest_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN_movielenslatest_x1/AFN_movielenslatest_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/AFN/AFN_movielenslatest_x1/AFN_movielenslatest_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_movielenslatest_x1/AFN_movielenslatest_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/AFN/AFN_movielenslatest_x1/AFN_movielenslatest_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_movielenslatest_x1/AFN_movielenslatest_x1_tuner_config_02/model_config.yaml b/ranking/ctr/AFN/AFN_movielenslatest_x1/AFN_movielenslatest_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_movielenslatest_x1/AFN_movielenslatest_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/AFN/AFN_movielenslatest_x1/AFN_movielenslatest_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/AFN/AFN_movielenslatest_x1/README.md b/ranking/ctr/AFN/AFN_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_movielenslatest_x1/README.md rename to ranking/ctr/AFN/AFN_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/AFN/AFN_movielenslatest_x1/environments.txt b/ranking/ctr/AFN/AFN_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_movielenslatest_x1/environments.txt rename to ranking/ctr/AFN/AFN_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AFN/AFN_movielenslatest_x1/results.csv b/ranking/ctr/AFN/AFN_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_movielenslatest_x1/results.csv rename to ranking/ctr/AFN/AFN_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/AFN/AFN_movielenslatest_x1/run_expid.py b/ranking/ctr/AFN/AFN_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AFN/AFN_movielenslatest_x1/run_expid.py rename to ranking/ctr/AFN/AFN_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_044_ac142790.log b/ranking/ctr/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_044_ac142790.log similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_044_ac142790.log rename to ranking/ctr/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_044_ac142790.log diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_tuner_config_02.csv b/ranking/ctr/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_tuner_config_02.csv rename to ranking/ctr/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_tuner_config_02.yaml b/ranking/ctr/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_tuner_config_02.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_tuner_config_02.yaml rename to ranking/ctr/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_tuner_config_02.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_tuner_config_02/model_config.yaml b/ranking/ctr/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/AOANet/AOANet_amazonelectronics_x1/AOANet_amazonelectronics_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_amazonelectronics_x1/README.md b/ranking/ctr/AOANet/AOANet_amazonelectronics_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_amazonelectronics_x1/README.md rename to ranking/ctr/AOANet/AOANet_amazonelectronics_x1/README.md diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_amazonelectronics_x1/environments.txt b/ranking/ctr/AOANet/AOANet_amazonelectronics_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_amazonelectronics_x1/environments.txt rename to ranking/ctr/AOANet/AOANet_amazonelectronics_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_amazonelectronics_x1/results.csv b/ranking/ctr/AOANet/AOANet_amazonelectronics_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_amazonelectronics_x1/results.csv rename to ranking/ctr/AOANet/AOANet_amazonelectronics_x1/results.csv diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_avazu_x1/AOANet_avazu_x1_004_a663f0bb.log b/ranking/ctr/AOANet/AOANet_avazu_x1/AOANet_avazu_x1_004_a663f0bb.log similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_avazu_x1/AOANet_avazu_x1_004_a663f0bb.log rename to ranking/ctr/AOANet/AOANet_avazu_x1/AOANet_avazu_x1_004_a663f0bb.log diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_avazu_x1/AOANet_avazu_x1_tuner_config_01.csv b/ranking/ctr/AOANet/AOANet_avazu_x1/AOANet_avazu_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_avazu_x1/AOANet_avazu_x1_tuner_config_01.csv rename to ranking/ctr/AOANet/AOANet_avazu_x1/AOANet_avazu_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_avazu_x1/AOANet_avazu_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/AOANet/AOANet_avazu_x1/AOANet_avazu_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_avazu_x1/AOANet_avazu_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/AOANet/AOANet_avazu_x1/AOANet_avazu_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_avazu_x1/AOANet_avazu_x1_tuner_config_01/model_config.yaml b/ranking/ctr/AOANet/AOANet_avazu_x1/AOANet_avazu_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_avazu_x1/AOANet_avazu_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/AOANet/AOANet_avazu_x1/AOANet_avazu_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_avazu_x1/README.md b/ranking/ctr/AOANet/AOANet_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_avazu_x1/README.md rename to ranking/ctr/AOANet/AOANet_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_avazu_x1/environments.txt b/ranking/ctr/AOANet/AOANet_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_avazu_x1/environments.txt rename to ranking/ctr/AOANet/AOANet_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_avazu_x1/results.csv b/ranking/ctr/AOANet/AOANet_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_avazu_x1/results.csv rename to ranking/ctr/AOANet/AOANet_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_avazu_x1/run_expid.py b/ranking/ctr/AOANet/AOANet_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_avazu_x1/run_expid.py rename to ranking/ctr/AOANet/AOANet_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_criteo_x1/AOANet_criteo_x1_005_faa15d7f.log b/ranking/ctr/AOANet/AOANet_criteo_x1/AOANet_criteo_x1_005_faa15d7f.log similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_criteo_x1/AOANet_criteo_x1_005_faa15d7f.log rename to ranking/ctr/AOANet/AOANet_criteo_x1/AOANet_criteo_x1_005_faa15d7f.log diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_criteo_x1/AOANet_criteo_x1_tuner_config_03.csv b/ranking/ctr/AOANet/AOANet_criteo_x1/AOANet_criteo_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_criteo_x1/AOANet_criteo_x1_tuner_config_03.csv rename to ranking/ctr/AOANet/AOANet_criteo_x1/AOANet_criteo_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_criteo_x1/AOANet_criteo_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/AOANet/AOANet_criteo_x1/AOANet_criteo_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_criteo_x1/AOANet_criteo_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/AOANet/AOANet_criteo_x1/AOANet_criteo_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_criteo_x1/AOANet_criteo_x1_tuner_config_03/model_config.yaml b/ranking/ctr/AOANet/AOANet_criteo_x1/AOANet_criteo_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_criteo_x1/AOANet_criteo_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/AOANet/AOANet_criteo_x1/AOANet_criteo_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_criteo_x1/README.md b/ranking/ctr/AOANet/AOANet_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_criteo_x1/README.md rename to ranking/ctr/AOANet/AOANet_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_criteo_x1/environments.txt b/ranking/ctr/AOANet/AOANet_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_criteo_x1/environments.txt rename to ranking/ctr/AOANet/AOANet_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_criteo_x1/results.csv b/ranking/ctr/AOANet/AOANet_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_criteo_x1/results.csv rename to ranking/ctr/AOANet/AOANet_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_criteo_x1/run_expid.py b/ranking/ctr/AOANet/AOANet_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_criteo_x1/run_expid.py rename to ranking/ctr/AOANet/AOANet_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_frappe_x1/AOANet_frappe_x1_009_29c57772.log b/ranking/ctr/AOANet/AOANet_frappe_x1/AOANet_frappe_x1_009_29c57772.log similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_frappe_x1/AOANet_frappe_x1_009_29c57772.log rename to ranking/ctr/AOANet/AOANet_frappe_x1/AOANet_frappe_x1_009_29c57772.log diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_frappe_x1/AOANet_frappe_x1_tuner_config_02.csv b/ranking/ctr/AOANet/AOANet_frappe_x1/AOANet_frappe_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_frappe_x1/AOANet_frappe_x1_tuner_config_02.csv rename to ranking/ctr/AOANet/AOANet_frappe_x1/AOANet_frappe_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_frappe_x1/AOANet_frappe_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/AOANet/AOANet_frappe_x1/AOANet_frappe_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_frappe_x1/AOANet_frappe_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/AOANet/AOANet_frappe_x1/AOANet_frappe_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_frappe_x1/AOANet_frappe_x1_tuner_config_02/model_config.yaml b/ranking/ctr/AOANet/AOANet_frappe_x1/AOANet_frappe_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_frappe_x1/AOANet_frappe_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/AOANet/AOANet_frappe_x1/AOANet_frappe_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_frappe_x1/README.md b/ranking/ctr/AOANet/AOANet_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_frappe_x1/README.md rename to ranking/ctr/AOANet/AOANet_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_frappe_x1/environments.txt b/ranking/ctr/AOANet/AOANet_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_frappe_x1/environments.txt rename to ranking/ctr/AOANet/AOANet_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_frappe_x1/results.csv b/ranking/ctr/AOANet/AOANet_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_frappe_x1/results.csv rename to ranking/ctr/AOANet/AOANet_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_frappe_x1/run_expid.py b/ranking/ctr/AOANet/AOANet_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_frappe_x1/run_expid.py rename to ranking/ctr/AOANet/AOANet_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_006_71fb5d8c.log b/ranking/ctr/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_006_71fb5d8c.log similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_006_71fb5d8c.log rename to ranking/ctr/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_006_71fb5d8c.log diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_tuner_config_03.csv b/ranking/ctr/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_tuner_config_03.csv rename to ranking/ctr/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_tuner_config_03.yaml b/ranking/ctr/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_tuner_config_03.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_tuner_config_03.yaml rename to ranking/ctr/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_tuner_config_03.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_tuner_config_03/model_config.yaml b/ranking/ctr/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/AOANet/AOANet_kuaivideo_x1/AOANet_kuaivideo_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_kuaivideo_x1/README.md b/ranking/ctr/AOANet/AOANet_kuaivideo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_kuaivideo_x1/README.md rename to ranking/ctr/AOANet/AOANet_kuaivideo_x1/README.md diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_kuaivideo_x1/environments.txt b/ranking/ctr/AOANet/AOANet_kuaivideo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_kuaivideo_x1/environments.txt rename to ranking/ctr/AOANet/AOANet_kuaivideo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_kuaivideo_x1/results.csv b/ranking/ctr/AOANet/AOANet_kuaivideo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_kuaivideo_x1/results.csv rename to ranking/ctr/AOANet/AOANet_kuaivideo_x1/results.csv diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_005_5a9d3de4.log b/ranking/ctr/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_005_5a9d3de4.log similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_005_5a9d3de4.log rename to ranking/ctr/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_005_5a9d3de4.log diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_tuner_config_01.csv b/ranking/ctr/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_tuner_config_01.csv rename to ranking/ctr/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_tuner_config_01.yaml b/ranking/ctr/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_tuner_config_01.yaml rename to ranking/ctr/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_tuner_config_01/model_config.yaml b/ranking/ctr/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/AOANet/AOANet_microvideo1.7m_x1/AOANet_microvideo1.7m_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_microvideo1.7m_x1/README.md b/ranking/ctr/AOANet/AOANet_microvideo1.7m_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_microvideo1.7m_x1/README.md rename to ranking/ctr/AOANet/AOANet_microvideo1.7m_x1/README.md diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_microvideo1.7m_x1/environments.txt b/ranking/ctr/AOANet/AOANet_microvideo1.7m_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_microvideo1.7m_x1/environments.txt rename to ranking/ctr/AOANet/AOANet_microvideo1.7m_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_microvideo1.7m_x1/results.csv b/ranking/ctr/AOANet/AOANet_microvideo1.7m_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_microvideo1.7m_x1/results.csv rename to ranking/ctr/AOANet/AOANet_microvideo1.7m_x1/results.csv diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_movielenslatest_x1/AOANet_movielenslatest_x1_031_bbf8c17a.log b/ranking/ctr/AOANet/AOANet_movielenslatest_x1/AOANet_movielenslatest_x1_031_bbf8c17a.log similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_movielenslatest_x1/AOANet_movielenslatest_x1_031_bbf8c17a.log rename to ranking/ctr/AOANet/AOANet_movielenslatest_x1/AOANet_movielenslatest_x1_031_bbf8c17a.log diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_movielenslatest_x1/AOANet_movielenslatest_x1_tuner_config_01.csv b/ranking/ctr/AOANet/AOANet_movielenslatest_x1/AOANet_movielenslatest_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_movielenslatest_x1/AOANet_movielenslatest_x1_tuner_config_01.csv rename to ranking/ctr/AOANet/AOANet_movielenslatest_x1/AOANet_movielenslatest_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_movielenslatest_x1/AOANet_movielenslatest_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/AOANet/AOANet_movielenslatest_x1/AOANet_movielenslatest_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_movielenslatest_x1/AOANet_movielenslatest_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/AOANet/AOANet_movielenslatest_x1/AOANet_movielenslatest_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_movielenslatest_x1/AOANet_movielenslatest_x1_tuner_config_01/model_config.yaml b/ranking/ctr/AOANet/AOANet_movielenslatest_x1/AOANet_movielenslatest_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_movielenslatest_x1/AOANet_movielenslatest_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/AOANet/AOANet_movielenslatest_x1/AOANet_movielenslatest_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_movielenslatest_x1/README.md b/ranking/ctr/AOANet/AOANet_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_movielenslatest_x1/README.md rename to ranking/ctr/AOANet/AOANet_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_movielenslatest_x1/environments.txt b/ranking/ctr/AOANet/AOANet_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_movielenslatest_x1/environments.txt rename to ranking/ctr/AOANet/AOANet_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_movielenslatest_x1/results.csv b/ranking/ctr/AOANet/AOANet_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_movielenslatest_x1/results.csv rename to ranking/ctr/AOANet/AOANet_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_movielenslatest_x1/run_expid.py b/ranking/ctr/AOANet/AOANet_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_movielenslatest_x1/run_expid.py rename to ranking/ctr/AOANet/AOANet_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_005_26a70b74.log b/ranking/ctr/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_005_26a70b74.log similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_005_26a70b74.log rename to ranking/ctr/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_005_26a70b74.log diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_tuner_config_01.csv b/ranking/ctr/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_tuner_config_01.csv rename to ranking/ctr/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_tuner_config_01.yaml b/ranking/ctr/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_tuner_config_01.yaml rename to ranking/ctr/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_tuner_config_01/model_config.yaml b/ranking/ctr/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/AOANet/AOANet_taobaoad_x1/AOANet_taobaoad_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_taobaoad_x1/README.md b/ranking/ctr/AOANet/AOANet_taobaoad_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_taobaoad_x1/README.md rename to ranking/ctr/AOANet/AOANet_taobaoad_x1/README.md diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_taobaoad_x1/environments.txt b/ranking/ctr/AOANet/AOANet_taobaoad_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_taobaoad_x1/environments.txt rename to ranking/ctr/AOANet/AOANet_taobaoad_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AOANet/AOANet_taobaoad_x1/results.csv b/ranking/ctr/AOANet/AOANet_taobaoad_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AOANet/AOANet_taobaoad_x1/results.csv rename to ranking/ctr/AOANet/AOANet_taobaoad_x1/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/AutoInt+_avazu_x1_tuner_config_06.csv b/ranking/ctr/AutoInt/AutoInt+_avazu_x1/AutoInt+_avazu_x1_tuner_config_06.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/AutoInt+_avazu_x1_tuner_config_06.csv rename to ranking/ctr/AutoInt/AutoInt+_avazu_x1/AutoInt+_avazu_x1_tuner_config_06.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/AutoInt+_avazu_x1_tuner_config_06/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt+_avazu_x1/AutoInt+_avazu_x1_tuner_config_06/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/AutoInt+_avazu_x1_tuner_config_06/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_avazu_x1/AutoInt+_avazu_x1_tuner_config_06/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/AutoInt+_avazu_x1_tuner_config_06/model_config.yaml b/ranking/ctr/AutoInt/AutoInt+_avazu_x1/AutoInt+_avazu_x1_tuner_config_06/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/AutoInt+_avazu_x1_tuner_config_06/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_avazu_x1/AutoInt+_avazu_x1_tuner_config_06/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/AutoInt_avazu_x1_005_73d0b026.log b/ranking/ctr/AutoInt/AutoInt+_avazu_x1/AutoInt_avazu_x1_005_73d0b026.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/AutoInt_avazu_x1_005_73d0b026.log rename to ranking/ctr/AutoInt/AutoInt+_avazu_x1/AutoInt_avazu_x1_005_73d0b026.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/README.md b/ranking/ctr/AutoInt/AutoInt+_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/README.md rename to ranking/ctr/AutoInt/AutoInt+_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/environments.txt b/ranking/ctr/AutoInt/AutoInt+_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/environments.txt rename to ranking/ctr/AutoInt/AutoInt+_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/fuxictr_version.py b/ranking/ctr/AutoInt/AutoInt+_avazu_x1/fuxictr_version.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/fuxictr_version.py rename to ranking/ctr/AutoInt/AutoInt+_avazu_x1/fuxictr_version.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/results.csv b/ranking/ctr/AutoInt/AutoInt+_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/results.csv rename to ranking/ctr/AutoInt/AutoInt+_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/run_expid.py b/ranking/ctr/AutoInt/AutoInt+_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x1/run_expid.py rename to ranking/ctr/AutoInt/AutoInt+_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_001/AutoInt+_avazu_x4_tuner_config_03/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt+_avazu_x4_001/AutoInt+_avazu_x4_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_001/AutoInt+_avazu_x4_tuner_config_03/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_avazu_x4_001/AutoInt+_avazu_x4_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_001/AutoInt+_avazu_x4_tuner_config_03/model_config.yaml b/ranking/ctr/AutoInt/AutoInt+_avazu_x4_001/AutoInt+_avazu_x4_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_001/AutoInt+_avazu_x4_tuner_config_03/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_avazu_x4_001/AutoInt+_avazu_x4_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_001/AutoInt_avazu_x4_001_eec3dc28.log b/ranking/ctr/AutoInt/AutoInt+_avazu_x4_001/AutoInt_avazu_x4_001_eec3dc28.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_001/AutoInt_avazu_x4_001_eec3dc28.log rename to ranking/ctr/AutoInt/AutoInt+_avazu_x4_001/AutoInt_avazu_x4_001_eec3dc28.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_001/README.md b/ranking/ctr/AutoInt/AutoInt+_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_001/README.md rename to ranking/ctr/AutoInt/AutoInt+_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_001/environments.txt b/ranking/ctr/AutoInt/AutoInt+_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_001/environments.txt rename to ranking/ctr/AutoInt/AutoInt+_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_001/results.csv b/ranking/ctr/AutoInt/AutoInt+_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_001/results.csv rename to ranking/ctr/AutoInt/AutoInt+_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_001/run_expid.py b/ranking/ctr/AutoInt/AutoInt+_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_001/run_expid.py rename to ranking/ctr/AutoInt/AutoInt+_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_002/AutoInt+_avazu_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt+_avazu_x4_002/AutoInt+_avazu_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_002/AutoInt+_avazu_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_avazu_x4_002/AutoInt+_avazu_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_002/AutoInt+_avazu_x4_tuner_config_02/model_config.yaml b/ranking/ctr/AutoInt/AutoInt+_avazu_x4_002/AutoInt+_avazu_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_002/AutoInt+_avazu_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_avazu_x4_002/AutoInt+_avazu_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_002/AutoInt_avazu_x4_003_cc789c7e.log b/ranking/ctr/AutoInt/AutoInt+_avazu_x4_002/AutoInt_avazu_x4_003_cc789c7e.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_002/AutoInt_avazu_x4_003_cc789c7e.log rename to ranking/ctr/AutoInt/AutoInt+_avazu_x4_002/AutoInt_avazu_x4_003_cc789c7e.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_002/README.md b/ranking/ctr/AutoInt/AutoInt+_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_002/README.md rename to ranking/ctr/AutoInt/AutoInt+_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_002/environments.txt b/ranking/ctr/AutoInt/AutoInt+_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_002/environments.txt rename to ranking/ctr/AutoInt/AutoInt+_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_002/results.csv b/ranking/ctr/AutoInt/AutoInt+_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_002/results.csv rename to ranking/ctr/AutoInt/AutoInt+_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_002/run_expid.py b/ranking/ctr/AutoInt/AutoInt+_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_avazu_x4_002/run_expid.py rename to ranking/ctr/AutoInt/AutoInt+_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/AutoInt+_criteo_x1_tuner_config_08.csv b/ranking/ctr/AutoInt/AutoInt+_criteo_x1/AutoInt+_criteo_x1_tuner_config_08.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/AutoInt+_criteo_x1_tuner_config_08.csv rename to ranking/ctr/AutoInt/AutoInt+_criteo_x1/AutoInt+_criteo_x1_tuner_config_08.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/AutoInt+_criteo_x1_tuner_config_08/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt+_criteo_x1/AutoInt+_criteo_x1_tuner_config_08/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/AutoInt+_criteo_x1_tuner_config_08/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_criteo_x1/AutoInt+_criteo_x1_tuner_config_08/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/AutoInt+_criteo_x1_tuner_config_08/model_config.yaml b/ranking/ctr/AutoInt/AutoInt+_criteo_x1/AutoInt+_criteo_x1_tuner_config_08/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/AutoInt+_criteo_x1_tuner_config_08/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_criteo_x1/AutoInt+_criteo_x1_tuner_config_08/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/AutoInt_criteo_x1_005_a4b5787e.log b/ranking/ctr/AutoInt/AutoInt+_criteo_x1/AutoInt_criteo_x1_005_a4b5787e.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/AutoInt_criteo_x1_005_a4b5787e.log rename to ranking/ctr/AutoInt/AutoInt+_criteo_x1/AutoInt_criteo_x1_005_a4b5787e.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/README.md b/ranking/ctr/AutoInt/AutoInt+_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/README.md rename to ranking/ctr/AutoInt/AutoInt+_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/environments.txt b/ranking/ctr/AutoInt/AutoInt+_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/environments.txt rename to ranking/ctr/AutoInt/AutoInt+_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/fuxictr_version.py b/ranking/ctr/AutoInt/AutoInt+_criteo_x1/fuxictr_version.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/fuxictr_version.py rename to ranking/ctr/AutoInt/AutoInt+_criteo_x1/fuxictr_version.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/results.csv b/ranking/ctr/AutoInt/AutoInt+_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/results.csv rename to ranking/ctr/AutoInt/AutoInt+_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/run_expid.py b/ranking/ctr/AutoInt/AutoInt+_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x1/run_expid.py rename to ranking/ctr/AutoInt/AutoInt+_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_001/AutoInt+_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt+_criteo_x4_001/AutoInt+_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_001/AutoInt+_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_criteo_x4_001/AutoInt+_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_001/AutoInt+_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/AutoInt/AutoInt+_criteo_x4_001/AutoInt+_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_001/AutoInt+_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_criteo_x4_001/AutoInt+_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_001/AutoInt_criteo_x4_009_a3d40ae3.log b/ranking/ctr/AutoInt/AutoInt+_criteo_x4_001/AutoInt_criteo_x4_009_a3d40ae3.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_001/AutoInt_criteo_x4_009_a3d40ae3.log rename to ranking/ctr/AutoInt/AutoInt+_criteo_x4_001/AutoInt_criteo_x4_009_a3d40ae3.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_001/README.md b/ranking/ctr/AutoInt/AutoInt+_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_001/README.md rename to ranking/ctr/AutoInt/AutoInt+_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_001/environments.txt b/ranking/ctr/AutoInt/AutoInt+_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_001/environments.txt rename to ranking/ctr/AutoInt/AutoInt+_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_001/results.csv b/ranking/ctr/AutoInt/AutoInt+_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_001/results.csv rename to ranking/ctr/AutoInt/AutoInt+_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_001/run_expid.py b/ranking/ctr/AutoInt/AutoInt+_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_001/run_expid.py rename to ranking/ctr/AutoInt/AutoInt+_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_002/AutoInt+_criteo_x4_tuner_config_03/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt+_criteo_x4_002/AutoInt+_criteo_x4_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_002/AutoInt+_criteo_x4_tuner_config_03/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_criteo_x4_002/AutoInt+_criteo_x4_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_002/AutoInt+_criteo_x4_tuner_config_03/model_config.yaml b/ranking/ctr/AutoInt/AutoInt+_criteo_x4_002/AutoInt+_criteo_x4_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_002/AutoInt+_criteo_x4_tuner_config_03/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_criteo_x4_002/AutoInt+_criteo_x4_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_002/AutoInt_criteo_x4_003_1a01a590.log b/ranking/ctr/AutoInt/AutoInt+_criteo_x4_002/AutoInt_criteo_x4_003_1a01a590.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_002/AutoInt_criteo_x4_003_1a01a590.log rename to ranking/ctr/AutoInt/AutoInt+_criteo_x4_002/AutoInt_criteo_x4_003_1a01a590.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_002/README.md b/ranking/ctr/AutoInt/AutoInt+_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_002/README.md rename to ranking/ctr/AutoInt/AutoInt+_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_002/environments.txt b/ranking/ctr/AutoInt/AutoInt+_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_002/environments.txt rename to ranking/ctr/AutoInt/AutoInt+_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_002/results.csv b/ranking/ctr/AutoInt/AutoInt+_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_002/results.csv rename to ranking/ctr/AutoInt/AutoInt+_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_002/run_expid.py b/ranking/ctr/AutoInt/AutoInt+_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_criteo_x4_002/run_expid.py rename to ranking/ctr/AutoInt/AutoInt+_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/AutoInt+_frappe_x1_tuner_config_06.csv b/ranking/ctr/AutoInt/AutoInt+_frappe_x1/AutoInt+_frappe_x1_tuner_config_06.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/AutoInt+_frappe_x1_tuner_config_06.csv rename to ranking/ctr/AutoInt/AutoInt+_frappe_x1/AutoInt+_frappe_x1_tuner_config_06.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/AutoInt+_frappe_x1_tuner_config_06/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt+_frappe_x1/AutoInt+_frappe_x1_tuner_config_06/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/AutoInt+_frappe_x1_tuner_config_06/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_frappe_x1/AutoInt+_frappe_x1_tuner_config_06/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/AutoInt+_frappe_x1_tuner_config_06/model_config.yaml b/ranking/ctr/AutoInt/AutoInt+_frappe_x1/AutoInt+_frappe_x1_tuner_config_06/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/AutoInt+_frappe_x1_tuner_config_06/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_frappe_x1/AutoInt+_frappe_x1_tuner_config_06/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/AutoInt_frappe_x1_006_dba8e61c.log b/ranking/ctr/AutoInt/AutoInt+_frappe_x1/AutoInt_frappe_x1_006_dba8e61c.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/AutoInt_frappe_x1_006_dba8e61c.log rename to ranking/ctr/AutoInt/AutoInt+_frappe_x1/AutoInt_frappe_x1_006_dba8e61c.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/README.md b/ranking/ctr/AutoInt/AutoInt+_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/README.md rename to ranking/ctr/AutoInt/AutoInt+_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/environments.txt b/ranking/ctr/AutoInt/AutoInt+_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/environments.txt rename to ranking/ctr/AutoInt/AutoInt+_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/fuxictr_version.py b/ranking/ctr/AutoInt/AutoInt+_frappe_x1/fuxictr_version.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/fuxictr_version.py rename to ranking/ctr/AutoInt/AutoInt+_frappe_x1/fuxictr_version.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/results.csv b/ranking/ctr/AutoInt/AutoInt+_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/results.csv rename to ranking/ctr/AutoInt/AutoInt+_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/run_expid.py b/ranking/ctr/AutoInt/AutoInt+_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_frappe_x1/run_expid.py rename to ranking/ctr/AutoInt/AutoInt+_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_kkbox_x1/AutoInt_kkbox_x1_003_df9d4ed6.log b/ranking/ctr/AutoInt/AutoInt+_kkbox_x1/AutoInt_kkbox_x1_003_df9d4ed6.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_kkbox_x1/AutoInt_kkbox_x1_003_df9d4ed6.log rename to ranking/ctr/AutoInt/AutoInt+_kkbox_x1/AutoInt_kkbox_x1_003_df9d4ed6.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_kkbox_x1/AutoInt_kkbox_x1_tuner_config_06/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt+_kkbox_x1/AutoInt_kkbox_x1_tuner_config_06/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_kkbox_x1/AutoInt_kkbox_x1_tuner_config_06/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_kkbox_x1/AutoInt_kkbox_x1_tuner_config_06/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_kkbox_x1/AutoInt_kkbox_x1_tuner_config_06/model_config.yaml b/ranking/ctr/AutoInt/AutoInt+_kkbox_x1/AutoInt_kkbox_x1_tuner_config_06/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_kkbox_x1/AutoInt_kkbox_x1_tuner_config_06/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_kkbox_x1/AutoInt_kkbox_x1_tuner_config_06/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_kkbox_x1/README.md b/ranking/ctr/AutoInt/AutoInt+_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_kkbox_x1/README.md rename to ranking/ctr/AutoInt/AutoInt+_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_kkbox_x1/environments.txt b/ranking/ctr/AutoInt/AutoInt+_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_kkbox_x1/environments.txt rename to ranking/ctr/AutoInt/AutoInt+_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_kkbox_x1/results.csv b/ranking/ctr/AutoInt/AutoInt+_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_kkbox_x1/results.csv rename to ranking/ctr/AutoInt/AutoInt+_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_kkbox_x1/run_expid.py b/ranking/ctr/AutoInt/AutoInt+_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_kkbox_x1/run_expid.py rename to ranking/ctr/AutoInt/AutoInt+_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/AutoInt+_movielenslatest_x1_tuner_config_08.csv b/ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/AutoInt+_movielenslatest_x1_tuner_config_08.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/AutoInt+_movielenslatest_x1_tuner_config_08.csv rename to ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/AutoInt+_movielenslatest_x1_tuner_config_08.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/AutoInt+_movielenslatest_x1_tuner_config_08/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/AutoInt+_movielenslatest_x1_tuner_config_08/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/AutoInt+_movielenslatest_x1_tuner_config_08/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/AutoInt+_movielenslatest_x1_tuner_config_08/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/AutoInt+_movielenslatest_x1_tuner_config_08/model_config.yaml b/ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/AutoInt+_movielenslatest_x1_tuner_config_08/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/AutoInt+_movielenslatest_x1_tuner_config_08/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/AutoInt+_movielenslatest_x1_tuner_config_08/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/AutoInt_movielenslatest_x1_006_a5e56596.log b/ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/AutoInt_movielenslatest_x1_006_a5e56596.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/AutoInt_movielenslatest_x1_006_a5e56596.log rename to ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/AutoInt_movielenslatest_x1_006_a5e56596.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/README.md b/ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/README.md rename to ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/environments.txt b/ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/environments.txt rename to ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/fuxictr_version.py b/ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/fuxictr_version.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/fuxictr_version.py rename to ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/fuxictr_version.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/results.csv b/ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/results.csv rename to ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/run_expid.py b/ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt+_movielenslatest_x1/run_expid.py rename to ranking/ctr/AutoInt/AutoInt+_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_amazonelectronics_x1/AutoInt+_amazonelectronics_x1_tuner_config_03.csv b/ranking/ctr/AutoInt/AutoInt_amazonelectronics_x1/AutoInt+_amazonelectronics_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_amazonelectronics_x1/AutoInt+_amazonelectronics_x1_tuner_config_03.csv rename to ranking/ctr/AutoInt/AutoInt_amazonelectronics_x1/AutoInt+_amazonelectronics_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_amazonelectronics_x1/AutoInt+_amazonelectronics_x1_tuner_config_03.yaml b/ranking/ctr/AutoInt/AutoInt_amazonelectronics_x1/AutoInt+_amazonelectronics_x1_tuner_config_03.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_amazonelectronics_x1/AutoInt+_amazonelectronics_x1_tuner_config_03.yaml rename to ranking/ctr/AutoInt/AutoInt_amazonelectronics_x1/AutoInt+_amazonelectronics_x1_tuner_config_03.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_amazonelectronics_x1/AutoInt+_amazonelectronics_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt_amazonelectronics_x1/AutoInt+_amazonelectronics_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_amazonelectronics_x1/AutoInt+_amazonelectronics_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt_amazonelectronics_x1/AutoInt+_amazonelectronics_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_amazonelectronics_x1/AutoInt+_amazonelectronics_x1_tuner_config_03/model_config.yaml b/ranking/ctr/AutoInt/AutoInt_amazonelectronics_x1/AutoInt+_amazonelectronics_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_amazonelectronics_x1/AutoInt+_amazonelectronics_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt_amazonelectronics_x1/AutoInt+_amazonelectronics_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_amazonelectronics_x1/AutoInt_amazonelectronics_x1_004_7a937c94.log b/ranking/ctr/AutoInt/AutoInt_amazonelectronics_x1/AutoInt_amazonelectronics_x1_004_7a937c94.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_amazonelectronics_x1/AutoInt_amazonelectronics_x1_004_7a937c94.log rename to ranking/ctr/AutoInt/AutoInt_amazonelectronics_x1/AutoInt_amazonelectronics_x1_004_7a937c94.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_amazonelectronics_x1/README.md b/ranking/ctr/AutoInt/AutoInt_amazonelectronics_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_amazonelectronics_x1/README.md rename to ranking/ctr/AutoInt/AutoInt_amazonelectronics_x1/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_amazonelectronics_x1/environments.txt b/ranking/ctr/AutoInt/AutoInt_amazonelectronics_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_amazonelectronics_x1/environments.txt rename to ranking/ctr/AutoInt/AutoInt_amazonelectronics_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_amazonelectronics_x1/results.csv b/ranking/ctr/AutoInt/AutoInt_amazonelectronics_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_amazonelectronics_x1/results.csv rename to ranking/ctr/AutoInt/AutoInt_amazonelectronics_x1/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/AutoInt_avazu_x1_001_af7937e6.log b/ranking/ctr/AutoInt/AutoInt_avazu_x1/AutoInt_avazu_x1_001_af7937e6.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/AutoInt_avazu_x1_001_af7937e6.log rename to ranking/ctr/AutoInt/AutoInt_avazu_x1/AutoInt_avazu_x1_001_af7937e6.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/AutoInt_avazu_x1_tuner_config_03.csv b/ranking/ctr/AutoInt/AutoInt_avazu_x1/AutoInt_avazu_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/AutoInt_avazu_x1_tuner_config_03.csv rename to ranking/ctr/AutoInt/AutoInt_avazu_x1/AutoInt_avazu_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/AutoInt_avazu_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt_avazu_x1/AutoInt_avazu_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/AutoInt_avazu_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt_avazu_x1/AutoInt_avazu_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/AutoInt_avazu_x1_tuner_config_03/model_config.yaml b/ranking/ctr/AutoInt/AutoInt_avazu_x1/AutoInt_avazu_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/AutoInt_avazu_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt_avazu_x1/AutoInt_avazu_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/README.md b/ranking/ctr/AutoInt/AutoInt_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/README.md rename to ranking/ctr/AutoInt/AutoInt_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/environments.txt b/ranking/ctr/AutoInt/AutoInt_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/environments.txt rename to ranking/ctr/AutoInt/AutoInt_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/fuxictr_version.py b/ranking/ctr/AutoInt/AutoInt_avazu_x1/fuxictr_version.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/fuxictr_version.py rename to ranking/ctr/AutoInt/AutoInt_avazu_x1/fuxictr_version.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/results.csv b/ranking/ctr/AutoInt/AutoInt_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/results.csv rename to ranking/ctr/AutoInt/AutoInt_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/run_expid.py b/ranking/ctr/AutoInt/AutoInt_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x1/run_expid.py rename to ranking/ctr/AutoInt/AutoInt_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_001/AutoInt_avazu_x4_048_8c3b072b.log b/ranking/ctr/AutoInt/AutoInt_avazu_x4_001/AutoInt_avazu_x4_048_8c3b072b.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_001/AutoInt_avazu_x4_048_8c3b072b.log rename to ranking/ctr/AutoInt/AutoInt_avazu_x4_001/AutoInt_avazu_x4_048_8c3b072b.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_001/AutoInt_avazu_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt_avazu_x4_001/AutoInt_avazu_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_001/AutoInt_avazu_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt_avazu_x4_001/AutoInt_avazu_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_001/AutoInt_avazu_x4_tuner_config_02/model_config.yaml b/ranking/ctr/AutoInt/AutoInt_avazu_x4_001/AutoInt_avazu_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_001/AutoInt_avazu_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt_avazu_x4_001/AutoInt_avazu_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_001/README.md b/ranking/ctr/AutoInt/AutoInt_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_001/README.md rename to ranking/ctr/AutoInt/AutoInt_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_001/environments.txt b/ranking/ctr/AutoInt/AutoInt_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_001/environments.txt rename to ranking/ctr/AutoInt/AutoInt_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_001/results.csv b/ranking/ctr/AutoInt/AutoInt_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_001/results.csv rename to ranking/ctr/AutoInt/AutoInt_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_001/run_expid.py b/ranking/ctr/AutoInt/AutoInt_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_001/run_expid.py rename to ranking/ctr/AutoInt/AutoInt_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_002/AutoInt_avazu_x4_013_3a66ab94.log b/ranking/ctr/AutoInt/AutoInt_avazu_x4_002/AutoInt_avazu_x4_013_3a66ab94.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_002/AutoInt_avazu_x4_013_3a66ab94.log rename to ranking/ctr/AutoInt/AutoInt_avazu_x4_002/AutoInt_avazu_x4_013_3a66ab94.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_002/AutoInt_avazu_x4_tuner_config_04/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt_avazu_x4_002/AutoInt_avazu_x4_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_002/AutoInt_avazu_x4_tuner_config_04/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt_avazu_x4_002/AutoInt_avazu_x4_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_002/AutoInt_avazu_x4_tuner_config_04/model_config.yaml b/ranking/ctr/AutoInt/AutoInt_avazu_x4_002/AutoInt_avazu_x4_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_002/AutoInt_avazu_x4_tuner_config_04/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt_avazu_x4_002/AutoInt_avazu_x4_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_002/README.md b/ranking/ctr/AutoInt/AutoInt_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_002/README.md rename to ranking/ctr/AutoInt/AutoInt_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_002/environments.txt b/ranking/ctr/AutoInt/AutoInt_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_002/environments.txt rename to ranking/ctr/AutoInt/AutoInt_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_002/results.csv b/ranking/ctr/AutoInt/AutoInt_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_002/results.csv rename to ranking/ctr/AutoInt/AutoInt_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_002/run_expid.py b/ranking/ctr/AutoInt/AutoInt_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_avazu_x4_002/run_expid.py rename to ranking/ctr/AutoInt/AutoInt_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/AutoInt_criteo_x1_012_e71394ec.log b/ranking/ctr/AutoInt/AutoInt_criteo_x1/AutoInt_criteo_x1_012_e71394ec.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/AutoInt_criteo_x1_012_e71394ec.log rename to ranking/ctr/AutoInt/AutoInt_criteo_x1/AutoInt_criteo_x1_012_e71394ec.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/AutoInt_criteo_x1_tuner_config_04.csv b/ranking/ctr/AutoInt/AutoInt_criteo_x1/AutoInt_criteo_x1_tuner_config_04.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/AutoInt_criteo_x1_tuner_config_04.csv rename to ranking/ctr/AutoInt/AutoInt_criteo_x1/AutoInt_criteo_x1_tuner_config_04.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/AutoInt_criteo_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt_criteo_x1/AutoInt_criteo_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/AutoInt_criteo_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt_criteo_x1/AutoInt_criteo_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/AutoInt_criteo_x1_tuner_config_04/model_config.yaml b/ranking/ctr/AutoInt/AutoInt_criteo_x1/AutoInt_criteo_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/AutoInt_criteo_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt_criteo_x1/AutoInt_criteo_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/README.md b/ranking/ctr/AutoInt/AutoInt_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/README.md rename to ranking/ctr/AutoInt/AutoInt_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/environments.txt b/ranking/ctr/AutoInt/AutoInt_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/environments.txt rename to ranking/ctr/AutoInt/AutoInt_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/fuxictr_version.py b/ranking/ctr/AutoInt/AutoInt_criteo_x1/fuxictr_version.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/fuxictr_version.py rename to ranking/ctr/AutoInt/AutoInt_criteo_x1/fuxictr_version.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/results.csv b/ranking/ctr/AutoInt/AutoInt_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/results.csv rename to ranking/ctr/AutoInt/AutoInt_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/run_expid.py b/ranking/ctr/AutoInt/AutoInt_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x1/run_expid.py rename to ranking/ctr/AutoInt/AutoInt_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_001/AutoInt_criteo_x4_024_e721f8ce.log b/ranking/ctr/AutoInt/AutoInt_criteo_x4_001/AutoInt_criteo_x4_024_e721f8ce.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_001/AutoInt_criteo_x4_024_e721f8ce.log rename to ranking/ctr/AutoInt/AutoInt_criteo_x4_001/AutoInt_criteo_x4_024_e721f8ce.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_001/AutoInt_criteo_x4_tuner_config_11/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt_criteo_x4_001/AutoInt_criteo_x4_tuner_config_11/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_001/AutoInt_criteo_x4_tuner_config_11/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt_criteo_x4_001/AutoInt_criteo_x4_tuner_config_11/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_001/AutoInt_criteo_x4_tuner_config_11/model_config.yaml b/ranking/ctr/AutoInt/AutoInt_criteo_x4_001/AutoInt_criteo_x4_tuner_config_11/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_001/AutoInt_criteo_x4_tuner_config_11/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt_criteo_x4_001/AutoInt_criteo_x4_tuner_config_11/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_001/README.md b/ranking/ctr/AutoInt/AutoInt_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_001/README.md rename to ranking/ctr/AutoInt/AutoInt_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_001/environments.txt b/ranking/ctr/AutoInt/AutoInt_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_001/environments.txt rename to ranking/ctr/AutoInt/AutoInt_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_001/results.csv b/ranking/ctr/AutoInt/AutoInt_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_001/results.csv rename to ranking/ctr/AutoInt/AutoInt_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_001/run_expid.py b/ranking/ctr/AutoInt/AutoInt_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_001/run_expid.py rename to ranking/ctr/AutoInt/AutoInt_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_002/AutoInt_criteo_x4_015_f1bb9713.log b/ranking/ctr/AutoInt/AutoInt_criteo_x4_002/AutoInt_criteo_x4_015_f1bb9713.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_002/AutoInt_criteo_x4_015_f1bb9713.log rename to ranking/ctr/AutoInt/AutoInt_criteo_x4_002/AutoInt_criteo_x4_015_f1bb9713.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_002/AutoInt_criteo_x4_tuner_config_11/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt_criteo_x4_002/AutoInt_criteo_x4_tuner_config_11/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_002/AutoInt_criteo_x4_tuner_config_11/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt_criteo_x4_002/AutoInt_criteo_x4_tuner_config_11/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_002/AutoInt_criteo_x4_tuner_config_11/model_config.yaml b/ranking/ctr/AutoInt/AutoInt_criteo_x4_002/AutoInt_criteo_x4_tuner_config_11/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_002/AutoInt_criteo_x4_tuner_config_11/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt_criteo_x4_002/AutoInt_criteo_x4_tuner_config_11/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_002/README.md b/ranking/ctr/AutoInt/AutoInt_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_002/README.md rename to ranking/ctr/AutoInt/AutoInt_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_002/environments.txt b/ranking/ctr/AutoInt/AutoInt_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_002/environments.txt rename to ranking/ctr/AutoInt/AutoInt_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_002/results.csv b/ranking/ctr/AutoInt/AutoInt_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_002/results.csv rename to ranking/ctr/AutoInt/AutoInt_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_002/run_expid.py b/ranking/ctr/AutoInt/AutoInt_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_criteo_x4_002/run_expid.py rename to ranking/ctr/AutoInt/AutoInt_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/AutoInt_frappe_x1_005_2b296630.log b/ranking/ctr/AutoInt/AutoInt_frappe_x1/AutoInt_frappe_x1_005_2b296630.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/AutoInt_frappe_x1_005_2b296630.log rename to ranking/ctr/AutoInt/AutoInt_frappe_x1/AutoInt_frappe_x1_005_2b296630.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/AutoInt_frappe_x1_tuner_config_05.csv b/ranking/ctr/AutoInt/AutoInt_frappe_x1/AutoInt_frappe_x1_tuner_config_05.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/AutoInt_frappe_x1_tuner_config_05.csv rename to ranking/ctr/AutoInt/AutoInt_frappe_x1/AutoInt_frappe_x1_tuner_config_05.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/AutoInt_frappe_x1_tuner_config_05/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt_frappe_x1/AutoInt_frappe_x1_tuner_config_05/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/AutoInt_frappe_x1_tuner_config_05/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt_frappe_x1/AutoInt_frappe_x1_tuner_config_05/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/AutoInt_frappe_x1_tuner_config_05/model_config.yaml b/ranking/ctr/AutoInt/AutoInt_frappe_x1/AutoInt_frappe_x1_tuner_config_05/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/AutoInt_frappe_x1_tuner_config_05/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt_frappe_x1/AutoInt_frappe_x1_tuner_config_05/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/README.md b/ranking/ctr/AutoInt/AutoInt_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/README.md rename to ranking/ctr/AutoInt/AutoInt_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/environments.txt b/ranking/ctr/AutoInt/AutoInt_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/environments.txt rename to ranking/ctr/AutoInt/AutoInt_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/fuxictr_version.py b/ranking/ctr/AutoInt/AutoInt_frappe_x1/fuxictr_version.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/fuxictr_version.py rename to ranking/ctr/AutoInt/AutoInt_frappe_x1/fuxictr_version.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/results.csv b/ranking/ctr/AutoInt/AutoInt_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/results.csv rename to ranking/ctr/AutoInt/AutoInt_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/run_expid.py b/ranking/ctr/AutoInt/AutoInt_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_frappe_x1/run_expid.py rename to ranking/ctr/AutoInt/AutoInt_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_kkbox_x1/AutoInt_kkbox_x1_004_85296442.log b/ranking/ctr/AutoInt/AutoInt_kkbox_x1/AutoInt_kkbox_x1_004_85296442.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_kkbox_x1/AutoInt_kkbox_x1_004_85296442.log rename to ranking/ctr/AutoInt/AutoInt_kkbox_x1/AutoInt_kkbox_x1_004_85296442.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_kkbox_x1/AutoInt_kkbox_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt_kkbox_x1/AutoInt_kkbox_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_kkbox_x1/AutoInt_kkbox_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt_kkbox_x1/AutoInt_kkbox_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_kkbox_x1/AutoInt_kkbox_x1_tuner_config_03/model_config.yaml b/ranking/ctr/AutoInt/AutoInt_kkbox_x1/AutoInt_kkbox_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_kkbox_x1/AutoInt_kkbox_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt_kkbox_x1/AutoInt_kkbox_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_kkbox_x1/README.md b/ranking/ctr/AutoInt/AutoInt_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_kkbox_x1/README.md rename to ranking/ctr/AutoInt/AutoInt_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_kkbox_x1/environments.txt b/ranking/ctr/AutoInt/AutoInt_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_kkbox_x1/environments.txt rename to ranking/ctr/AutoInt/AutoInt_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_kkbox_x1/results.csv b/ranking/ctr/AutoInt/AutoInt_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_kkbox_x1/results.csv rename to ranking/ctr/AutoInt/AutoInt_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_kkbox_x1/run_expid.py b/ranking/ctr/AutoInt/AutoInt_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_kkbox_x1/run_expid.py rename to ranking/ctr/AutoInt/AutoInt_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_kuaivideo_x1/AutoInt+_kuaivideo_x1_tuner_config_02.csv b/ranking/ctr/AutoInt/AutoInt_kuaivideo_x1/AutoInt+_kuaivideo_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_kuaivideo_x1/AutoInt+_kuaivideo_x1_tuner_config_02.csv rename to ranking/ctr/AutoInt/AutoInt_kuaivideo_x1/AutoInt+_kuaivideo_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_kuaivideo_x1/AutoInt+_kuaivideo_x1_tuner_config_02.yaml b/ranking/ctr/AutoInt/AutoInt_kuaivideo_x1/AutoInt+_kuaivideo_x1_tuner_config_02.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_kuaivideo_x1/AutoInt+_kuaivideo_x1_tuner_config_02.yaml rename to ranking/ctr/AutoInt/AutoInt_kuaivideo_x1/AutoInt+_kuaivideo_x1_tuner_config_02.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_kuaivideo_x1/AutoInt+_kuaivideo_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt_kuaivideo_x1/AutoInt+_kuaivideo_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_kuaivideo_x1/AutoInt+_kuaivideo_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt_kuaivideo_x1/AutoInt+_kuaivideo_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_kuaivideo_x1/AutoInt+_kuaivideo_x1_tuner_config_02/model_config.yaml b/ranking/ctr/AutoInt/AutoInt_kuaivideo_x1/AutoInt+_kuaivideo_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_kuaivideo_x1/AutoInt+_kuaivideo_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt_kuaivideo_x1/AutoInt+_kuaivideo_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_kuaivideo_x1/AutoInt_kuaivideo_x1_011_9ed2831b.log b/ranking/ctr/AutoInt/AutoInt_kuaivideo_x1/AutoInt_kuaivideo_x1_011_9ed2831b.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_kuaivideo_x1/AutoInt_kuaivideo_x1_011_9ed2831b.log rename to ranking/ctr/AutoInt/AutoInt_kuaivideo_x1/AutoInt_kuaivideo_x1_011_9ed2831b.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_kuaivideo_x1/README.md b/ranking/ctr/AutoInt/AutoInt_kuaivideo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_kuaivideo_x1/README.md rename to ranking/ctr/AutoInt/AutoInt_kuaivideo_x1/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_kuaivideo_x1/environments.txt b/ranking/ctr/AutoInt/AutoInt_kuaivideo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_kuaivideo_x1/environments.txt rename to ranking/ctr/AutoInt/AutoInt_kuaivideo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_kuaivideo_x1/results.csv b/ranking/ctr/AutoInt/AutoInt_kuaivideo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_kuaivideo_x1/results.csv rename to ranking/ctr/AutoInt/AutoInt_kuaivideo_x1/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt+_microvideo1.7m_x1_tuner_config_08.csv b/ranking/ctr/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt+_microvideo1.7m_x1_tuner_config_08.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt+_microvideo1.7m_x1_tuner_config_08.csv rename to ranking/ctr/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt+_microvideo1.7m_x1_tuner_config_08.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt+_microvideo1.7m_x1_tuner_config_08.yaml b/ranking/ctr/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt+_microvideo1.7m_x1_tuner_config_08.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt+_microvideo1.7m_x1_tuner_config_08.yaml rename to ranking/ctr/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt+_microvideo1.7m_x1_tuner_config_08.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt+_microvideo1.7m_x1_tuner_config_08/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt+_microvideo1.7m_x1_tuner_config_08/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt+_microvideo1.7m_x1_tuner_config_08/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt+_microvideo1.7m_x1_tuner_config_08/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt+_microvideo1.7m_x1_tuner_config_08/model_config.yaml b/ranking/ctr/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt+_microvideo1.7m_x1_tuner_config_08/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt+_microvideo1.7m_x1_tuner_config_08/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt+_microvideo1.7m_x1_tuner_config_08/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt_microvideo1.7m_x1_029_f813da5f.log b/ranking/ctr/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt_microvideo1.7m_x1_029_f813da5f.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt_microvideo1.7m_x1_029_f813da5f.log rename to ranking/ctr/AutoInt/AutoInt_microvideo1.7m_x1/AutoInt_microvideo1.7m_x1_029_f813da5f.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_microvideo1.7m_x1/README.md b/ranking/ctr/AutoInt/AutoInt_microvideo1.7m_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_microvideo1.7m_x1/README.md rename to ranking/ctr/AutoInt/AutoInt_microvideo1.7m_x1/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_microvideo1.7m_x1/environments.txt b/ranking/ctr/AutoInt/AutoInt_microvideo1.7m_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_microvideo1.7m_x1/environments.txt rename to ranking/ctr/AutoInt/AutoInt_microvideo1.7m_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_microvideo1.7m_x1/results.csv b/ranking/ctr/AutoInt/AutoInt_microvideo1.7m_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_microvideo1.7m_x1/results.csv rename to ranking/ctr/AutoInt/AutoInt_microvideo1.7m_x1/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/AutoInt_movielenslatest_x1_004_4795ccb3.log b/ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/AutoInt_movielenslatest_x1_004_4795ccb3.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/AutoInt_movielenslatest_x1_004_4795ccb3.log rename to ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/AutoInt_movielenslatest_x1_004_4795ccb3.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/AutoInt_movielenslatest_x1_tuner_config_04.csv b/ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/AutoInt_movielenslatest_x1_tuner_config_04.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/AutoInt_movielenslatest_x1_tuner_config_04.csv rename to ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/AutoInt_movielenslatest_x1_tuner_config_04.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/AutoInt_movielenslatest_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/AutoInt_movielenslatest_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/AutoInt_movielenslatest_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/AutoInt_movielenslatest_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/AutoInt_movielenslatest_x1_tuner_config_04/model_config.yaml b/ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/AutoInt_movielenslatest_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/AutoInt_movielenslatest_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/AutoInt_movielenslatest_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/README.md b/ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/README.md rename to ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/environments.txt b/ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/environments.txt rename to ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/fuxictr_version.py b/ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/fuxictr_version.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/fuxictr_version.py rename to ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/fuxictr_version.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/results.csv b/ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/results.csv rename to ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/run_expid.py b/ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_movielenslatest_x1/run_expid.py rename to ranking/ctr/AutoInt/AutoInt_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_taobaoad_x1/AutoInt+_taobaoad_x1_tuner_config_02.csv b/ranking/ctr/AutoInt/AutoInt_taobaoad_x1/AutoInt+_taobaoad_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_taobaoad_x1/AutoInt+_taobaoad_x1_tuner_config_02.csv rename to ranking/ctr/AutoInt/AutoInt_taobaoad_x1/AutoInt+_taobaoad_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_taobaoad_x1/AutoInt+_taobaoad_x1_tuner_config_02.yaml b/ranking/ctr/AutoInt/AutoInt_taobaoad_x1/AutoInt+_taobaoad_x1_tuner_config_02.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_taobaoad_x1/AutoInt+_taobaoad_x1_tuner_config_02.yaml rename to ranking/ctr/AutoInt/AutoInt_taobaoad_x1/AutoInt+_taobaoad_x1_tuner_config_02.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_taobaoad_x1/AutoInt+_taobaoad_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/AutoInt/AutoInt_taobaoad_x1/AutoInt+_taobaoad_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_taobaoad_x1/AutoInt+_taobaoad_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/AutoInt/AutoInt_taobaoad_x1/AutoInt+_taobaoad_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_taobaoad_x1/AutoInt+_taobaoad_x1_tuner_config_02/model_config.yaml b/ranking/ctr/AutoInt/AutoInt_taobaoad_x1/AutoInt+_taobaoad_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_taobaoad_x1/AutoInt+_taobaoad_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/AutoInt/AutoInt_taobaoad_x1/AutoInt+_taobaoad_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_taobaoad_x1/AutoInt_taobaoad_x1_012_778cf7e5.log b/ranking/ctr/AutoInt/AutoInt_taobaoad_x1/AutoInt_taobaoad_x1_012_778cf7e5.log similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_taobaoad_x1/AutoInt_taobaoad_x1_012_778cf7e5.log rename to ranking/ctr/AutoInt/AutoInt_taobaoad_x1/AutoInt_taobaoad_x1_012_778cf7e5.log diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_taobaoad_x1/README.md b/ranking/ctr/AutoInt/AutoInt_taobaoad_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_taobaoad_x1/README.md rename to ranking/ctr/AutoInt/AutoInt_taobaoad_x1/README.md diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_taobaoad_x1/environments.txt b/ranking/ctr/AutoInt/AutoInt_taobaoad_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_taobaoad_x1/environments.txt rename to ranking/ctr/AutoInt/AutoInt_taobaoad_x1/environments.txt diff --git a/ctr_prediction/benchmarks/AutoInt/AutoInt_taobaoad_x1/results.csv b/ranking/ctr/AutoInt/AutoInt_taobaoad_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/AutoInt/AutoInt_taobaoad_x1/results.csv rename to ranking/ctr/AutoInt/AutoInt_taobaoad_x1/results.csv diff --git a/ctr_prediction/benchmarks/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_044_3f4303a7.log b/ranking/ctr/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_044_3f4303a7.log similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_044_3f4303a7.log rename to ranking/ctr/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_044_3f4303a7.log diff --git a/ctr_prediction/benchmarks/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_tuner_config_09.csv b/ranking/ctr/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_tuner_config_09.csv similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_tuner_config_09.csv rename to ranking/ctr/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_tuner_config_09.csv diff --git a/ctr_prediction/benchmarks/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_tuner_config_09.yaml b/ranking/ctr/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_tuner_config_09.yaml similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_tuner_config_09.yaml rename to ranking/ctr/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_tuner_config_09.yaml diff --git a/ctr_prediction/benchmarks/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_tuner_config_09/dataset_config.yaml b/ranking/ctr/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_tuner_config_09/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_tuner_config_09/dataset_config.yaml rename to ranking/ctr/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_tuner_config_09/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_tuner_config_09/model_config.yaml b/ranking/ctr/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_tuner_config_09/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_tuner_config_09/model_config.yaml rename to ranking/ctr/BST/BST_amazonelectronics_x1/BST_amazonelectronics_x1_tuner_config_09/model_config.yaml diff --git a/ctr_prediction/benchmarks/BST/BST_amazonelectronics_x1/README.md b/ranking/ctr/BST/BST_amazonelectronics_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_amazonelectronics_x1/README.md rename to ranking/ctr/BST/BST_amazonelectronics_x1/README.md diff --git a/ctr_prediction/benchmarks/BST/BST_amazonelectronics_x1/environments.txt b/ranking/ctr/BST/BST_amazonelectronics_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_amazonelectronics_x1/environments.txt rename to ranking/ctr/BST/BST_amazonelectronics_x1/environments.txt diff --git a/ctr_prediction/benchmarks/BST/BST_amazonelectronics_x1/results.csv b/ranking/ctr/BST/BST_amazonelectronics_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_amazonelectronics_x1/results.csv rename to ranking/ctr/BST/BST_amazonelectronics_x1/results.csv diff --git a/ctr_prediction/benchmarks/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_003_7ed4faca.log b/ranking/ctr/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_003_7ed4faca.log similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_003_7ed4faca.log rename to ranking/ctr/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_003_7ed4faca.log diff --git a/ctr_prediction/benchmarks/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_tuner_config_02.csv b/ranking/ctr/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_tuner_config_02.csv rename to ranking/ctr/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_tuner_config_02.yaml b/ranking/ctr/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_tuner_config_02.yaml similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_tuner_config_02.yaml rename to ranking/ctr/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_tuner_config_02.yaml diff --git a/ctr_prediction/benchmarks/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_tuner_config_02/model_config.yaml b/ranking/ctr/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/BST/BST_kuaivideo_x1/BST_kuaivideo_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/BST/BST_kuaivideo_x1/README.md b/ranking/ctr/BST/BST_kuaivideo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_kuaivideo_x1/README.md rename to ranking/ctr/BST/BST_kuaivideo_x1/README.md diff --git a/ctr_prediction/benchmarks/BST/BST_kuaivideo_x1/environments.txt b/ranking/ctr/BST/BST_kuaivideo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_kuaivideo_x1/environments.txt rename to ranking/ctr/BST/BST_kuaivideo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/BST/BST_kuaivideo_x1/results.csv b/ranking/ctr/BST/BST_kuaivideo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_kuaivideo_x1/results.csv rename to ranking/ctr/BST/BST_kuaivideo_x1/results.csv diff --git a/ctr_prediction/benchmarks/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_021_c05a2c83.log b/ranking/ctr/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_021_c05a2c83.log similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_021_c05a2c83.log rename to ranking/ctr/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_021_c05a2c83.log diff --git a/ctr_prediction/benchmarks/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_tuner_config_04.csv b/ranking/ctr/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_tuner_config_04.csv similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_tuner_config_04.csv rename to ranking/ctr/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_tuner_config_04.csv diff --git a/ctr_prediction/benchmarks/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_tuner_config_04.yaml b/ranking/ctr/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_tuner_config_04.yaml similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_tuner_config_04.yaml rename to ranking/ctr/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_tuner_config_04.yaml diff --git a/ctr_prediction/benchmarks/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_tuner_config_04/model_config.yaml b/ranking/ctr/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/BST/BST_microvideo1.7m_x1/BST_microvideo1.7m_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/BST/BST_microvideo1.7m_x1/README.md b/ranking/ctr/BST/BST_microvideo1.7m_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_microvideo1.7m_x1/README.md rename to ranking/ctr/BST/BST_microvideo1.7m_x1/README.md diff --git a/ctr_prediction/benchmarks/BST/BST_microvideo1.7m_x1/environments.txt b/ranking/ctr/BST/BST_microvideo1.7m_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_microvideo1.7m_x1/environments.txt rename to ranking/ctr/BST/BST_microvideo1.7m_x1/environments.txt diff --git a/ctr_prediction/benchmarks/BST/BST_microvideo1.7m_x1/results.csv b/ranking/ctr/BST/BST_microvideo1.7m_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_microvideo1.7m_x1/results.csv rename to ranking/ctr/BST/BST_microvideo1.7m_x1/results.csv diff --git a/ctr_prediction/benchmarks/BST/BST_taobaoad_x1/BST_taobaoad_x1_021_e30ae99a.log b/ranking/ctr/BST/BST_taobaoad_x1/BST_taobaoad_x1_021_e30ae99a.log similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_taobaoad_x1/BST_taobaoad_x1_021_e30ae99a.log rename to ranking/ctr/BST/BST_taobaoad_x1/BST_taobaoad_x1_021_e30ae99a.log diff --git a/ctr_prediction/benchmarks/BST/BST_taobaoad_x1/BST_taobaoad_x1_tuner_config_03.csv b/ranking/ctr/BST/BST_taobaoad_x1/BST_taobaoad_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_taobaoad_x1/BST_taobaoad_x1_tuner_config_03.csv rename to ranking/ctr/BST/BST_taobaoad_x1/BST_taobaoad_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/BST/BST_taobaoad_x1/BST_taobaoad_x1_tuner_config_03.yaml b/ranking/ctr/BST/BST_taobaoad_x1/BST_taobaoad_x1_tuner_config_03.yaml similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_taobaoad_x1/BST_taobaoad_x1_tuner_config_03.yaml rename to ranking/ctr/BST/BST_taobaoad_x1/BST_taobaoad_x1_tuner_config_03.yaml diff --git a/ctr_prediction/benchmarks/BST/BST_taobaoad_x1/BST_taobaoad_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/BST/BST_taobaoad_x1/BST_taobaoad_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_taobaoad_x1/BST_taobaoad_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/BST/BST_taobaoad_x1/BST_taobaoad_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/BST/BST_taobaoad_x1/BST_taobaoad_x1_tuner_config_03/model_config.yaml b/ranking/ctr/BST/BST_taobaoad_x1/BST_taobaoad_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_taobaoad_x1/BST_taobaoad_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/BST/BST_taobaoad_x1/BST_taobaoad_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/BST/BST_taobaoad_x1/README.md b/ranking/ctr/BST/BST_taobaoad_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_taobaoad_x1/README.md rename to ranking/ctr/BST/BST_taobaoad_x1/README.md diff --git a/ctr_prediction/benchmarks/BST/BST_taobaoad_x1/environments.txt b/ranking/ctr/BST/BST_taobaoad_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_taobaoad_x1/environments.txt rename to ranking/ctr/BST/BST_taobaoad_x1/environments.txt diff --git a/ctr_prediction/benchmarks/BST/BST_taobaoad_x1/results.csv b/ranking/ctr/BST/BST_taobaoad_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/BST/BST_taobaoad_x1/results.csv rename to ranking/ctr/BST/BST_taobaoad_x1/results.csv diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_001/CCPM_avazu_x4_017_75c68752.log b/ranking/ctr/CCPM/CCPM_avazu_x4_001/CCPM_avazu_x4_017_75c68752.log similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_001/CCPM_avazu_x4_017_75c68752.log rename to ranking/ctr/CCPM/CCPM_avazu_x4_001/CCPM_avazu_x4_017_75c68752.log diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_001/CCPM_avazu_x4_tuner_config_04/dataset_config.yaml b/ranking/ctr/CCPM/CCPM_avazu_x4_001/CCPM_avazu_x4_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_001/CCPM_avazu_x4_tuner_config_04/dataset_config.yaml rename to ranking/ctr/CCPM/CCPM_avazu_x4_001/CCPM_avazu_x4_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_001/CCPM_avazu_x4_tuner_config_04/model_config.yaml b/ranking/ctr/CCPM/CCPM_avazu_x4_001/CCPM_avazu_x4_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_001/CCPM_avazu_x4_tuner_config_04/model_config.yaml rename to ranking/ctr/CCPM/CCPM_avazu_x4_001/CCPM_avazu_x4_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_001/README.md b/ranking/ctr/CCPM/CCPM_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_001/README.md rename to ranking/ctr/CCPM/CCPM_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_001/environments.txt b/ranking/ctr/CCPM/CCPM_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_001/environments.txt rename to ranking/ctr/CCPM/CCPM_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_001/results.csv b/ranking/ctr/CCPM/CCPM_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_001/results.csv rename to ranking/ctr/CCPM/CCPM_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_001/run_expid.py b/ranking/ctr/CCPM/CCPM_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_001/run_expid.py rename to ranking/ctr/CCPM/CCPM_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_002/CCPM_avazu_x4_025_e2254c14.log b/ranking/ctr/CCPM/CCPM_avazu_x4_002/CCPM_avazu_x4_025_e2254c14.log similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_002/CCPM_avazu_x4_025_e2254c14.log rename to ranking/ctr/CCPM/CCPM_avazu_x4_002/CCPM_avazu_x4_025_e2254c14.log diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_002/CCPM_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/CCPM/CCPM_avazu_x4_002/CCPM_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_002/CCPM_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/CCPM/CCPM_avazu_x4_002/CCPM_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_002/CCPM_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/CCPM/CCPM_avazu_x4_002/CCPM_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_002/CCPM_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/CCPM/CCPM_avazu_x4_002/CCPM_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_002/README.md b/ranking/ctr/CCPM/CCPM_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_002/README.md rename to ranking/ctr/CCPM/CCPM_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_002/environments.txt b/ranking/ctr/CCPM/CCPM_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_002/environments.txt rename to ranking/ctr/CCPM/CCPM_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_002/results.csv b/ranking/ctr/CCPM/CCPM_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_002/results.csv rename to ranking/ctr/CCPM/CCPM_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_002/run_expid.py b/ranking/ctr/CCPM/CCPM_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_avazu_x4_002/run_expid.py rename to ranking/ctr/CCPM/CCPM_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_001/CCPM_criteo_x4_008_9b98f944.log b/ranking/ctr/CCPM/CCPM_criteo_x4_001/CCPM_criteo_x4_008_9b98f944.log similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_001/CCPM_criteo_x4_008_9b98f944.log rename to ranking/ctr/CCPM/CCPM_criteo_x4_001/CCPM_criteo_x4_008_9b98f944.log diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_001/CCPM_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/CCPM/CCPM_criteo_x4_001/CCPM_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_001/CCPM_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/CCPM/CCPM_criteo_x4_001/CCPM_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_001/CCPM_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/CCPM/CCPM_criteo_x4_001/CCPM_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_001/CCPM_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/CCPM/CCPM_criteo_x4_001/CCPM_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_001/README.md b/ranking/ctr/CCPM/CCPM_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_001/README.md rename to ranking/ctr/CCPM/CCPM_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_001/environments.txt b/ranking/ctr/CCPM/CCPM_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_001/environments.txt rename to ranking/ctr/CCPM/CCPM_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_001/results.csv b/ranking/ctr/CCPM/CCPM_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_001/results.csv rename to ranking/ctr/CCPM/CCPM_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_001/run_expid.py b/ranking/ctr/CCPM/CCPM_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_001/run_expid.py rename to ranking/ctr/CCPM/CCPM_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_002/CCPM_criteo_x4_011_3f46109b.log b/ranking/ctr/CCPM/CCPM_criteo_x4_002/CCPM_criteo_x4_011_3f46109b.log similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_002/CCPM_criteo_x4_011_3f46109b.log rename to ranking/ctr/CCPM/CCPM_criteo_x4_002/CCPM_criteo_x4_011_3f46109b.log diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_002/CCPM_criteo_x4_tuner_config_04/dataset_config.yaml b/ranking/ctr/CCPM/CCPM_criteo_x4_002/CCPM_criteo_x4_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_002/CCPM_criteo_x4_tuner_config_04/dataset_config.yaml rename to ranking/ctr/CCPM/CCPM_criteo_x4_002/CCPM_criteo_x4_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_002/CCPM_criteo_x4_tuner_config_04/model_config.yaml b/ranking/ctr/CCPM/CCPM_criteo_x4_002/CCPM_criteo_x4_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_002/CCPM_criteo_x4_tuner_config_04/model_config.yaml rename to ranking/ctr/CCPM/CCPM_criteo_x4_002/CCPM_criteo_x4_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_002/README.md b/ranking/ctr/CCPM/CCPM_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_002/README.md rename to ranking/ctr/CCPM/CCPM_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_002/environments.txt b/ranking/ctr/CCPM/CCPM_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_002/environments.txt rename to ranking/ctr/CCPM/CCPM_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_002/results.csv b/ranking/ctr/CCPM/CCPM_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_002/results.csv rename to ranking/ctr/CCPM/CCPM_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_002/run_expid.py b/ranking/ctr/CCPM/CCPM_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/CCPM/CCPM_criteo_x4_002/run_expid.py rename to ranking/ctr/CCPM/CCPM_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x1/CrossNet_avazu_x1_tuner_config_01.csv b/ranking/ctr/DCN/CrossNet_avazu_x1/CrossNet_avazu_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x1/CrossNet_avazu_x1_tuner_config_01.csv rename to ranking/ctr/DCN/CrossNet_avazu_x1/CrossNet_avazu_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x1/CrossNet_avazu_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DCN/CrossNet_avazu_x1/CrossNet_avazu_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x1/CrossNet_avazu_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DCN/CrossNet_avazu_x1/CrossNet_avazu_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x1/CrossNet_avazu_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DCN/CrossNet_avazu_x1/CrossNet_avazu_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x1/CrossNet_avazu_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DCN/CrossNet_avazu_x1/CrossNet_avazu_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x1/DCN_avazu_x1_002_ffba88fe.log b/ranking/ctr/DCN/CrossNet_avazu_x1/DCN_avazu_x1_002_ffba88fe.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x1/DCN_avazu_x1_002_ffba88fe.log rename to ranking/ctr/DCN/CrossNet_avazu_x1/DCN_avazu_x1_002_ffba88fe.log diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x1/README.md b/ranking/ctr/DCN/CrossNet_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x1/README.md rename to ranking/ctr/DCN/CrossNet_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x1/environments.txt b/ranking/ctr/DCN/CrossNet_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x1/environments.txt rename to ranking/ctr/DCN/CrossNet_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x1/results.csv b/ranking/ctr/DCN/CrossNet_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x1/results.csv rename to ranking/ctr/DCN/CrossNet_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x1/run_expid.py b/ranking/ctr/DCN/CrossNet_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x1/run_expid.py rename to ranking/ctr/DCN/CrossNet_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_001/CrossNet_avazu_x4_tuner_config_11/dataset_config.yaml b/ranking/ctr/DCN/CrossNet_avazu_x4_001/CrossNet_avazu_x4_tuner_config_11/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_001/CrossNet_avazu_x4_tuner_config_11/dataset_config.yaml rename to ranking/ctr/DCN/CrossNet_avazu_x4_001/CrossNet_avazu_x4_tuner_config_11/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_001/CrossNet_avazu_x4_tuner_config_11/model_config.yaml b/ranking/ctr/DCN/CrossNet_avazu_x4_001/CrossNet_avazu_x4_tuner_config_11/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_001/CrossNet_avazu_x4_tuner_config_11/model_config.yaml rename to ranking/ctr/DCN/CrossNet_avazu_x4_001/CrossNet_avazu_x4_tuner_config_11/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_001/DCN_avazu_x4_030_85717c33.log b/ranking/ctr/DCN/CrossNet_avazu_x4_001/DCN_avazu_x4_030_85717c33.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_001/DCN_avazu_x4_030_85717c33.log rename to ranking/ctr/DCN/CrossNet_avazu_x4_001/DCN_avazu_x4_030_85717c33.log diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_001/README.md b/ranking/ctr/DCN/CrossNet_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_001/README.md rename to ranking/ctr/DCN/CrossNet_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_001/environments.txt b/ranking/ctr/DCN/CrossNet_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_001/environments.txt rename to ranking/ctr/DCN/CrossNet_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_001/results.csv b/ranking/ctr/DCN/CrossNet_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_001/results.csv rename to ranking/ctr/DCN/CrossNet_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_001/run_expid.py b/ranking/ctr/DCN/CrossNet_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_001/run_expid.py rename to ranking/ctr/DCN/CrossNet_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_002/CrossNet_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/DCN/CrossNet_avazu_x4_002/CrossNet_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_002/CrossNet_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DCN/CrossNet_avazu_x4_002/CrossNet_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_002/CrossNet_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/DCN/CrossNet_avazu_x4_002/CrossNet_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_002/CrossNet_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/DCN/CrossNet_avazu_x4_002/CrossNet_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_002/DCN_avazu_x4_006_32b15b95.log b/ranking/ctr/DCN/CrossNet_avazu_x4_002/DCN_avazu_x4_006_32b15b95.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_002/DCN_avazu_x4_006_32b15b95.log rename to ranking/ctr/DCN/CrossNet_avazu_x4_002/DCN_avazu_x4_006_32b15b95.log diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_002/README.md b/ranking/ctr/DCN/CrossNet_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_002/README.md rename to ranking/ctr/DCN/CrossNet_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_002/environments.txt b/ranking/ctr/DCN/CrossNet_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_002/environments.txt rename to ranking/ctr/DCN/CrossNet_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_002/results.csv b/ranking/ctr/DCN/CrossNet_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_002/results.csv rename to ranking/ctr/DCN/CrossNet_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_002/run_expid.py b/ranking/ctr/DCN/CrossNet_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_avazu_x4_002/run_expid.py rename to ranking/ctr/DCN/CrossNet_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x1/CrossNet_criteo_x1_tuner_config_03.csv b/ranking/ctr/DCN/CrossNet_criteo_x1/CrossNet_criteo_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x1/CrossNet_criteo_x1_tuner_config_03.csv rename to ranking/ctr/DCN/CrossNet_criteo_x1/CrossNet_criteo_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x1/CrossNet_criteo_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/DCN/CrossNet_criteo_x1/CrossNet_criteo_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x1/CrossNet_criteo_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/DCN/CrossNet_criteo_x1/CrossNet_criteo_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x1/CrossNet_criteo_x1_tuner_config_03/model_config.yaml b/ranking/ctr/DCN/CrossNet_criteo_x1/CrossNet_criteo_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x1/CrossNet_criteo_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/DCN/CrossNet_criteo_x1/CrossNet_criteo_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x1/DCN_criteo_x1_001_95179395.log b/ranking/ctr/DCN/CrossNet_criteo_x1/DCN_criteo_x1_001_95179395.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x1/DCN_criteo_x1_001_95179395.log rename to ranking/ctr/DCN/CrossNet_criteo_x1/DCN_criteo_x1_001_95179395.log diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x1/README.md b/ranking/ctr/DCN/CrossNet_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x1/README.md rename to ranking/ctr/DCN/CrossNet_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x1/environments.txt b/ranking/ctr/DCN/CrossNet_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x1/environments.txt rename to ranking/ctr/DCN/CrossNet_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x1/results.csv b/ranking/ctr/DCN/CrossNet_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x1/results.csv rename to ranking/ctr/DCN/CrossNet_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x1/run_expid.py b/ranking/ctr/DCN/CrossNet_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x1/run_expid.py rename to ranking/ctr/DCN/CrossNet_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_001/CrossNet_criteo_x4_tuner_config_12/dataset_config.yaml b/ranking/ctr/DCN/CrossNet_criteo_x4_001/CrossNet_criteo_x4_tuner_config_12/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_001/CrossNet_criteo_x4_tuner_config_12/dataset_config.yaml rename to ranking/ctr/DCN/CrossNet_criteo_x4_001/CrossNet_criteo_x4_tuner_config_12/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_001/CrossNet_criteo_x4_tuner_config_12/model_config.yaml b/ranking/ctr/DCN/CrossNet_criteo_x4_001/CrossNet_criteo_x4_tuner_config_12/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_001/CrossNet_criteo_x4_tuner_config_12/model_config.yaml rename to ranking/ctr/DCN/CrossNet_criteo_x4_001/CrossNet_criteo_x4_tuner_config_12/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_001/DCN_criteo_x4_001_964751c8.log b/ranking/ctr/DCN/CrossNet_criteo_x4_001/DCN_criteo_x4_001_964751c8.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_001/DCN_criteo_x4_001_964751c8.log rename to ranking/ctr/DCN/CrossNet_criteo_x4_001/DCN_criteo_x4_001_964751c8.log diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_001/README.md b/ranking/ctr/DCN/CrossNet_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_001/README.md rename to ranking/ctr/DCN/CrossNet_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_001/environments.txt b/ranking/ctr/DCN/CrossNet_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_001/environments.txt rename to ranking/ctr/DCN/CrossNet_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_001/results.csv b/ranking/ctr/DCN/CrossNet_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_001/results.csv rename to ranking/ctr/DCN/CrossNet_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_001/run_expid.py b/ranking/ctr/DCN/CrossNet_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_001/run_expid.py rename to ranking/ctr/DCN/CrossNet_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_002/CrossNet_criteo_x4_tuner_config_12.yaml b/ranking/ctr/DCN/CrossNet_criteo_x4_002/CrossNet_criteo_x4_tuner_config_12.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_002/CrossNet_criteo_x4_tuner_config_12.yaml rename to ranking/ctr/DCN/CrossNet_criteo_x4_002/CrossNet_criteo_x4_tuner_config_12.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_002/CrossNet_criteo_x4_tuner_config_12/dataset_config.yaml b/ranking/ctr/DCN/CrossNet_criteo_x4_002/CrossNet_criteo_x4_tuner_config_12/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_002/CrossNet_criteo_x4_tuner_config_12/dataset_config.yaml rename to ranking/ctr/DCN/CrossNet_criteo_x4_002/CrossNet_criteo_x4_tuner_config_12/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_002/CrossNet_criteo_x4_tuner_config_12/model_config.yaml b/ranking/ctr/DCN/CrossNet_criteo_x4_002/CrossNet_criteo_x4_tuner_config_12/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_002/CrossNet_criteo_x4_tuner_config_12/model_config.yaml rename to ranking/ctr/DCN/CrossNet_criteo_x4_002/CrossNet_criteo_x4_tuner_config_12/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_002/DCN_criteo_x4_002_eeeaa5c1.log b/ranking/ctr/DCN/CrossNet_criteo_x4_002/DCN_criteo_x4_002_eeeaa5c1.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_002/DCN_criteo_x4_002_eeeaa5c1.log rename to ranking/ctr/DCN/CrossNet_criteo_x4_002/DCN_criteo_x4_002_eeeaa5c1.log diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_002/README.md b/ranking/ctr/DCN/CrossNet_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_002/README.md rename to ranking/ctr/DCN/CrossNet_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_002/environments.txt b/ranking/ctr/DCN/CrossNet_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_002/environments.txt rename to ranking/ctr/DCN/CrossNet_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_002/results.csv b/ranking/ctr/DCN/CrossNet_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_002/results.csv rename to ranking/ctr/DCN/CrossNet_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_002/run_expid.py b/ranking/ctr/DCN/CrossNet_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_criteo_x4_002/run_expid.py rename to ranking/ctr/DCN/CrossNet_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_frappe_x1/CrossNet_criteo_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/DCN/CrossNet_frappe_x1/CrossNet_criteo_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_frappe_x1/CrossNet_criteo_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DCN/CrossNet_frappe_x1/CrossNet_criteo_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_frappe_x1/CrossNet_criteo_x1_tuner_config_02/model_config.yaml b/ranking/ctr/DCN/CrossNet_frappe_x1/CrossNet_criteo_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_frappe_x1/CrossNet_criteo_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/DCN/CrossNet_frappe_x1/CrossNet_criteo_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_frappe_x1/CrossNet_frappe_x1_tuner_config_02.csv b/ranking/ctr/DCN/CrossNet_frappe_x1/CrossNet_frappe_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_frappe_x1/CrossNet_frappe_x1_tuner_config_02.csv rename to ranking/ctr/DCN/CrossNet_frappe_x1/CrossNet_frappe_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_frappe_x1/DCN_frappe_x1_005_880c69b8.log b/ranking/ctr/DCN/CrossNet_frappe_x1/DCN_frappe_x1_005_880c69b8.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_frappe_x1/DCN_frappe_x1_005_880c69b8.log rename to ranking/ctr/DCN/CrossNet_frappe_x1/DCN_frappe_x1_005_880c69b8.log diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_frappe_x1/README.md b/ranking/ctr/DCN/CrossNet_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_frappe_x1/README.md rename to ranking/ctr/DCN/CrossNet_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_frappe_x1/environments.txt b/ranking/ctr/DCN/CrossNet_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_frappe_x1/environments.txt rename to ranking/ctr/DCN/CrossNet_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_frappe_x1/results.csv b/ranking/ctr/DCN/CrossNet_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_frappe_x1/results.csv rename to ranking/ctr/DCN/CrossNet_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_frappe_x1/run_expid.py b/ranking/ctr/DCN/CrossNet_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_frappe_x1/run_expid.py rename to ranking/ctr/DCN/CrossNet_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_kkbox_x1/CrossNet_kkbox_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DCN/CrossNet_kkbox_x1/CrossNet_kkbox_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_kkbox_x1/CrossNet_kkbox_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DCN/CrossNet_kkbox_x1/CrossNet_kkbox_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_kkbox_x1/CrossNet_kkbox_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DCN/CrossNet_kkbox_x1/CrossNet_kkbox_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_kkbox_x1/CrossNet_kkbox_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DCN/CrossNet_kkbox_x1/CrossNet_kkbox_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_kkbox_x1/DCN_kkbox_x1_024_b363d48e.log b/ranking/ctr/DCN/CrossNet_kkbox_x1/DCN_kkbox_x1_024_b363d48e.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_kkbox_x1/DCN_kkbox_x1_024_b363d48e.log rename to ranking/ctr/DCN/CrossNet_kkbox_x1/DCN_kkbox_x1_024_b363d48e.log diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_kkbox_x1/README.md b/ranking/ctr/DCN/CrossNet_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_kkbox_x1/README.md rename to ranking/ctr/DCN/CrossNet_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_kkbox_x1/environments.txt b/ranking/ctr/DCN/CrossNet_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_kkbox_x1/environments.txt rename to ranking/ctr/DCN/CrossNet_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_kkbox_x1/results.csv b/ranking/ctr/DCN/CrossNet_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_kkbox_x1/results.csv rename to ranking/ctr/DCN/CrossNet_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_kkbox_x1/run_expid.py b/ranking/ctr/DCN/CrossNet_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_kkbox_x1/run_expid.py rename to ranking/ctr/DCN/CrossNet_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_movielenslatest_x1/CrossNet_movielenslatest_x1_tuner_config_01.csv b/ranking/ctr/DCN/CrossNet_movielenslatest_x1/CrossNet_movielenslatest_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_movielenslatest_x1/CrossNet_movielenslatest_x1_tuner_config_01.csv rename to ranking/ctr/DCN/CrossNet_movielenslatest_x1/CrossNet_movielenslatest_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_movielenslatest_x1/CrossNet_movielenslatest_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DCN/CrossNet_movielenslatest_x1/CrossNet_movielenslatest_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_movielenslatest_x1/CrossNet_movielenslatest_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DCN/CrossNet_movielenslatest_x1/CrossNet_movielenslatest_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_movielenslatest_x1/CrossNet_movielenslatest_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DCN/CrossNet_movielenslatest_x1/CrossNet_movielenslatest_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_movielenslatest_x1/CrossNet_movielenslatest_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DCN/CrossNet_movielenslatest_x1/CrossNet_movielenslatest_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_movielenslatest_x1/DCN_movielenslatest_x1_006_6ae4ad60.log b/ranking/ctr/DCN/CrossNet_movielenslatest_x1/DCN_movielenslatest_x1_006_6ae4ad60.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_movielenslatest_x1/DCN_movielenslatest_x1_006_6ae4ad60.log rename to ranking/ctr/DCN/CrossNet_movielenslatest_x1/DCN_movielenslatest_x1_006_6ae4ad60.log diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_movielenslatest_x1/README.md b/ranking/ctr/DCN/CrossNet_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_movielenslatest_x1/README.md rename to ranking/ctr/DCN/CrossNet_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_movielenslatest_x1/environments.txt b/ranking/ctr/DCN/CrossNet_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_movielenslatest_x1/environments.txt rename to ranking/ctr/DCN/CrossNet_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_movielenslatest_x1/results.csv b/ranking/ctr/DCN/CrossNet_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_movielenslatest_x1/results.csv rename to ranking/ctr/DCN/CrossNet_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCN/CrossNet_movielenslatest_x1/run_expid.py b/ranking/ctr/DCN/CrossNet_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/CrossNet_movielenslatest_x1/run_expid.py rename to ranking/ctr/DCN/CrossNet_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_029_366c6259.log b/ranking/ctr/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_029_366c6259.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_029_366c6259.log rename to ranking/ctr/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_029_366c6259.log diff --git a/ctr_prediction/benchmarks/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_tuner_config_04.csv b/ranking/ctr/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_tuner_config_04.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_tuner_config_04.csv rename to ranking/ctr/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_tuner_config_04.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_tuner_config_04.yaml b/ranking/ctr/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_tuner_config_04.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_tuner_config_04.yaml rename to ranking/ctr/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_tuner_config_04.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_tuner_config_04/model_config.yaml b/ranking/ctr/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/DCN/DCN_amazonelectronics_x1/DCN_amazonelectronics_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_amazonelectronics_x1/README.md b/ranking/ctr/DCN/DCN_amazonelectronics_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_amazonelectronics_x1/README.md rename to ranking/ctr/DCN/DCN_amazonelectronics_x1/README.md diff --git a/ctr_prediction/benchmarks/DCN/DCN_amazonelectronics_x1/environments.txt b/ranking/ctr/DCN/DCN_amazonelectronics_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_amazonelectronics_x1/environments.txt rename to ranking/ctr/DCN/DCN_amazonelectronics_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/DCN_amazonelectronics_x1/results.csv b/ranking/ctr/DCN/DCN_amazonelectronics_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_amazonelectronics_x1/results.csv rename to ranking/ctr/DCN/DCN_amazonelectronics_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x1/DCN_avazu_x1_004_e58d35c1.log b/ranking/ctr/DCN/DCN_avazu_x1/DCN_avazu_x1_004_e58d35c1.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x1/DCN_avazu_x1_004_e58d35c1.log rename to ranking/ctr/DCN/DCN_avazu_x1/DCN_avazu_x1_004_e58d35c1.log diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x1/DCN_avazu_x1_tuner_config_03.csv b/ranking/ctr/DCN/DCN_avazu_x1/DCN_avazu_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x1/DCN_avazu_x1_tuner_config_03.csv rename to ranking/ctr/DCN/DCN_avazu_x1/DCN_avazu_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x1/DCN_avazu_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/DCN/DCN_avazu_x1/DCN_avazu_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x1/DCN_avazu_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/DCN/DCN_avazu_x1/DCN_avazu_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x1/DCN_avazu_x1_tuner_config_03/model_config.yaml b/ranking/ctr/DCN/DCN_avazu_x1/DCN_avazu_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x1/DCN_avazu_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/DCN/DCN_avazu_x1/DCN_avazu_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x1/README.md b/ranking/ctr/DCN/DCN_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x1/README.md rename to ranking/ctr/DCN/DCN_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x1/environments.txt b/ranking/ctr/DCN/DCN_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x1/environments.txt rename to ranking/ctr/DCN/DCN_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x1/results.csv b/ranking/ctr/DCN/DCN_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x1/results.csv rename to ranking/ctr/DCN/DCN_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x1/run_expid.py b/ranking/ctr/DCN/DCN_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x1/run_expid.py rename to ranking/ctr/DCN/DCN_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x4_001/DCN_avazu_x4_018_8f445da6.log b/ranking/ctr/DCN/DCN_avazu_x4_001/DCN_avazu_x4_018_8f445da6.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x4_001/DCN_avazu_x4_018_8f445da6.log rename to ranking/ctr/DCN/DCN_avazu_x4_001/DCN_avazu_x4_018_8f445da6.log diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x4_001/DCN_avazu_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/DCN/DCN_avazu_x4_001/DCN_avazu_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x4_001/DCN_avazu_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DCN/DCN_avazu_x4_001/DCN_avazu_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x4_001/DCN_avazu_x4_tuner_config_02/model_config.yaml b/ranking/ctr/DCN/DCN_avazu_x4_001/DCN_avazu_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x4_001/DCN_avazu_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/DCN/DCN_avazu_x4_001/DCN_avazu_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x4_001/README.md b/ranking/ctr/DCN/DCN_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x4_001/README.md rename to ranking/ctr/DCN/DCN_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x4_001/environments.txt b/ranking/ctr/DCN/DCN_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x4_001/environments.txt rename to ranking/ctr/DCN/DCN_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x4_001/results.csv b/ranking/ctr/DCN/DCN_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x4_001/results.csv rename to ranking/ctr/DCN/DCN_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x4_001/run_expid.py b/ranking/ctr/DCN/DCN_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x4_001/run_expid.py rename to ranking/ctr/DCN/DCN_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x4_002/DCN_avazu_x4_008_94ed77b8.log b/ranking/ctr/DCN/DCN_avazu_x4_002/DCN_avazu_x4_008_94ed77b8.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x4_002/DCN_avazu_x4_008_94ed77b8.log rename to ranking/ctr/DCN/DCN_avazu_x4_002/DCN_avazu_x4_008_94ed77b8.log diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x4_002/DCN_avazu_x4_tuner_config_03/dataset_config.yaml b/ranking/ctr/DCN/DCN_avazu_x4_002/DCN_avazu_x4_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x4_002/DCN_avazu_x4_tuner_config_03/dataset_config.yaml rename to ranking/ctr/DCN/DCN_avazu_x4_002/DCN_avazu_x4_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x4_002/DCN_avazu_x4_tuner_config_03/model_config.yaml b/ranking/ctr/DCN/DCN_avazu_x4_002/DCN_avazu_x4_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x4_002/DCN_avazu_x4_tuner_config_03/model_config.yaml rename to ranking/ctr/DCN/DCN_avazu_x4_002/DCN_avazu_x4_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x4_002/README.md b/ranking/ctr/DCN/DCN_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x4_002/README.md rename to ranking/ctr/DCN/DCN_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x4_002/environments.txt b/ranking/ctr/DCN/DCN_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x4_002/environments.txt rename to ranking/ctr/DCN/DCN_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x4_002/results.csv b/ranking/ctr/DCN/DCN_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x4_002/results.csv rename to ranking/ctr/DCN/DCN_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_avazu_x4_002/run_expid.py b/ranking/ctr/DCN/DCN_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_avazu_x4_002/run_expid.py rename to ranking/ctr/DCN/DCN_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x1/DCN_criteo_x1_001_fa7fcfea.log b/ranking/ctr/DCN/DCN_criteo_x1/DCN_criteo_x1_001_fa7fcfea.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x1/DCN_criteo_x1_001_fa7fcfea.log rename to ranking/ctr/DCN/DCN_criteo_x1/DCN_criteo_x1_001_fa7fcfea.log diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x1/DCN_criteo_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/DCN/DCN_criteo_x1/DCN_criteo_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x1/DCN_criteo_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/DCN/DCN_criteo_x1/DCN_criteo_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x1/DCN_criteo_x1_tuner_config_03/model_config.yaml b/ranking/ctr/DCN/DCN_criteo_x1/DCN_criteo_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x1/DCN_criteo_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/DCN/DCN_criteo_x1/DCN_criteo_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x1/README.md b/ranking/ctr/DCN/DCN_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x1/README.md rename to ranking/ctr/DCN/DCN_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x1/environments.txt b/ranking/ctr/DCN/DCN_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x1/environments.txt rename to ranking/ctr/DCN/DCN_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x1/results.csv b/ranking/ctr/DCN/DCN_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x1/results.csv rename to ranking/ctr/DCN/DCN_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x1/run_expid.py b/ranking/ctr/DCN/DCN_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x1/run_expid.py rename to ranking/ctr/DCN/DCN_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x4_001/DCN_criteo_x4_012_dc8ab363.log b/ranking/ctr/DCN/DCN_criteo_x4_001/DCN_criteo_x4_012_dc8ab363.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x4_001/DCN_criteo_x4_012_dc8ab363.log rename to ranking/ctr/DCN/DCN_criteo_x4_001/DCN_criteo_x4_012_dc8ab363.log diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x4_001/DCN_criteo_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/DCN/DCN_criteo_x4_001/DCN_criteo_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x4_001/DCN_criteo_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DCN/DCN_criteo_x4_001/DCN_criteo_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x4_001/DCN_criteo_x4_tuner_config_02/model_config.yaml b/ranking/ctr/DCN/DCN_criteo_x4_001/DCN_criteo_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x4_001/DCN_criteo_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/DCN/DCN_criteo_x4_001/DCN_criteo_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x4_001/README.md b/ranking/ctr/DCN/DCN_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x4_001/README.md rename to ranking/ctr/DCN/DCN_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x4_001/environments.txt b/ranking/ctr/DCN/DCN_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x4_001/environments.txt rename to ranking/ctr/DCN/DCN_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x4_001/results.csv b/ranking/ctr/DCN/DCN_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x4_001/results.csv rename to ranking/ctr/DCN/DCN_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x4_001/run_expid.py b/ranking/ctr/DCN/DCN_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x4_001/run_expid.py rename to ranking/ctr/DCN/DCN_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x4_002/DCN_criteo_x4_001_0a86fe97.log b/ranking/ctr/DCN/DCN_criteo_x4_002/DCN_criteo_x4_001_0a86fe97.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x4_002/DCN_criteo_x4_001_0a86fe97.log rename to ranking/ctr/DCN/DCN_criteo_x4_002/DCN_criteo_x4_001_0a86fe97.log diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x4_002/DCN_criteo_x4_tuner_config_04/dataset_config.yaml b/ranking/ctr/DCN/DCN_criteo_x4_002/DCN_criteo_x4_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x4_002/DCN_criteo_x4_tuner_config_04/dataset_config.yaml rename to ranking/ctr/DCN/DCN_criteo_x4_002/DCN_criteo_x4_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x4_002/DCN_criteo_x4_tuner_config_04/model_config.yaml b/ranking/ctr/DCN/DCN_criteo_x4_002/DCN_criteo_x4_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x4_002/DCN_criteo_x4_tuner_config_04/model_config.yaml rename to ranking/ctr/DCN/DCN_criteo_x4_002/DCN_criteo_x4_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x4_002/README.md b/ranking/ctr/DCN/DCN_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x4_002/README.md rename to ranking/ctr/DCN/DCN_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x4_002/environments.txt b/ranking/ctr/DCN/DCN_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x4_002/environments.txt rename to ranking/ctr/DCN/DCN_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x4_002/results.csv b/ranking/ctr/DCN/DCN_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x4_002/results.csv rename to ranking/ctr/DCN/DCN_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_criteo_x4_002/run_expid.py b/ranking/ctr/DCN/DCN_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_criteo_x4_002/run_expid.py rename to ranking/ctr/DCN/DCN_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/DCN_frappe_x1/DCN_frappe_x1_013_efa58c31.log b/ranking/ctr/DCN/DCN_frappe_x1/DCN_frappe_x1_013_efa58c31.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_frappe_x1/DCN_frappe_x1_013_efa58c31.log rename to ranking/ctr/DCN/DCN_frappe_x1/DCN_frappe_x1_013_efa58c31.log diff --git a/ctr_prediction/benchmarks/DCN/DCN_frappe_x1/DCN_frappe_x1_tuner_config_02.csv b/ranking/ctr/DCN/DCN_frappe_x1/DCN_frappe_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_frappe_x1/DCN_frappe_x1_tuner_config_02.csv rename to ranking/ctr/DCN/DCN_frappe_x1/DCN_frappe_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_frappe_x1/DCN_frappe_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/DCN/DCN_frappe_x1/DCN_frappe_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_frappe_x1/DCN_frappe_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DCN/DCN_frappe_x1/DCN_frappe_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_frappe_x1/DCN_frappe_x1_tuner_config_02/model_config.yaml b/ranking/ctr/DCN/DCN_frappe_x1/DCN_frappe_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_frappe_x1/DCN_frappe_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/DCN/DCN_frappe_x1/DCN_frappe_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_frappe_x1/README.md b/ranking/ctr/DCN/DCN_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_frappe_x1/README.md rename to ranking/ctr/DCN/DCN_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/DCN/DCN_frappe_x1/environments.txt b/ranking/ctr/DCN/DCN_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_frappe_x1/environments.txt rename to ranking/ctr/DCN/DCN_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/DCN_frappe_x1/results.csv b/ranking/ctr/DCN/DCN_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_frappe_x1/results.csv rename to ranking/ctr/DCN/DCN_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_frappe_x1/run_expid.py b/ranking/ctr/DCN/DCN_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_frappe_x1/run_expid.py rename to ranking/ctr/DCN/DCN_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/DCN_kkbox_x1/DCN_kkbox_x1_005_362e6c13.log b/ranking/ctr/DCN/DCN_kkbox_x1/DCN_kkbox_x1_005_362e6c13.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_kkbox_x1/DCN_kkbox_x1_005_362e6c13.log rename to ranking/ctr/DCN/DCN_kkbox_x1/DCN_kkbox_x1_005_362e6c13.log diff --git a/ctr_prediction/benchmarks/DCN/DCN_kkbox_x1/DCN_kkbox_x1_tuner_config_05/dataset_config.yaml b/ranking/ctr/DCN/DCN_kkbox_x1/DCN_kkbox_x1_tuner_config_05/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_kkbox_x1/DCN_kkbox_x1_tuner_config_05/dataset_config.yaml rename to ranking/ctr/DCN/DCN_kkbox_x1/DCN_kkbox_x1_tuner_config_05/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_kkbox_x1/DCN_kkbox_x1_tuner_config_05/model_config.yaml b/ranking/ctr/DCN/DCN_kkbox_x1/DCN_kkbox_x1_tuner_config_05/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_kkbox_x1/DCN_kkbox_x1_tuner_config_05/model_config.yaml rename to ranking/ctr/DCN/DCN_kkbox_x1/DCN_kkbox_x1_tuner_config_05/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_kkbox_x1/README.md b/ranking/ctr/DCN/DCN_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_kkbox_x1/README.md rename to ranking/ctr/DCN/DCN_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/DCN/DCN_kkbox_x1/environments.txt b/ranking/ctr/DCN/DCN_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_kkbox_x1/environments.txt rename to ranking/ctr/DCN/DCN_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/DCN_kkbox_x1/results.csv b/ranking/ctr/DCN/DCN_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_kkbox_x1/results.csv rename to ranking/ctr/DCN/DCN_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_kkbox_x1/run_expid.py b/ranking/ctr/DCN/DCN_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_kkbox_x1/run_expid.py rename to ranking/ctr/DCN/DCN_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_004_4191aa76.log b/ranking/ctr/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_004_4191aa76.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_004_4191aa76.log rename to ranking/ctr/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_004_4191aa76.log diff --git a/ctr_prediction/benchmarks/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_tuner_config_06.csv b/ranking/ctr/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_tuner_config_06.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_tuner_config_06.csv rename to ranking/ctr/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_tuner_config_06.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_tuner_config_06.yaml b/ranking/ctr/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_tuner_config_06.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_tuner_config_06.yaml rename to ranking/ctr/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_tuner_config_06.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_tuner_config_06/dataset_config.yaml b/ranking/ctr/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_tuner_config_06/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_tuner_config_06/dataset_config.yaml rename to ranking/ctr/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_tuner_config_06/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_tuner_config_06/model_config.yaml b/ranking/ctr/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_tuner_config_06/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_tuner_config_06/model_config.yaml rename to ranking/ctr/DCN/DCN_kuaivideo_x1/DCN_kuaivideo_x1_tuner_config_06/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_kuaivideo_x1/README.md b/ranking/ctr/DCN/DCN_kuaivideo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_kuaivideo_x1/README.md rename to ranking/ctr/DCN/DCN_kuaivideo_x1/README.md diff --git a/ctr_prediction/benchmarks/DCN/DCN_kuaivideo_x1/environments.txt b/ranking/ctr/DCN/DCN_kuaivideo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_kuaivideo_x1/environments.txt rename to ranking/ctr/DCN/DCN_kuaivideo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/DCN_kuaivideo_x1/results.csv b/ranking/ctr/DCN/DCN_kuaivideo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_kuaivideo_x1/results.csv rename to ranking/ctr/DCN/DCN_kuaivideo_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_010_a992df63.log b/ranking/ctr/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_010_a992df63.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_010_a992df63.log rename to ranking/ctr/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_010_a992df63.log diff --git a/ctr_prediction/benchmarks/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_tuner_config_07.csv b/ranking/ctr/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_tuner_config_07.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_tuner_config_07.csv rename to ranking/ctr/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_tuner_config_07.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_tuner_config_07.yaml b/ranking/ctr/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_tuner_config_07.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_tuner_config_07.yaml rename to ranking/ctr/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_tuner_config_07.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_tuner_config_07/dataset_config.yaml b/ranking/ctr/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_tuner_config_07/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_tuner_config_07/dataset_config.yaml rename to ranking/ctr/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_tuner_config_07/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_tuner_config_07/model_config.yaml b/ranking/ctr/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_tuner_config_07/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_tuner_config_07/model_config.yaml rename to ranking/ctr/DCN/DCN_microvideo1.7m_x1/DCN_microvideo1.7m_x1_tuner_config_07/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_microvideo1.7m_x1/README.md b/ranking/ctr/DCN/DCN_microvideo1.7m_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_microvideo1.7m_x1/README.md rename to ranking/ctr/DCN/DCN_microvideo1.7m_x1/README.md diff --git a/ctr_prediction/benchmarks/DCN/DCN_microvideo1.7m_x1/environments.txt b/ranking/ctr/DCN/DCN_microvideo1.7m_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_microvideo1.7m_x1/environments.txt rename to ranking/ctr/DCN/DCN_microvideo1.7m_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/DCN_microvideo1.7m_x1/results.csv b/ranking/ctr/DCN/DCN_microvideo1.7m_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_microvideo1.7m_x1/results.csv rename to ranking/ctr/DCN/DCN_microvideo1.7m_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_movielenslatest_x1/DCN_movielenslatest_x1_017_4810b636.log b/ranking/ctr/DCN/DCN_movielenslatest_x1/DCN_movielenslatest_x1_017_4810b636.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_movielenslatest_x1/DCN_movielenslatest_x1_017_4810b636.log rename to ranking/ctr/DCN/DCN_movielenslatest_x1/DCN_movielenslatest_x1_017_4810b636.log diff --git a/ctr_prediction/benchmarks/DCN/DCN_movielenslatest_x1/DCN_movielenslatest_x1_tuner_config_01.csv b/ranking/ctr/DCN/DCN_movielenslatest_x1/DCN_movielenslatest_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_movielenslatest_x1/DCN_movielenslatest_x1_tuner_config_01.csv rename to ranking/ctr/DCN/DCN_movielenslatest_x1/DCN_movielenslatest_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_movielenslatest_x1/DCN_movielenslatest_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DCN/DCN_movielenslatest_x1/DCN_movielenslatest_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_movielenslatest_x1/DCN_movielenslatest_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DCN/DCN_movielenslatest_x1/DCN_movielenslatest_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_movielenslatest_x1/DCN_movielenslatest_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DCN/DCN_movielenslatest_x1/DCN_movielenslatest_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_movielenslatest_x1/DCN_movielenslatest_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DCN/DCN_movielenslatest_x1/DCN_movielenslatest_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_movielenslatest_x1/README.md b/ranking/ctr/DCN/DCN_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_movielenslatest_x1/README.md rename to ranking/ctr/DCN/DCN_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/DCN/DCN_movielenslatest_x1/environments.txt b/ranking/ctr/DCN/DCN_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_movielenslatest_x1/environments.txt rename to ranking/ctr/DCN/DCN_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/DCN_movielenslatest_x1/results.csv b/ranking/ctr/DCN/DCN_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_movielenslatest_x1/results.csv rename to ranking/ctr/DCN/DCN_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_movielenslatest_x1/run_expid.py b/ranking/ctr/DCN/DCN_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_movielenslatest_x1/run_expid.py rename to ranking/ctr/DCN/DCN_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_048_95110842.log b/ranking/ctr/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_048_95110842.log similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_048_95110842.log rename to ranking/ctr/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_048_95110842.log diff --git a/ctr_prediction/benchmarks/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_tuner_config_07.csv b/ranking/ctr/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_tuner_config_07.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_tuner_config_07.csv rename to ranking/ctr/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_tuner_config_07.csv diff --git a/ctr_prediction/benchmarks/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_tuner_config_07.yaml b/ranking/ctr/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_tuner_config_07.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_tuner_config_07.yaml rename to ranking/ctr/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_tuner_config_07.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_tuner_config_07/dataset_config.yaml b/ranking/ctr/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_tuner_config_07/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_tuner_config_07/dataset_config.yaml rename to ranking/ctr/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_tuner_config_07/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_tuner_config_07/model_config.yaml b/ranking/ctr/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_tuner_config_07/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_tuner_config_07/model_config.yaml rename to ranking/ctr/DCN/DCN_taobaoad_x1/DCN_taobaoad_x1_tuner_config_07/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCN/DCN_taobaoad_x1/README.md b/ranking/ctr/DCN/DCN_taobaoad_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_taobaoad_x1/README.md rename to ranking/ctr/DCN/DCN_taobaoad_x1/README.md diff --git a/ctr_prediction/benchmarks/DCN/DCN_taobaoad_x1/environments.txt b/ranking/ctr/DCN/DCN_taobaoad_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_taobaoad_x1/environments.txt rename to ranking/ctr/DCN/DCN_taobaoad_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCN/DCN_taobaoad_x1/results.csv b/ranking/ctr/DCN/DCN_taobaoad_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCN/DCN_taobaoad_x1/results.csv rename to ranking/ctr/DCN/DCN_taobaoad_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_avazu_x1/CrossNetv2_avazu_x1_tuner_config_01.csv b/ranking/ctr/DCNv2/CrossNetv2_avazu_x1/CrossNetv2_avazu_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_avazu_x1/CrossNetv2_avazu_x1_tuner_config_01.csv rename to ranking/ctr/DCNv2/CrossNetv2_avazu_x1/CrossNetv2_avazu_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_avazu_x1/CrossNetv2_avazu_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DCNv2/CrossNetv2_avazu_x1/CrossNetv2_avazu_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_avazu_x1/CrossNetv2_avazu_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DCNv2/CrossNetv2_avazu_x1/CrossNetv2_avazu_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_avazu_x1/CrossNetv2_avazu_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DCNv2/CrossNetv2_avazu_x1/CrossNetv2_avazu_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_avazu_x1/CrossNetv2_avazu_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DCNv2/CrossNetv2_avazu_x1/CrossNetv2_avazu_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_avazu_x1/DCNv2_avazu_x1_011_e4cbb525.log b/ranking/ctr/DCNv2/CrossNetv2_avazu_x1/DCNv2_avazu_x1_011_e4cbb525.log similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_avazu_x1/DCNv2_avazu_x1_011_e4cbb525.log rename to ranking/ctr/DCNv2/CrossNetv2_avazu_x1/DCNv2_avazu_x1_011_e4cbb525.log diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_avazu_x1/README.md b/ranking/ctr/DCNv2/CrossNetv2_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_avazu_x1/README.md rename to ranking/ctr/DCNv2/CrossNetv2_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_avazu_x1/environments.txt b/ranking/ctr/DCNv2/CrossNetv2_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_avazu_x1/environments.txt rename to ranking/ctr/DCNv2/CrossNetv2_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_avazu_x1/results.csv b/ranking/ctr/DCNv2/CrossNetv2_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_avazu_x1/results.csv rename to ranking/ctr/DCNv2/CrossNetv2_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_avazu_x1/run_expid.py b/ranking/ctr/DCNv2/CrossNetv2_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_avazu_x1/run_expid.py rename to ranking/ctr/DCNv2/CrossNetv2_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_criteo_x1/CrossNetv2_criteo_x1_tuner_config_01.csv b/ranking/ctr/DCNv2/CrossNetv2_criteo_x1/CrossNetv2_criteo_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_criteo_x1/CrossNetv2_criteo_x1_tuner_config_01.csv rename to ranking/ctr/DCNv2/CrossNetv2_criteo_x1/CrossNetv2_criteo_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_criteo_x1/CrossNetv2_criteo_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DCNv2/CrossNetv2_criteo_x1/CrossNetv2_criteo_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_criteo_x1/CrossNetv2_criteo_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DCNv2/CrossNetv2_criteo_x1/CrossNetv2_criteo_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_criteo_x1/CrossNetv2_criteo_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DCNv2/CrossNetv2_criteo_x1/CrossNetv2_criteo_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_criteo_x1/CrossNetv2_criteo_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DCNv2/CrossNetv2_criteo_x1/CrossNetv2_criteo_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_criteo_x1/DCNv2_criteo_x1_008_43e17586.log b/ranking/ctr/DCNv2/CrossNetv2_criteo_x1/DCNv2_criteo_x1_008_43e17586.log similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_criteo_x1/DCNv2_criteo_x1_008_43e17586.log rename to ranking/ctr/DCNv2/CrossNetv2_criteo_x1/DCNv2_criteo_x1_008_43e17586.log diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_criteo_x1/README.md b/ranking/ctr/DCNv2/CrossNetv2_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_criteo_x1/README.md rename to ranking/ctr/DCNv2/CrossNetv2_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_criteo_x1/environments.txt b/ranking/ctr/DCNv2/CrossNetv2_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_criteo_x1/environments.txt rename to ranking/ctr/DCNv2/CrossNetv2_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_criteo_x1/results.csv b/ranking/ctr/DCNv2/CrossNetv2_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_criteo_x1/results.csv rename to ranking/ctr/DCNv2/CrossNetv2_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_criteo_x1/run_expid.py b/ranking/ctr/DCNv2/CrossNetv2_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_criteo_x1/run_expid.py rename to ranking/ctr/DCNv2/CrossNetv2_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_frappe_x1/CrossNetv2_frappe_x1_tuner_config_01.csv b/ranking/ctr/DCNv2/CrossNetv2_frappe_x1/CrossNetv2_frappe_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_frappe_x1/CrossNetv2_frappe_x1_tuner_config_01.csv rename to ranking/ctr/DCNv2/CrossNetv2_frappe_x1/CrossNetv2_frappe_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_frappe_x1/CrossNetv2_frappe_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DCNv2/CrossNetv2_frappe_x1/CrossNetv2_frappe_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_frappe_x1/CrossNetv2_frappe_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DCNv2/CrossNetv2_frappe_x1/CrossNetv2_frappe_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_frappe_x1/CrossNetv2_frappe_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DCNv2/CrossNetv2_frappe_x1/CrossNetv2_frappe_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_frappe_x1/CrossNetv2_frappe_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DCNv2/CrossNetv2_frappe_x1/CrossNetv2_frappe_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_frappe_x1/DCNv2_frappe_x1_001_881fe47a.log b/ranking/ctr/DCNv2/CrossNetv2_frappe_x1/DCNv2_frappe_x1_001_881fe47a.log similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_frappe_x1/DCNv2_frappe_x1_001_881fe47a.log rename to ranking/ctr/DCNv2/CrossNetv2_frappe_x1/DCNv2_frappe_x1_001_881fe47a.log diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_frappe_x1/README.md b/ranking/ctr/DCNv2/CrossNetv2_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_frappe_x1/README.md rename to ranking/ctr/DCNv2/CrossNetv2_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_frappe_x1/environments.txt b/ranking/ctr/DCNv2/CrossNetv2_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_frappe_x1/environments.txt rename to ranking/ctr/DCNv2/CrossNetv2_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_frappe_x1/results.csv b/ranking/ctr/DCNv2/CrossNetv2_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_frappe_x1/results.csv rename to ranking/ctr/DCNv2/CrossNetv2_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_frappe_x1/run_expid.py b/ranking/ctr/DCNv2/CrossNetv2_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_frappe_x1/run_expid.py rename to ranking/ctr/DCNv2/CrossNetv2_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_movielenslatest_x1/CrossNetv2_movielenslatest_x1_tuner_config_01.csv b/ranking/ctr/DCNv2/CrossNetv2_movielenslatest_x1/CrossNetv2_movielenslatest_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_movielenslatest_x1/CrossNetv2_movielenslatest_x1_tuner_config_01.csv rename to ranking/ctr/DCNv2/CrossNetv2_movielenslatest_x1/CrossNetv2_movielenslatest_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_movielenslatest_x1/CrossNetv2_movielenslatest_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DCNv2/CrossNetv2_movielenslatest_x1/CrossNetv2_movielenslatest_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_movielenslatest_x1/CrossNetv2_movielenslatest_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DCNv2/CrossNetv2_movielenslatest_x1/CrossNetv2_movielenslatest_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_movielenslatest_x1/CrossNetv2_movielenslatest_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DCNv2/CrossNetv2_movielenslatest_x1/CrossNetv2_movielenslatest_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_movielenslatest_x1/CrossNetv2_movielenslatest_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DCNv2/CrossNetv2_movielenslatest_x1/CrossNetv2_movielenslatest_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_movielenslatest_x1/DCNv2_movielenslatest_x1_005_a53d8bd5.log b/ranking/ctr/DCNv2/CrossNetv2_movielenslatest_x1/DCNv2_movielenslatest_x1_005_a53d8bd5.log similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_movielenslatest_x1/DCNv2_movielenslatest_x1_005_a53d8bd5.log rename to ranking/ctr/DCNv2/CrossNetv2_movielenslatest_x1/DCNv2_movielenslatest_x1_005_a53d8bd5.log diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_movielenslatest_x1/README.md b/ranking/ctr/DCNv2/CrossNetv2_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_movielenslatest_x1/README.md rename to ranking/ctr/DCNv2/CrossNetv2_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_movielenslatest_x1/environments.txt b/ranking/ctr/DCNv2/CrossNetv2_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_movielenslatest_x1/environments.txt rename to ranking/ctr/DCNv2/CrossNetv2_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_movielenslatest_x1/results.csv b/ranking/ctr/DCNv2/CrossNetv2_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_movielenslatest_x1/results.csv rename to ranking/ctr/DCNv2/CrossNetv2_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCNv2/CrossNetv2_movielenslatest_x1/run_expid.py b/ranking/ctr/DCNv2/CrossNetv2_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/CrossNetv2_movielenslatest_x1/run_expid.py rename to ranking/ctr/DCNv2/CrossNetv2_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_010_7212260c.log b/ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_010_7212260c.log similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_010_7212260c.log rename to ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_010_7212260c.log diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04.csv b/ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04.csv rename to ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04.csv diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04.yaml b/ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04.yaml similarity index 96% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04.yaml rename to ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04.yaml index 7c0d3bb1..a72e3fe8 100644 --- a/ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04.yaml +++ b/ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04.yaml @@ -33,7 +33,7 @@ tuner_space: learning_rate: 5.e-4 batch_size: 1024 seed: 2022 - group_index: user_id + group_id: user_id metrics: [[gAUC, AUC, logloss]] monitor: {"gAUC": 1, "AUC": 1} diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04/model_config.yaml b/ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04/model_config.yaml similarity index 97% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04/model_config.yaml index 6b565aab..62783a9a 100644 --- a/ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04/model_config.yaml +++ b/ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/DCNv2_amazonelectronics_x1_tuner_config_04/model_config.yaml @@ -9,7 +9,7 @@ DCNv2_amazonelectronics_x1_014_3d86d84a: embedding_regularizer: 0.005 epochs: 100 eval_interval: 1 - group_index: user_id + group_id: user_id learning_rate: 0.0005 loss: binary_crossentropy low_rank: 32 @@ -46,7 +46,7 @@ DCNv2_amazonelectronics_x1_026_3f36bb39: embedding_regularizer: 0.01 epochs: 100 eval_interval: 1 - group_index: user_id + group_id: user_id learning_rate: 0.0005 loss: binary_crossentropy low_rank: 32 diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/README.md b/ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/README.md rename to ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/README.md diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/environments.txt b/ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/environments.txt rename to ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/results.csv b/ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_amazonelectronics_x1/results.csv rename to ranking/ctr/DCNv2/DCNv2_amazonelectronics_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_avazu_x1/DCNv2_avazu_x1_029_81904473.log b/ranking/ctr/DCNv2/DCNv2_avazu_x1/DCNv2_avazu_x1_029_81904473.log similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_avazu_x1/DCNv2_avazu_x1_029_81904473.log rename to ranking/ctr/DCNv2/DCNv2_avazu_x1/DCNv2_avazu_x1_029_81904473.log diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_avazu_x1/DCNv2_avazu_x1_tuner_config_05.csv b/ranking/ctr/DCNv2/DCNv2_avazu_x1/DCNv2_avazu_x1_tuner_config_05.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_avazu_x1/DCNv2_avazu_x1_tuner_config_05.csv rename to ranking/ctr/DCNv2/DCNv2_avazu_x1/DCNv2_avazu_x1_tuner_config_05.csv diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_avazu_x1/DCNv2_avazu_x1_tuner_config_05/dataset_config.yaml b/ranking/ctr/DCNv2/DCNv2_avazu_x1/DCNv2_avazu_x1_tuner_config_05/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_avazu_x1/DCNv2_avazu_x1_tuner_config_05/dataset_config.yaml rename to ranking/ctr/DCNv2/DCNv2_avazu_x1/DCNv2_avazu_x1_tuner_config_05/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_avazu_x1/DCNv2_avazu_x1_tuner_config_05/model_config.yaml b/ranking/ctr/DCNv2/DCNv2_avazu_x1/DCNv2_avazu_x1_tuner_config_05/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_avazu_x1/DCNv2_avazu_x1_tuner_config_05/model_config.yaml rename to ranking/ctr/DCNv2/DCNv2_avazu_x1/DCNv2_avazu_x1_tuner_config_05/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_avazu_x1/README.md b/ranking/ctr/DCNv2/DCNv2_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_avazu_x1/README.md rename to ranking/ctr/DCNv2/DCNv2_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_avazu_x1/environments.txt b/ranking/ctr/DCNv2/DCNv2_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_avazu_x1/environments.txt rename to ranking/ctr/DCNv2/DCNv2_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_avazu_x1/results.csv b/ranking/ctr/DCNv2/DCNv2_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_avazu_x1/results.csv rename to ranking/ctr/DCNv2/DCNv2_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_avazu_x1/run_expid.py b/ranking/ctr/DCNv2/DCNv2_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_avazu_x1/run_expid.py rename to ranking/ctr/DCNv2/DCNv2_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_criteo_x1/DCNv2_criteo_x1_001_28c60688.log b/ranking/ctr/DCNv2/DCNv2_criteo_x1/DCNv2_criteo_x1_001_28c60688.log similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_criteo_x1/DCNv2_criteo_x1_001_28c60688.log rename to ranking/ctr/DCNv2/DCNv2_criteo_x1/DCNv2_criteo_x1_001_28c60688.log diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_criteo_x1/DCNv2_criteo_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/DCNv2/DCNv2_criteo_x1/DCNv2_criteo_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_criteo_x1/DCNv2_criteo_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/DCNv2/DCNv2_criteo_x1/DCNv2_criteo_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_criteo_x1/DCNv2_criteo_x1_tuner_config_04/model_config.yaml b/ranking/ctr/DCNv2/DCNv2_criteo_x1/DCNv2_criteo_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_criteo_x1/DCNv2_criteo_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/DCNv2/DCNv2_criteo_x1/DCNv2_criteo_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_criteo_x1/README.md b/ranking/ctr/DCNv2/DCNv2_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_criteo_x1/README.md rename to ranking/ctr/DCNv2/DCNv2_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_criteo_x1/environments.txt b/ranking/ctr/DCNv2/DCNv2_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_criteo_x1/environments.txt rename to ranking/ctr/DCNv2/DCNv2_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_criteo_x1/results.csv b/ranking/ctr/DCNv2/DCNv2_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_criteo_x1/results.csv rename to ranking/ctr/DCNv2/DCNv2_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_criteo_x1/run_expid.py b/ranking/ctr/DCNv2/DCNv2_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_criteo_x1/run_expid.py rename to ranking/ctr/DCNv2/DCNv2_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_frappe_x1/DCNv2_frappe_x1_007_c207b717.log b/ranking/ctr/DCNv2/DCNv2_frappe_x1/DCNv2_frappe_x1_007_c207b717.log similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_frappe_x1/DCNv2_frappe_x1_007_c207b717.log rename to ranking/ctr/DCNv2/DCNv2_frappe_x1/DCNv2_frappe_x1_007_c207b717.log diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_frappe_x1/DCNv2_frappe_x1_tuner_config_03.csv b/ranking/ctr/DCNv2/DCNv2_frappe_x1/DCNv2_frappe_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_frappe_x1/DCNv2_frappe_x1_tuner_config_03.csv rename to ranking/ctr/DCNv2/DCNv2_frappe_x1/DCNv2_frappe_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_frappe_x1/DCNv2_frappe_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/DCNv2/DCNv2_frappe_x1/DCNv2_frappe_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_frappe_x1/DCNv2_frappe_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/DCNv2/DCNv2_frappe_x1/DCNv2_frappe_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_frappe_x1/DCNv2_frappe_x1_tuner_config_03/model_config.yaml b/ranking/ctr/DCNv2/DCNv2_frappe_x1/DCNv2_frappe_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_frappe_x1/DCNv2_frappe_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/DCNv2/DCNv2_frappe_x1/DCNv2_frappe_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_frappe_x1/README.md b/ranking/ctr/DCNv2/DCNv2_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_frappe_x1/README.md rename to ranking/ctr/DCNv2/DCNv2_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_frappe_x1/environments.txt b/ranking/ctr/DCNv2/DCNv2_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_frappe_x1/environments.txt rename to ranking/ctr/DCNv2/DCNv2_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_frappe_x1/results.csv b/ranking/ctr/DCNv2/DCNv2_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_frappe_x1/results.csv rename to ranking/ctr/DCNv2/DCNv2_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_frappe_x1/run_expid.py b/ranking/ctr/DCNv2/DCNv2_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_frappe_x1/run_expid.py rename to ranking/ctr/DCNv2/DCNv2_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_008_7047bff3.log b/ranking/ctr/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_008_7047bff3.log similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_008_7047bff3.log rename to ranking/ctr/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_008_7047bff3.log diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_tuner_config_01.csv b/ranking/ctr/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_tuner_config_01.csv rename to ranking/ctr/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_tuner_config_01.yaml b/ranking/ctr/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_tuner_config_01.yaml rename to ranking/ctr/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DCNv2/DCNv2_kuaivideo_x1/DCNv2_kuaivideo_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_kuaivideo_x1/README.md b/ranking/ctr/DCNv2/DCNv2_kuaivideo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_kuaivideo_x1/README.md rename to ranking/ctr/DCNv2/DCNv2_kuaivideo_x1/README.md diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_kuaivideo_x1/environments.txt b/ranking/ctr/DCNv2/DCNv2_kuaivideo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_kuaivideo_x1/environments.txt rename to ranking/ctr/DCNv2/DCNv2_kuaivideo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_kuaivideo_x1/results.csv b/ranking/ctr/DCNv2/DCNv2_kuaivideo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_kuaivideo_x1/results.csv rename to ranking/ctr/DCNv2/DCNv2_kuaivideo_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_017_9199218b.log b/ranking/ctr/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_017_9199218b.log similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_017_9199218b.log rename to ranking/ctr/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_017_9199218b.log diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_tuner_config_05.csv b/ranking/ctr/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_tuner_config_05.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_tuner_config_05.csv rename to ranking/ctr/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_tuner_config_05.csv diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_tuner_config_05.yaml b/ranking/ctr/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_tuner_config_05.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_tuner_config_05.yaml rename to ranking/ctr/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_tuner_config_05.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_tuner_config_05/dataset_config.yaml b/ranking/ctr/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_tuner_config_05/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_tuner_config_05/dataset_config.yaml rename to ranking/ctr/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_tuner_config_05/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_tuner_config_05/model_config.yaml b/ranking/ctr/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_tuner_config_05/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_tuner_config_05/model_config.yaml rename to ranking/ctr/DCNv2/DCNv2_microvideo1.7m_x1/DCNv2_microvideo1.7m_x1_tuner_config_05/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_microvideo1.7m_x1/README.md b/ranking/ctr/DCNv2/DCNv2_microvideo1.7m_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_microvideo1.7m_x1/README.md rename to ranking/ctr/DCNv2/DCNv2_microvideo1.7m_x1/README.md diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_microvideo1.7m_x1/environments.txt b/ranking/ctr/DCNv2/DCNv2_microvideo1.7m_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_microvideo1.7m_x1/environments.txt rename to ranking/ctr/DCNv2/DCNv2_microvideo1.7m_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_microvideo1.7m_x1/results.csv b/ranking/ctr/DCNv2/DCNv2_microvideo1.7m_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_microvideo1.7m_x1/results.csv rename to ranking/ctr/DCNv2/DCNv2_microvideo1.7m_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_movielenslatest_x1/DCNv2_movielenslatest_x1_016_98ea1c72.log b/ranking/ctr/DCNv2/DCNv2_movielenslatest_x1/DCNv2_movielenslatest_x1_016_98ea1c72.log similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_movielenslatest_x1/DCNv2_movielenslatest_x1_016_98ea1c72.log rename to ranking/ctr/DCNv2/DCNv2_movielenslatest_x1/DCNv2_movielenslatest_x1_016_98ea1c72.log diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_movielenslatest_x1/DCNv2_movielenslatest_x1_tuner_config_01.csv b/ranking/ctr/DCNv2/DCNv2_movielenslatest_x1/DCNv2_movielenslatest_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_movielenslatest_x1/DCNv2_movielenslatest_x1_tuner_config_01.csv rename to ranking/ctr/DCNv2/DCNv2_movielenslatest_x1/DCNv2_movielenslatest_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_movielenslatest_x1/DCNv2_movielenslatest_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DCNv2/DCNv2_movielenslatest_x1/DCNv2_movielenslatest_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_movielenslatest_x1/DCNv2_movielenslatest_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DCNv2/DCNv2_movielenslatest_x1/DCNv2_movielenslatest_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_movielenslatest_x1/DCNv2_movielenslatest_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DCNv2/DCNv2_movielenslatest_x1/DCNv2_movielenslatest_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_movielenslatest_x1/DCNv2_movielenslatest_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DCNv2/DCNv2_movielenslatest_x1/DCNv2_movielenslatest_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_movielenslatest_x1/README.md b/ranking/ctr/DCNv2/DCNv2_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_movielenslatest_x1/README.md rename to ranking/ctr/DCNv2/DCNv2_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_movielenslatest_x1/environments.txt b/ranking/ctr/DCNv2/DCNv2_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_movielenslatest_x1/environments.txt rename to ranking/ctr/DCNv2/DCNv2_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_movielenslatest_x1/results.csv b/ranking/ctr/DCNv2/DCNv2_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_movielenslatest_x1/results.csv rename to ranking/ctr/DCNv2/DCNv2_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_movielenslatest_x1/run_expid.py b/ranking/ctr/DCNv2/DCNv2_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_movielenslatest_x1/run_expid.py rename to ranking/ctr/DCNv2/DCNv2_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_026_55d3948a.log b/ranking/ctr/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_026_55d3948a.log similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_026_55d3948a.log rename to ranking/ctr/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_026_55d3948a.log diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_tuner_config_01.csv b/ranking/ctr/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_tuner_config_01.csv rename to ranking/ctr/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_tuner_config_01.yaml b/ranking/ctr/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_tuner_config_01.yaml rename to ranking/ctr/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DCNv2/DCNv2_taobaoad_x1/DCNv2_taobaoad_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_taobaoad_x1/README.md b/ranking/ctr/DCNv2/DCNv2_taobaoad_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_taobaoad_x1/README.md rename to ranking/ctr/DCNv2/DCNv2_taobaoad_x1/README.md diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_taobaoad_x1/environments.txt b/ranking/ctr/DCNv2/DCNv2_taobaoad_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_taobaoad_x1/environments.txt rename to ranking/ctr/DCNv2/DCNv2_taobaoad_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DCNv2/DCNv2_taobaoad_x1/results.csv b/ranking/ctr/DCNv2/DCNv2_taobaoad_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DCNv2/DCNv2_taobaoad_x1/results.csv rename to ranking/ctr/DCNv2/DCNv2_taobaoad_x1/results.csv diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/DESTINE_avazu_x1_016_61d63533.log b/ranking/ctr/DESTINE/DESTINE_avazu_x1/DESTINE_avazu_x1_016_61d63533.log similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/DESTINE_avazu_x1_016_61d63533.log rename to ranking/ctr/DESTINE/DESTINE_avazu_x1/DESTINE_avazu_x1_016_61d63533.log diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/DESTINE_avazu_x1_tuner_config_02.csv b/ranking/ctr/DESTINE/DESTINE_avazu_x1/DESTINE_avazu_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/DESTINE_avazu_x1_tuner_config_02.csv rename to ranking/ctr/DESTINE/DESTINE_avazu_x1/DESTINE_avazu_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/DESTINE_avazu_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/DESTINE/DESTINE_avazu_x1/DESTINE_avazu_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/DESTINE_avazu_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DESTINE/DESTINE_avazu_x1/DESTINE_avazu_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/DESTINE_avazu_x1_tuner_config_02/model_config.yaml b/ranking/ctr/DESTINE/DESTINE_avazu_x1/DESTINE_avazu_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/DESTINE_avazu_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/DESTINE/DESTINE_avazu_x1/DESTINE_avazu_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/README.md b/ranking/ctr/DESTINE/DESTINE_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/README.md rename to ranking/ctr/DESTINE/DESTINE_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/environments.txt b/ranking/ctr/DESTINE/DESTINE_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/environments.txt rename to ranking/ctr/DESTINE/DESTINE_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/fuxictr_version.py b/ranking/ctr/DESTINE/DESTINE_avazu_x1/fuxictr_version.py similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/fuxictr_version.py rename to ranking/ctr/DESTINE/DESTINE_avazu_x1/fuxictr_version.py diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/results.csv b/ranking/ctr/DESTINE/DESTINE_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/results.csv rename to ranking/ctr/DESTINE/DESTINE_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/run_expid.py b/ranking/ctr/DESTINE/DESTINE_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_avazu_x1/run_expid.py rename to ranking/ctr/DESTINE/DESTINE_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/DESTINE_criteo_x1_001_767ae9b5.log b/ranking/ctr/DESTINE/DESTINE_criteo_x1/DESTINE_criteo_x1_001_767ae9b5.log similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/DESTINE_criteo_x1_001_767ae9b5.log rename to ranking/ctr/DESTINE/DESTINE_criteo_x1/DESTINE_criteo_x1_001_767ae9b5.log diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/DESTINE_criteo_x1_tuner_config_03.csv b/ranking/ctr/DESTINE/DESTINE_criteo_x1/DESTINE_criteo_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/DESTINE_criteo_x1_tuner_config_03.csv rename to ranking/ctr/DESTINE/DESTINE_criteo_x1/DESTINE_criteo_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/DESTINE_criteo_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/DESTINE/DESTINE_criteo_x1/DESTINE_criteo_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/DESTINE_criteo_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/DESTINE/DESTINE_criteo_x1/DESTINE_criteo_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/DESTINE_criteo_x1_tuner_config_03/model_config.yaml b/ranking/ctr/DESTINE/DESTINE_criteo_x1/DESTINE_criteo_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/DESTINE_criteo_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/DESTINE/DESTINE_criteo_x1/DESTINE_criteo_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/README.md b/ranking/ctr/DESTINE/DESTINE_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/README.md rename to ranking/ctr/DESTINE/DESTINE_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/environments.txt b/ranking/ctr/DESTINE/DESTINE_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/environments.txt rename to ranking/ctr/DESTINE/DESTINE_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/fuxictr_version.py b/ranking/ctr/DESTINE/DESTINE_criteo_x1/fuxictr_version.py similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/fuxictr_version.py rename to ranking/ctr/DESTINE/DESTINE_criteo_x1/fuxictr_version.py diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/results.csv b/ranking/ctr/DESTINE/DESTINE_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/results.csv rename to ranking/ctr/DESTINE/DESTINE_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/run_expid.py b/ranking/ctr/DESTINE/DESTINE_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_criteo_x1/run_expid.py rename to ranking/ctr/DESTINE/DESTINE_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/DESTINE_frappe_x1_018_8f368b53.log b/ranking/ctr/DESTINE/DESTINE_frappe_x1/DESTINE_frappe_x1_018_8f368b53.log similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/DESTINE_frappe_x1_018_8f368b53.log rename to ranking/ctr/DESTINE/DESTINE_frappe_x1/DESTINE_frappe_x1_018_8f368b53.log diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/DESTINE_frappe_x1_tuner_config_01.csv b/ranking/ctr/DESTINE/DESTINE_frappe_x1/DESTINE_frappe_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/DESTINE_frappe_x1_tuner_config_01.csv rename to ranking/ctr/DESTINE/DESTINE_frappe_x1/DESTINE_frappe_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/DESTINE_frappe_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DESTINE/DESTINE_frappe_x1/DESTINE_frappe_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/DESTINE_frappe_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DESTINE/DESTINE_frappe_x1/DESTINE_frappe_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/DESTINE_frappe_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DESTINE/DESTINE_frappe_x1/DESTINE_frappe_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/DESTINE_frappe_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DESTINE/DESTINE_frappe_x1/DESTINE_frappe_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/README.md b/ranking/ctr/DESTINE/DESTINE_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/README.md rename to ranking/ctr/DESTINE/DESTINE_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/environments.txt b/ranking/ctr/DESTINE/DESTINE_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/environments.txt rename to ranking/ctr/DESTINE/DESTINE_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/fuxictr_version.py b/ranking/ctr/DESTINE/DESTINE_frappe_x1/fuxictr_version.py similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/fuxictr_version.py rename to ranking/ctr/DESTINE/DESTINE_frappe_x1/fuxictr_version.py diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/results.csv b/ranking/ctr/DESTINE/DESTINE_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/results.csv rename to ranking/ctr/DESTINE/DESTINE_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/run_expid.py b/ranking/ctr/DESTINE/DESTINE_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_frappe_x1/run_expid.py rename to ranking/ctr/DESTINE/DESTINE_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/DESTINE_movielenslatest_x1_011_f64ca64d.log b/ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/DESTINE_movielenslatest_x1_011_f64ca64d.log similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/DESTINE_movielenslatest_x1_011_f64ca64d.log rename to ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/DESTINE_movielenslatest_x1_011_f64ca64d.log diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/DESTINE_movielenslatest_x1_tuner_config_03.csv b/ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/DESTINE_movielenslatest_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/DESTINE_movielenslatest_x1_tuner_config_03.csv rename to ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/DESTINE_movielenslatest_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/DESTINE_movielenslatest_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/DESTINE_movielenslatest_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/DESTINE_movielenslatest_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/DESTINE_movielenslatest_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/DESTINE_movielenslatest_x1_tuner_config_03/model_config.yaml b/ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/DESTINE_movielenslatest_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/DESTINE_movielenslatest_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/DESTINE_movielenslatest_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/README.md b/ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/README.md rename to ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/environments.txt b/ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/environments.txt rename to ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/fuxictr_version.py b/ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/fuxictr_version.py similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/fuxictr_version.py rename to ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/fuxictr_version.py diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/results.csv b/ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/results.csv rename to ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/run_expid.py b/ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DESTINE/DESTINE_movielenslatest_x1/run_expid.py rename to ranking/ctr/DESTINE/DESTINE_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_022_a22ee885.log b/ranking/ctr/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_022_a22ee885.log similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_022_a22ee885.log rename to ranking/ctr/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_022_a22ee885.log diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_tuner_config_03.csv b/ranking/ctr/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_tuner_config_03.csv rename to ranking/ctr/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_tuner_config_03.yaml b/ranking/ctr/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_tuner_config_03.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_tuner_config_03.yaml rename to ranking/ctr/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_tuner_config_03.yaml diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_tuner_config_03/model_config.yaml b/ranking/ctr/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/DIEN/DIEN_amazonelectronics_x1/DIEN_amazonelectronics_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_amazonelectronics_x1/README.md b/ranking/ctr/DIEN/DIEN_amazonelectronics_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_amazonelectronics_x1/README.md rename to ranking/ctr/DIEN/DIEN_amazonelectronics_x1/README.md diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_amazonelectronics_x1/environments.txt b/ranking/ctr/DIEN/DIEN_amazonelectronics_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_amazonelectronics_x1/environments.txt rename to ranking/ctr/DIEN/DIEN_amazonelectronics_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_amazonelectronics_x1/results.csv b/ranking/ctr/DIEN/DIEN_amazonelectronics_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_amazonelectronics_x1/results.csv rename to ranking/ctr/DIEN/DIEN_amazonelectronics_x1/results.csv diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_009_657021a9.log b/ranking/ctr/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_009_657021a9.log similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_009_657021a9.log rename to ranking/ctr/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_009_657021a9.log diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_tuner_config_02.csv b/ranking/ctr/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_tuner_config_02.csv rename to ranking/ctr/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_tuner_config_02.yaml b/ranking/ctr/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_tuner_config_02.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_tuner_config_02.yaml rename to ranking/ctr/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_tuner_config_02.yaml diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_tuner_config_02/model_config.yaml b/ranking/ctr/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/DIEN/DIEN_kuaivideo_x1/DIEN_kuaivideo_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_kuaivideo_x1/README.md b/ranking/ctr/DIEN/DIEN_kuaivideo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_kuaivideo_x1/README.md rename to ranking/ctr/DIEN/DIEN_kuaivideo_x1/README.md diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_kuaivideo_x1/environments.txt b/ranking/ctr/DIEN/DIEN_kuaivideo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_kuaivideo_x1/environments.txt rename to ranking/ctr/DIEN/DIEN_kuaivideo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_kuaivideo_x1/results.csv b/ranking/ctr/DIEN/DIEN_kuaivideo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_kuaivideo_x1/results.csv rename to ranking/ctr/DIEN/DIEN_kuaivideo_x1/results.csv diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_013_a9650956.log b/ranking/ctr/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_013_a9650956.log similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_013_a9650956.log rename to ranking/ctr/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_013_a9650956.log diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_tuner_config_03.csv b/ranking/ctr/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_tuner_config_03.csv rename to ranking/ctr/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_tuner_config_03.yaml b/ranking/ctr/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_tuner_config_03.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_tuner_config_03.yaml rename to ranking/ctr/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_tuner_config_03.yaml diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_tuner_config_03/model_config.yaml b/ranking/ctr/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/DIEN/DIEN_microvideo1.7m_x1/DIEN_microvideo1.7m_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_microvideo1.7m_x1/README.md b/ranking/ctr/DIEN/DIEN_microvideo1.7m_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_microvideo1.7m_x1/README.md rename to ranking/ctr/DIEN/DIEN_microvideo1.7m_x1/README.md diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_microvideo1.7m_x1/environments.txt b/ranking/ctr/DIEN/DIEN_microvideo1.7m_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_microvideo1.7m_x1/environments.txt rename to ranking/ctr/DIEN/DIEN_microvideo1.7m_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_microvideo1.7m_x1/results.csv b/ranking/ctr/DIEN/DIEN_microvideo1.7m_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_microvideo1.7m_x1/results.csv rename to ranking/ctr/DIEN/DIEN_microvideo1.7m_x1/results.csv diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_019_5d9b3874.log b/ranking/ctr/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_019_5d9b3874.log similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_019_5d9b3874.log rename to ranking/ctr/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_019_5d9b3874.log diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_tuner_config_01.csv b/ranking/ctr/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_tuner_config_01.csv rename to ranking/ctr/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_tuner_config_01.yaml b/ranking/ctr/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_tuner_config_01.yaml rename to ranking/ctr/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DIEN/DIEN_taobaoad_x1/DIEN_taobaoad_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_taobaoad_x1/README.md b/ranking/ctr/DIEN/DIEN_taobaoad_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_taobaoad_x1/README.md rename to ranking/ctr/DIEN/DIEN_taobaoad_x1/README.md diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_taobaoad_x1/environments.txt b/ranking/ctr/DIEN/DIEN_taobaoad_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_taobaoad_x1/environments.txt rename to ranking/ctr/DIEN/DIEN_taobaoad_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DIEN/DIEN_taobaoad_x1/results.csv b/ranking/ctr/DIEN/DIEN_taobaoad_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DIEN/DIEN_taobaoad_x1/results.csv rename to ranking/ctr/DIEN/DIEN_taobaoad_x1/results.csv diff --git a/ctr_prediction/benchmarks/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_015_8539c013.log b/ranking/ctr/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_015_8539c013.log similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_015_8539c013.log rename to ranking/ctr/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_015_8539c013.log diff --git a/ctr_prediction/benchmarks/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_tuner_config_11.csv b/ranking/ctr/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_tuner_config_11.csv similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_tuner_config_11.csv rename to ranking/ctr/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_tuner_config_11.csv diff --git a/ctr_prediction/benchmarks/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_tuner_config_11.yaml b/ranking/ctr/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_tuner_config_11.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_tuner_config_11.yaml rename to ranking/ctr/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_tuner_config_11.yaml diff --git a/ctr_prediction/benchmarks/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_tuner_config_11/dataset_config.yaml b/ranking/ctr/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_tuner_config_11/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_tuner_config_11/dataset_config.yaml rename to ranking/ctr/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_tuner_config_11/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_tuner_config_11/model_config.yaml b/ranking/ctr/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_tuner_config_11/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_tuner_config_11/model_config.yaml rename to ranking/ctr/DIN/DIN_amazonelectronics_x1/DIN_amazonelectronics_x1_tuner_config_11/model_config.yaml diff --git a/ctr_prediction/benchmarks/DIN/DIN_amazonelectronics_x1/README.md b/ranking/ctr/DIN/DIN_amazonelectronics_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_amazonelectronics_x1/README.md rename to ranking/ctr/DIN/DIN_amazonelectronics_x1/README.md diff --git a/ctr_prediction/benchmarks/DIN/DIN_amazonelectronics_x1/environments.txt b/ranking/ctr/DIN/DIN_amazonelectronics_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_amazonelectronics_x1/environments.txt rename to ranking/ctr/DIN/DIN_amazonelectronics_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DIN/DIN_amazonelectronics_x1/results.csv b/ranking/ctr/DIN/DIN_amazonelectronics_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_amazonelectronics_x1/results.csv rename to ranking/ctr/DIN/DIN_amazonelectronics_x1/results.csv diff --git a/ctr_prediction/benchmarks/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_013_fc4bf206.log b/ranking/ctr/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_013_fc4bf206.log similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_013_fc4bf206.log rename to ranking/ctr/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_013_fc4bf206.log diff --git a/ctr_prediction/benchmarks/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_tuner_config_02.csv b/ranking/ctr/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_tuner_config_02.csv rename to ranking/ctr/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_tuner_config_02.yaml b/ranking/ctr/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_tuner_config_02.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_tuner_config_02.yaml rename to ranking/ctr/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_tuner_config_02.yaml diff --git a/ctr_prediction/benchmarks/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_tuner_config_02/model_config.yaml b/ranking/ctr/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/DIN/DIN_kuaivideo_x1/DIN_kuaivideo_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DIN/DIN_kuaivideo_x1/README.md b/ranking/ctr/DIN/DIN_kuaivideo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_kuaivideo_x1/README.md rename to ranking/ctr/DIN/DIN_kuaivideo_x1/README.md diff --git a/ctr_prediction/benchmarks/DIN/DIN_kuaivideo_x1/environments.txt b/ranking/ctr/DIN/DIN_kuaivideo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_kuaivideo_x1/environments.txt rename to ranking/ctr/DIN/DIN_kuaivideo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DIN/DIN_kuaivideo_x1/results.csv b/ranking/ctr/DIN/DIN_kuaivideo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_kuaivideo_x1/results.csv rename to ranking/ctr/DIN/DIN_kuaivideo_x1/results.csv diff --git a/ctr_prediction/benchmarks/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_006_ab4e3b7f.log b/ranking/ctr/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_006_ab4e3b7f.log similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_006_ab4e3b7f.log rename to ranking/ctr/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_006_ab4e3b7f.log diff --git a/ctr_prediction/benchmarks/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_tuner_config_06.csv b/ranking/ctr/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_tuner_config_06.csv similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_tuner_config_06.csv rename to ranking/ctr/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_tuner_config_06.csv diff --git a/ctr_prediction/benchmarks/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_tuner_config_06.yaml b/ranking/ctr/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_tuner_config_06.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_tuner_config_06.yaml rename to ranking/ctr/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_tuner_config_06.yaml diff --git a/ctr_prediction/benchmarks/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_tuner_config_06/dataset_config.yaml b/ranking/ctr/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_tuner_config_06/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_tuner_config_06/dataset_config.yaml rename to ranking/ctr/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_tuner_config_06/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_tuner_config_06/model_config.yaml b/ranking/ctr/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_tuner_config_06/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_tuner_config_06/model_config.yaml rename to ranking/ctr/DIN/DIN_microvideo1.7m_x1/DIN_microvideo1.7m_x1_tuner_config_06/model_config.yaml diff --git a/ctr_prediction/benchmarks/DIN/DIN_microvideo1.7m_x1/README.md b/ranking/ctr/DIN/DIN_microvideo1.7m_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_microvideo1.7m_x1/README.md rename to ranking/ctr/DIN/DIN_microvideo1.7m_x1/README.md diff --git a/ctr_prediction/benchmarks/DIN/DIN_microvideo1.7m_x1/environments.txt b/ranking/ctr/DIN/DIN_microvideo1.7m_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_microvideo1.7m_x1/environments.txt rename to ranking/ctr/DIN/DIN_microvideo1.7m_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DIN/DIN_microvideo1.7m_x1/results.csv b/ranking/ctr/DIN/DIN_microvideo1.7m_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_microvideo1.7m_x1/results.csv rename to ranking/ctr/DIN/DIN_microvideo1.7m_x1/results.csv diff --git a/ctr_prediction/benchmarks/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_012_13c23a36.log b/ranking/ctr/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_012_13c23a36.log similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_012_13c23a36.log rename to ranking/ctr/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_012_13c23a36.log diff --git a/ctr_prediction/benchmarks/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_tuner_config_07.csv b/ranking/ctr/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_tuner_config_07.csv similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_tuner_config_07.csv rename to ranking/ctr/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_tuner_config_07.csv diff --git a/ctr_prediction/benchmarks/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_tuner_config_07.yaml b/ranking/ctr/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_tuner_config_07.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_tuner_config_07.yaml rename to ranking/ctr/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_tuner_config_07.yaml diff --git a/ctr_prediction/benchmarks/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_tuner_config_07/dataset_config.yaml b/ranking/ctr/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_tuner_config_07/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_tuner_config_07/dataset_config.yaml rename to ranking/ctr/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_tuner_config_07/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_tuner_config_07/model_config.yaml b/ranking/ctr/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_tuner_config_07/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_tuner_config_07/model_config.yaml rename to ranking/ctr/DIN/DIN_taobaoad_x1/DIN_taobaoad_x1_tuner_config_07/model_config.yaml diff --git a/ctr_prediction/benchmarks/DIN/DIN_taobaoad_x1/README.md b/ranking/ctr/DIN/DIN_taobaoad_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_taobaoad_x1/README.md rename to ranking/ctr/DIN/DIN_taobaoad_x1/README.md diff --git a/ctr_prediction/benchmarks/DIN/DIN_taobaoad_x1/environments.txt b/ranking/ctr/DIN/DIN_taobaoad_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_taobaoad_x1/environments.txt rename to ranking/ctr/DIN/DIN_taobaoad_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DIN/DIN_taobaoad_x1/results.csv b/ranking/ctr/DIN/DIN_taobaoad_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DIN/DIN_taobaoad_x1/results.csv rename to ranking/ctr/DIN/DIN_taobaoad_x1/results.csv diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_avazu_x1/DLRM_avazu_x1_015_cf6fdabe.log b/ranking/ctr/DLRM/DLRM_avazu_x1/DLRM_avazu_x1_015_cf6fdabe.log similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_avazu_x1/DLRM_avazu_x1_015_cf6fdabe.log rename to ranking/ctr/DLRM/DLRM_avazu_x1/DLRM_avazu_x1_015_cf6fdabe.log diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_avazu_x1/DLRM_avazu_x1_tuner_config_01.csv b/ranking/ctr/DLRM/DLRM_avazu_x1/DLRM_avazu_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_avazu_x1/DLRM_avazu_x1_tuner_config_01.csv rename to ranking/ctr/DLRM/DLRM_avazu_x1/DLRM_avazu_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_avazu_x1/DLRM_avazu_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DLRM/DLRM_avazu_x1/DLRM_avazu_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_avazu_x1/DLRM_avazu_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DLRM/DLRM_avazu_x1/DLRM_avazu_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_avazu_x1/DLRM_avazu_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DLRM/DLRM_avazu_x1/DLRM_avazu_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_avazu_x1/DLRM_avazu_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DLRM/DLRM_avazu_x1/DLRM_avazu_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_avazu_x1/README.md b/ranking/ctr/DLRM/DLRM_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_avazu_x1/README.md rename to ranking/ctr/DLRM/DLRM_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_avazu_x1/environments.txt b/ranking/ctr/DLRM/DLRM_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_avazu_x1/environments.txt rename to ranking/ctr/DLRM/DLRM_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_avazu_x1/results.csv b/ranking/ctr/DLRM/DLRM_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_avazu_x1/results.csv rename to ranking/ctr/DLRM/DLRM_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_avazu_x1/run_expid.py b/ranking/ctr/DLRM/DLRM_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_avazu_x1/run_expid.py rename to ranking/ctr/DLRM/DLRM_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_criteo_x1/DLRM_criteo_x1_001_4d897285.log b/ranking/ctr/DLRM/DLRM_criteo_x1/DLRM_criteo_x1_001_4d897285.log similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_criteo_x1/DLRM_criteo_x1_001_4d897285.log rename to ranking/ctr/DLRM/DLRM_criteo_x1/DLRM_criteo_x1_001_4d897285.log diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_criteo_x1/DLRM_criteo_x1_tuner_config_02.csv b/ranking/ctr/DLRM/DLRM_criteo_x1/DLRM_criteo_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_criteo_x1/DLRM_criteo_x1_tuner_config_02.csv rename to ranking/ctr/DLRM/DLRM_criteo_x1/DLRM_criteo_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_criteo_x1/DLRM_criteo_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/DLRM/DLRM_criteo_x1/DLRM_criteo_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_criteo_x1/DLRM_criteo_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DLRM/DLRM_criteo_x1/DLRM_criteo_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_criteo_x1/DLRM_criteo_x1_tuner_config_02/model_config.yaml b/ranking/ctr/DLRM/DLRM_criteo_x1/DLRM_criteo_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_criteo_x1/DLRM_criteo_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/DLRM/DLRM_criteo_x1/DLRM_criteo_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_criteo_x1/README.md b/ranking/ctr/DLRM/DLRM_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_criteo_x1/README.md rename to ranking/ctr/DLRM/DLRM_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_criteo_x1/environments.txt b/ranking/ctr/DLRM/DLRM_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_criteo_x1/environments.txt rename to ranking/ctr/DLRM/DLRM_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_criteo_x1/results.csv b/ranking/ctr/DLRM/DLRM_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_criteo_x1/results.csv rename to ranking/ctr/DLRM/DLRM_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_criteo_x1/run_expid.py b/ranking/ctr/DLRM/DLRM_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_criteo_x1/run_expid.py rename to ranking/ctr/DLRM/DLRM_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_frappe_x1/DLRM_frappe_x1_006_216831a3.log b/ranking/ctr/DLRM/DLRM_frappe_x1/DLRM_frappe_x1_006_216831a3.log similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_frappe_x1/DLRM_frappe_x1_006_216831a3.log rename to ranking/ctr/DLRM/DLRM_frappe_x1/DLRM_frappe_x1_006_216831a3.log diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_frappe_x1/DLRM_frappe_x1_tuner_config_02.csv b/ranking/ctr/DLRM/DLRM_frappe_x1/DLRM_frappe_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_frappe_x1/DLRM_frappe_x1_tuner_config_02.csv rename to ranking/ctr/DLRM/DLRM_frappe_x1/DLRM_frappe_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_frappe_x1/DLRM_frappe_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/DLRM/DLRM_frappe_x1/DLRM_frappe_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_frappe_x1/DLRM_frappe_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DLRM/DLRM_frappe_x1/DLRM_frappe_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_frappe_x1/DLRM_frappe_x1_tuner_config_02/model_config.yaml b/ranking/ctr/DLRM/DLRM_frappe_x1/DLRM_frappe_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_frappe_x1/DLRM_frappe_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/DLRM/DLRM_frappe_x1/DLRM_frappe_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_frappe_x1/README.md b/ranking/ctr/DLRM/DLRM_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_frappe_x1/README.md rename to ranking/ctr/DLRM/DLRM_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_frappe_x1/environments.txt b/ranking/ctr/DLRM/DLRM_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_frappe_x1/environments.txt rename to ranking/ctr/DLRM/DLRM_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_frappe_x1/results.csv b/ranking/ctr/DLRM/DLRM_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_frappe_x1/results.csv rename to ranking/ctr/DLRM/DLRM_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_frappe_x1/run_expid.py b/ranking/ctr/DLRM/DLRM_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_frappe_x1/run_expid.py rename to ranking/ctr/DLRM/DLRM_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_movielenslatest_x1/DLRM_movielenslatest_x1_002_333e0a39.log b/ranking/ctr/DLRM/DLRM_movielenslatest_x1/DLRM_movielenslatest_x1_002_333e0a39.log similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_movielenslatest_x1/DLRM_movielenslatest_x1_002_333e0a39.log rename to ranking/ctr/DLRM/DLRM_movielenslatest_x1/DLRM_movielenslatest_x1_002_333e0a39.log diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_movielenslatest_x1/DLRM_movielenslatest_x1_tuner_config_04.csv b/ranking/ctr/DLRM/DLRM_movielenslatest_x1/DLRM_movielenslatest_x1_tuner_config_04.csv similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_movielenslatest_x1/DLRM_movielenslatest_x1_tuner_config_04.csv rename to ranking/ctr/DLRM/DLRM_movielenslatest_x1/DLRM_movielenslatest_x1_tuner_config_04.csv diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_movielenslatest_x1/DLRM_movielenslatest_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/DLRM/DLRM_movielenslatest_x1/DLRM_movielenslatest_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_movielenslatest_x1/DLRM_movielenslatest_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/DLRM/DLRM_movielenslatest_x1/DLRM_movielenslatest_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_movielenslatest_x1/DLRM_movielenslatest_x1_tuner_config_04/model_config.yaml b/ranking/ctr/DLRM/DLRM_movielenslatest_x1/DLRM_movielenslatest_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_movielenslatest_x1/DLRM_movielenslatest_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/DLRM/DLRM_movielenslatest_x1/DLRM_movielenslatest_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_movielenslatest_x1/README.md b/ranking/ctr/DLRM/DLRM_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_movielenslatest_x1/README.md rename to ranking/ctr/DLRM/DLRM_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_movielenslatest_x1/environments.txt b/ranking/ctr/DLRM/DLRM_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_movielenslatest_x1/environments.txt rename to ranking/ctr/DLRM/DLRM_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_movielenslatest_x1/results.csv b/ranking/ctr/DLRM/DLRM_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_movielenslatest_x1/results.csv rename to ranking/ctr/DLRM/DLRM_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/DLRM/DLRM_movielenslatest_x1/run_expid.py b/ranking/ctr/DLRM/DLRM_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DLRM/DLRM_movielenslatest_x1/run_expid.py rename to ranking/ctr/DLRM/DLRM_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_005_0a5f1f4b.log b/ranking/ctr/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_005_0a5f1f4b.log similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_005_0a5f1f4b.log rename to ranking/ctr/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_005_0a5f1f4b.log diff --git a/ctr_prediction/benchmarks/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_tuner_config_01.csv b/ranking/ctr/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_tuner_config_01.csv rename to ranking/ctr/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_tuner_config_01.yaml b/ranking/ctr/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_tuner_config_01.yaml rename to ranking/ctr/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DNN/DNN_amazonelectronics_x1/DNN_amazonelectronics_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_amazonelectronics_x1/README.md b/ranking/ctr/DNN/DNN_amazonelectronics_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_amazonelectronics_x1/README.md rename to ranking/ctr/DNN/DNN_amazonelectronics_x1/README.md diff --git a/ctr_prediction/benchmarks/DNN/DNN_amazonelectronics_x1/environments.txt b/ranking/ctr/DNN/DNN_amazonelectronics_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_amazonelectronics_x1/environments.txt rename to ranking/ctr/DNN/DNN_amazonelectronics_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DNN/DNN_amazonelectronics_x1/results.csv b/ranking/ctr/DNN/DNN_amazonelectronics_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_amazonelectronics_x1/results.csv rename to ranking/ctr/DNN/DNN_amazonelectronics_x1/results.csv diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x1/DNN_avazu_x1_001_3da2d674.log b/ranking/ctr/DNN/DNN_avazu_x1/DNN_avazu_x1_001_3da2d674.log similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x1/DNN_avazu_x1_001_3da2d674.log rename to ranking/ctr/DNN/DNN_avazu_x1/DNN_avazu_x1_001_3da2d674.log diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x1/DNN_avazu_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DNN/DNN_avazu_x1/DNN_avazu_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x1/DNN_avazu_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DNN/DNN_avazu_x1/DNN_avazu_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x1/DNN_avazu_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DNN/DNN_avazu_x1/DNN_avazu_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x1/DNN_avazu_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DNN/DNN_avazu_x1/DNN_avazu_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x1/README.md b/ranking/ctr/DNN/DNN_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x1/README.md rename to ranking/ctr/DNN/DNN_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x1/environments.txt b/ranking/ctr/DNN/DNN_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x1/environments.txt rename to ranking/ctr/DNN/DNN_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x1/results.csv b/ranking/ctr/DNN/DNN_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x1/results.csv rename to ranking/ctr/DNN/DNN_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x1/run_expid.py b/ranking/ctr/DNN/DNN_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x1/run_expid.py rename to ranking/ctr/DNN/DNN_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x4_001/DNN_avazu_x4_021_144a901f.log b/ranking/ctr/DNN/DNN_avazu_x4_001/DNN_avazu_x4_021_144a901f.log similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x4_001/DNN_avazu_x4_021_144a901f.log rename to ranking/ctr/DNN/DNN_avazu_x4_001/DNN_avazu_x4_021_144a901f.log diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x4_001/DNN_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/DNN/DNN_avazu_x4_001/DNN_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x4_001/DNN_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DNN/DNN_avazu_x4_001/DNN_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x4_001/DNN_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/DNN/DNN_avazu_x4_001/DNN_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x4_001/DNN_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/DNN/DNN_avazu_x4_001/DNN_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x4_001/README.md b/ranking/ctr/DNN/DNN_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x4_001/README.md rename to ranking/ctr/DNN/DNN_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x4_001/environments.txt b/ranking/ctr/DNN/DNN_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x4_001/environments.txt rename to ranking/ctr/DNN/DNN_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x4_001/results.csv b/ranking/ctr/DNN/DNN_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x4_001/results.csv rename to ranking/ctr/DNN/DNN_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x4_001/run_expid.py b/ranking/ctr/DNN/DNN_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x4_001/run_expid.py rename to ranking/ctr/DNN/DNN_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x4_002/DNN_avazu_x4_003_7fd77485.log b/ranking/ctr/DNN/DNN_avazu_x4_002/DNN_avazu_x4_003_7fd77485.log similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x4_002/DNN_avazu_x4_003_7fd77485.log rename to ranking/ctr/DNN/DNN_avazu_x4_002/DNN_avazu_x4_003_7fd77485.log diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x4_002/DNN_avazu_x4_tuner_config_03/dataset_config.yaml b/ranking/ctr/DNN/DNN_avazu_x4_002/DNN_avazu_x4_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x4_002/DNN_avazu_x4_tuner_config_03/dataset_config.yaml rename to ranking/ctr/DNN/DNN_avazu_x4_002/DNN_avazu_x4_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x4_002/DNN_avazu_x4_tuner_config_03/model_config.yaml b/ranking/ctr/DNN/DNN_avazu_x4_002/DNN_avazu_x4_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x4_002/DNN_avazu_x4_tuner_config_03/model_config.yaml rename to ranking/ctr/DNN/DNN_avazu_x4_002/DNN_avazu_x4_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x4_002/README.md b/ranking/ctr/DNN/DNN_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x4_002/README.md rename to ranking/ctr/DNN/DNN_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x4_002/environments.txt b/ranking/ctr/DNN/DNN_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x4_002/environments.txt rename to ranking/ctr/DNN/DNN_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x4_002/results.csv b/ranking/ctr/DNN/DNN_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x4_002/results.csv rename to ranking/ctr/DNN/DNN_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/DNN/DNN_avazu_x4_002/run_expid.py b/ranking/ctr/DNN/DNN_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_avazu_x4_002/run_expid.py rename to ranking/ctr/DNN/DNN_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x1/DNN_criteo_x1_001_be50edb0.log b/ranking/ctr/DNN/DNN_criteo_x1/DNN_criteo_x1_001_be50edb0.log similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x1/DNN_criteo_x1_001_be50edb0.log rename to ranking/ctr/DNN/DNN_criteo_x1/DNN_criteo_x1_001_be50edb0.log diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x1/DNN_criteo_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DNN/DNN_criteo_x1/DNN_criteo_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x1/DNN_criteo_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DNN/DNN_criteo_x1/DNN_criteo_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x1/DNN_criteo_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DNN/DNN_criteo_x1/DNN_criteo_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x1/DNN_criteo_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DNN/DNN_criteo_x1/DNN_criteo_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x1/README.md b/ranking/ctr/DNN/DNN_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x1/README.md rename to ranking/ctr/DNN/DNN_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x1/environments.txt b/ranking/ctr/DNN/DNN_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x1/environments.txt rename to ranking/ctr/DNN/DNN_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x1/results.csv b/ranking/ctr/DNN/DNN_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x1/results.csv rename to ranking/ctr/DNN/DNN_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x1/run_expid.py b/ranking/ctr/DNN/DNN_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x1/run_expid.py rename to ranking/ctr/DNN/DNN_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x4_001/DNN_criteo_x4_024_673e1651.log b/ranking/ctr/DNN/DNN_criteo_x4_001/DNN_criteo_x4_024_673e1651.log similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x4_001/DNN_criteo_x4_024_673e1651.log rename to ranking/ctr/DNN/DNN_criteo_x4_001/DNN_criteo_x4_024_673e1651.log diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x4_001/DNN_criteo_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/DNN/DNN_criteo_x4_001/DNN_criteo_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x4_001/DNN_criteo_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DNN/DNN_criteo_x4_001/DNN_criteo_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x4_001/DNN_criteo_x4_tuner_config_02/model_config.yaml b/ranking/ctr/DNN/DNN_criteo_x4_001/DNN_criteo_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x4_001/DNN_criteo_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/DNN/DNN_criteo_x4_001/DNN_criteo_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x4_001/README.md b/ranking/ctr/DNN/DNN_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x4_001/README.md rename to ranking/ctr/DNN/DNN_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x4_001/environments.txt b/ranking/ctr/DNN/DNN_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x4_001/environments.txt rename to ranking/ctr/DNN/DNN_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x4_001/results.csv b/ranking/ctr/DNN/DNN_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x4_001/results.csv rename to ranking/ctr/DNN/DNN_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x4_001/run_expid.py b/ranking/ctr/DNN/DNN_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x4_001/run_expid.py rename to ranking/ctr/DNN/DNN_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x4_002/DNN_criteo_x4_031_90b5a63c.log b/ranking/ctr/DNN/DNN_criteo_x4_002/DNN_criteo_x4_031_90b5a63c.log similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x4_002/DNN_criteo_x4_031_90b5a63c.log rename to ranking/ctr/DNN/DNN_criteo_x4_002/DNN_criteo_x4_031_90b5a63c.log diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x4_002/DNN_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/DNN/DNN_criteo_x4_002/DNN_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x4_002/DNN_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DNN/DNN_criteo_x4_002/DNN_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x4_002/DNN_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/DNN/DNN_criteo_x4_002/DNN_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x4_002/DNN_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/DNN/DNN_criteo_x4_002/DNN_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x4_002/README.md b/ranking/ctr/DNN/DNN_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x4_002/README.md rename to ranking/ctr/DNN/DNN_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x4_002/environments.txt b/ranking/ctr/DNN/DNN_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x4_002/environments.txt rename to ranking/ctr/DNN/DNN_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x4_002/results.csv b/ranking/ctr/DNN/DNN_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x4_002/results.csv rename to ranking/ctr/DNN/DNN_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/DNN/DNN_criteo_x4_002/run_expid.py b/ranking/ctr/DNN/DNN_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_criteo_x4_002/run_expid.py rename to ranking/ctr/DNN/DNN_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/DNN/DNN_frappe_x1/DNN_frappe_x1_001_ad0f0fce.log b/ranking/ctr/DNN/DNN_frappe_x1/DNN_frappe_x1_001_ad0f0fce.log similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_frappe_x1/DNN_frappe_x1_001_ad0f0fce.log rename to ranking/ctr/DNN/DNN_frappe_x1/DNN_frappe_x1_001_ad0f0fce.log diff --git a/ctr_prediction/benchmarks/DNN/DNN_frappe_x1/DNN_frappe_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/DNN/DNN_frappe_x1/DNN_frappe_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_frappe_x1/DNN_frappe_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DNN/DNN_frappe_x1/DNN_frappe_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_frappe_x1/DNN_frappe_x1_tuner_config_02/model_config.yaml b/ranking/ctr/DNN/DNN_frappe_x1/DNN_frappe_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_frappe_x1/DNN_frappe_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/DNN/DNN_frappe_x1/DNN_frappe_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_frappe_x1/README.md b/ranking/ctr/DNN/DNN_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_frappe_x1/README.md rename to ranking/ctr/DNN/DNN_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/DNN/DNN_frappe_x1/environments.txt b/ranking/ctr/DNN/DNN_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_frappe_x1/environments.txt rename to ranking/ctr/DNN/DNN_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DNN/DNN_frappe_x1/results.csv b/ranking/ctr/DNN/DNN_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_frappe_x1/results.csv rename to ranking/ctr/DNN/DNN_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/DNN/DNN_frappe_x1/run_expid.py b/ranking/ctr/DNN/DNN_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_frappe_x1/run_expid.py rename to ranking/ctr/DNN/DNN_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DNN/DNN_kkbox_x1/DNN_kkbox_x1_021_6036677c.log b/ranking/ctr/DNN/DNN_kkbox_x1/DNN_kkbox_x1_021_6036677c.log similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_kkbox_x1/DNN_kkbox_x1_021_6036677c.log rename to ranking/ctr/DNN/DNN_kkbox_x1/DNN_kkbox_x1_021_6036677c.log diff --git a/ctr_prediction/benchmarks/DNN/DNN_kkbox_x1/DNN_kkbox_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/DNN/DNN_kkbox_x1/DNN_kkbox_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_kkbox_x1/DNN_kkbox_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DNN/DNN_kkbox_x1/DNN_kkbox_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_kkbox_x1/DNN_kkbox_x1_tuner_config_02/model_config.yaml b/ranking/ctr/DNN/DNN_kkbox_x1/DNN_kkbox_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_kkbox_x1/DNN_kkbox_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/DNN/DNN_kkbox_x1/DNN_kkbox_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_kkbox_x1/README.md b/ranking/ctr/DNN/DNN_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_kkbox_x1/README.md rename to ranking/ctr/DNN/DNN_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/DNN/DNN_kkbox_x1/environments.txt b/ranking/ctr/DNN/DNN_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_kkbox_x1/environments.txt rename to ranking/ctr/DNN/DNN_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DNN/DNN_kkbox_x1/results.csv b/ranking/ctr/DNN/DNN_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_kkbox_x1/results.csv rename to ranking/ctr/DNN/DNN_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/DNN/DNN_kkbox_x1/run_expid.py b/ranking/ctr/DNN/DNN_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_kkbox_x1/run_expid.py rename to ranking/ctr/DNN/DNN_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_023_2da45570.log b/ranking/ctr/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_023_2da45570.log similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_023_2da45570.log rename to ranking/ctr/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_023_2da45570.log diff --git a/ctr_prediction/benchmarks/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_tuner_config_01.csv b/ranking/ctr/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_tuner_config_01.csv rename to ranking/ctr/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_tuner_config_01.yaml b/ranking/ctr/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_tuner_config_01.yaml rename to ranking/ctr/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DNN/DNN_kuaivideo_x1/DNN_kuaivideo_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_kuaivideo_x1/README.md b/ranking/ctr/DNN/DNN_kuaivideo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_kuaivideo_x1/README.md rename to ranking/ctr/DNN/DNN_kuaivideo_x1/README.md diff --git a/ctr_prediction/benchmarks/DNN/DNN_kuaivideo_x1/environments.txt b/ranking/ctr/DNN/DNN_kuaivideo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_kuaivideo_x1/environments.txt rename to ranking/ctr/DNN/DNN_kuaivideo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DNN/DNN_kuaivideo_x1/results.csv b/ranking/ctr/DNN/DNN_kuaivideo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_kuaivideo_x1/results.csv rename to ranking/ctr/DNN/DNN_kuaivideo_x1/results.csv diff --git a/ctr_prediction/benchmarks/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_020_eb07caea.log b/ranking/ctr/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_020_eb07caea.log similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_020_eb07caea.log rename to ranking/ctr/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_020_eb07caea.log diff --git a/ctr_prediction/benchmarks/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_tuner_config_01.csv b/ranking/ctr/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_tuner_config_01.csv rename to ranking/ctr/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_tuner_config_01.yaml b/ranking/ctr/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_tuner_config_01.yaml rename to ranking/ctr/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DNN/DNN_microvideo1.7m_x1/DNN_microvideo1.7m_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_microvideo1.7m_x1/README.md b/ranking/ctr/DNN/DNN_microvideo1.7m_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_microvideo1.7m_x1/README.md rename to ranking/ctr/DNN/DNN_microvideo1.7m_x1/README.md diff --git a/ctr_prediction/benchmarks/DNN/DNN_microvideo1.7m_x1/environments.txt b/ranking/ctr/DNN/DNN_microvideo1.7m_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_microvideo1.7m_x1/environments.txt rename to ranking/ctr/DNN/DNN_microvideo1.7m_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DNN/DNN_microvideo1.7m_x1/results.csv b/ranking/ctr/DNN/DNN_microvideo1.7m_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_microvideo1.7m_x1/results.csv rename to ranking/ctr/DNN/DNN_microvideo1.7m_x1/results.csv diff --git a/ctr_prediction/benchmarks/DNN/DNN_movielenslatest_x1/DNN_movielenslatest_x1_001_338b5be6.log b/ranking/ctr/DNN/DNN_movielenslatest_x1/DNN_movielenslatest_x1_001_338b5be6.log similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_movielenslatest_x1/DNN_movielenslatest_x1_001_338b5be6.log rename to ranking/ctr/DNN/DNN_movielenslatest_x1/DNN_movielenslatest_x1_001_338b5be6.log diff --git a/ctr_prediction/benchmarks/DNN/DNN_movielenslatest_x1/DNN_movielenslatest_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/DNN/DNN_movielenslatest_x1/DNN_movielenslatest_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_movielenslatest_x1/DNN_movielenslatest_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DNN/DNN_movielenslatest_x1/DNN_movielenslatest_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_movielenslatest_x1/DNN_movielenslatest_x1_tuner_config_02/model_config.yaml b/ranking/ctr/DNN/DNN_movielenslatest_x1/DNN_movielenslatest_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_movielenslatest_x1/DNN_movielenslatest_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/DNN/DNN_movielenslatest_x1/DNN_movielenslatest_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_movielenslatest_x1/README.md b/ranking/ctr/DNN/DNN_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_movielenslatest_x1/README.md rename to ranking/ctr/DNN/DNN_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/DNN/DNN_movielenslatest_x1/environments.txt b/ranking/ctr/DNN/DNN_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_movielenslatest_x1/environments.txt rename to ranking/ctr/DNN/DNN_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DNN/DNN_movielenslatest_x1/results.csv b/ranking/ctr/DNN/DNN_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_movielenslatest_x1/results.csv rename to ranking/ctr/DNN/DNN_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/DNN/DNN_movielenslatest_x1/run_expid.py b/ranking/ctr/DNN/DNN_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_movielenslatest_x1/run_expid.py rename to ranking/ctr/DNN/DNN_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DNN/DNN_taobaoad_x1/DNN_taobao_x1_tuner_config_01.csv b/ranking/ctr/DNN/DNN_taobaoad_x1/DNN_taobao_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_taobaoad_x1/DNN_taobao_x1_tuner_config_01.csv rename to ranking/ctr/DNN/DNN_taobaoad_x1/DNN_taobao_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DNN/DNN_taobaoad_x1/DNN_taobao_x1_tuner_config_01.yaml b/ranking/ctr/DNN/DNN_taobaoad_x1/DNN_taobao_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_taobaoad_x1/DNN_taobao_x1_tuner_config_01.yaml rename to ranking/ctr/DNN/DNN_taobaoad_x1/DNN_taobao_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_taobaoad_x1/DNN_taobao_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DNN/DNN_taobaoad_x1/DNN_taobao_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_taobaoad_x1/DNN_taobao_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DNN/DNN_taobaoad_x1/DNN_taobao_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_taobaoad_x1/DNN_taobao_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DNN/DNN_taobaoad_x1/DNN_taobao_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_taobaoad_x1/DNN_taobao_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DNN/DNN_taobaoad_x1/DNN_taobao_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DNN/DNN_taobaoad_x1/DNN_taobaoad_x1_009_0f4cc5a8.log b/ranking/ctr/DNN/DNN_taobaoad_x1/DNN_taobaoad_x1_009_0f4cc5a8.log similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_taobaoad_x1/DNN_taobaoad_x1_009_0f4cc5a8.log rename to ranking/ctr/DNN/DNN_taobaoad_x1/DNN_taobaoad_x1_009_0f4cc5a8.log diff --git a/ctr_prediction/benchmarks/DNN/DNN_taobaoad_x1/README.md b/ranking/ctr/DNN/DNN_taobaoad_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_taobaoad_x1/README.md rename to ranking/ctr/DNN/DNN_taobaoad_x1/README.md diff --git a/ctr_prediction/benchmarks/DNN/DNN_taobaoad_x1/environments.txt b/ranking/ctr/DNN/DNN_taobaoad_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_taobaoad_x1/environments.txt rename to ranking/ctr/DNN/DNN_taobaoad_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DNN/DNN_taobaoad_x1/results.csv b/ranking/ctr/DNN/DNN_taobaoad_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DNN/DNN_taobaoad_x1/results.csv rename to ranking/ctr/DNN/DNN_taobaoad_x1/results.csv diff --git a/ctr_prediction/benchmarks/DSSM/DSSM_frappe_x1/DSSM_frappe_x1_006_4be388eb.log b/ranking/ctr/DSSM/DSSM_frappe_x1/DSSM_frappe_x1_006_4be388eb.log similarity index 100% rename from ctr_prediction/benchmarks/DSSM/DSSM_frappe_x1/DSSM_frappe_x1_006_4be388eb.log rename to ranking/ctr/DSSM/DSSM_frappe_x1/DSSM_frappe_x1_006_4be388eb.log diff --git a/ctr_prediction/benchmarks/DSSM/DSSM_frappe_x1/DSSM_frappe_x1_tuner_config_02.csv b/ranking/ctr/DSSM/DSSM_frappe_x1/DSSM_frappe_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/DSSM/DSSM_frappe_x1/DSSM_frappe_x1_tuner_config_02.csv rename to ranking/ctr/DSSM/DSSM_frappe_x1/DSSM_frappe_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/DSSM/DSSM_frappe_x1/DSSM_frappe_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/DSSM/DSSM_frappe_x1/DSSM_frappe_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DSSM/DSSM_frappe_x1/DSSM_frappe_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DSSM/DSSM_frappe_x1/DSSM_frappe_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DSSM/DSSM_frappe_x1/DSSM_frappe_x1_tuner_config_02/model_config.yaml b/ranking/ctr/DSSM/DSSM_frappe_x1/DSSM_frappe_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DSSM/DSSM_frappe_x1/DSSM_frappe_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/DSSM/DSSM_frappe_x1/DSSM_frappe_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DSSM/DSSM_frappe_x1/README.md b/ranking/ctr/DSSM/DSSM_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DSSM/DSSM_frappe_x1/README.md rename to ranking/ctr/DSSM/DSSM_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/DSSM/DSSM_frappe_x1/environments.txt b/ranking/ctr/DSSM/DSSM_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DSSM/DSSM_frappe_x1/environments.txt rename to ranking/ctr/DSSM/DSSM_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DSSM/DSSM_frappe_x1/results.csv b/ranking/ctr/DSSM/DSSM_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DSSM/DSSM_frappe_x1/results.csv rename to ranking/ctr/DSSM/DSSM_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/DSSM/DSSM_frappe_x1/run_expid.py b/ranking/ctr/DSSM/DSSM_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DSSM/DSSM_frappe_x1/run_expid.py rename to ranking/ctr/DSSM/DSSM_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DSSM/DSSM_movielenslatest_x1/DSSM_movielenslatest_x1_001_945a31b2.log b/ranking/ctr/DSSM/DSSM_movielenslatest_x1/DSSM_movielenslatest_x1_001_945a31b2.log similarity index 100% rename from ctr_prediction/benchmarks/DSSM/DSSM_movielenslatest_x1/DSSM_movielenslatest_x1_001_945a31b2.log rename to ranking/ctr/DSSM/DSSM_movielenslatest_x1/DSSM_movielenslatest_x1_001_945a31b2.log diff --git a/ctr_prediction/benchmarks/DSSM/DSSM_movielenslatest_x1/DSSM_movielenslatest_x1_tuner_config_01.csv b/ranking/ctr/DSSM/DSSM_movielenslatest_x1/DSSM_movielenslatest_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DSSM/DSSM_movielenslatest_x1/DSSM_movielenslatest_x1_tuner_config_01.csv rename to ranking/ctr/DSSM/DSSM_movielenslatest_x1/DSSM_movielenslatest_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DSSM/DSSM_movielenslatest_x1/DSSM_movielenslatest_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DSSM/DSSM_movielenslatest_x1/DSSM_movielenslatest_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DSSM/DSSM_movielenslatest_x1/DSSM_movielenslatest_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DSSM/DSSM_movielenslatest_x1/DSSM_movielenslatest_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DSSM/DSSM_movielenslatest_x1/DSSM_movielenslatest_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DSSM/DSSM_movielenslatest_x1/DSSM_movielenslatest_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DSSM/DSSM_movielenslatest_x1/DSSM_movielenslatest_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DSSM/DSSM_movielenslatest_x1/DSSM_movielenslatest_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DSSM/DSSM_movielenslatest_x1/README.md b/ranking/ctr/DSSM/DSSM_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DSSM/DSSM_movielenslatest_x1/README.md rename to ranking/ctr/DSSM/DSSM_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/DSSM/DSSM_movielenslatest_x1/environments.txt b/ranking/ctr/DSSM/DSSM_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DSSM/DSSM_movielenslatest_x1/environments.txt rename to ranking/ctr/DSSM/DSSM_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DSSM/DSSM_movielenslatest_x1/results.csv b/ranking/ctr/DSSM/DSSM_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DSSM/DSSM_movielenslatest_x1/results.csv rename to ranking/ctr/DSSM/DSSM_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/DSSM/DSSM_movielenslatest_x1/run_expid.py b/ranking/ctr/DSSM/DSSM_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DSSM/DSSM_movielenslatest_x1/run_expid.py rename to ranking/ctr/DSSM/DSSM_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_001/DeepCrossing_avazu_x4_005_a6cf8324.log b/ranking/ctr/DeepCrossing/DeepCross_avazu_x4_001/DeepCrossing_avazu_x4_005_a6cf8324.log similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_001/DeepCrossing_avazu_x4_005_a6cf8324.log rename to ranking/ctr/DeepCrossing/DeepCross_avazu_x4_001/DeepCrossing_avazu_x4_005_a6cf8324.log diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_001/DeepCrossing_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/DeepCrossing/DeepCross_avazu_x4_001/DeepCrossing_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_001/DeepCrossing_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DeepCrossing/DeepCross_avazu_x4_001/DeepCrossing_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_001/DeepCrossing_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/DeepCrossing/DeepCross_avazu_x4_001/DeepCrossing_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_001/DeepCrossing_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/DeepCrossing/DeepCross_avazu_x4_001/DeepCrossing_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_001/README.md b/ranking/ctr/DeepCrossing/DeepCross_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_001/README.md rename to ranking/ctr/DeepCrossing/DeepCross_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_001/environments.txt b/ranking/ctr/DeepCrossing/DeepCross_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_001/environments.txt rename to ranking/ctr/DeepCrossing/DeepCross_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_001/results.csv b/ranking/ctr/DeepCrossing/DeepCross_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_001/results.csv rename to ranking/ctr/DeepCrossing/DeepCross_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_001/run_expid.py b/ranking/ctr/DeepCrossing/DeepCross_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_001/run_expid.py rename to ranking/ctr/DeepCrossing/DeepCross_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_002/DeepCrossing_avazu_x4_040_aa3259c8.log b/ranking/ctr/DeepCrossing/DeepCross_avazu_x4_002/DeepCrossing_avazu_x4_040_aa3259c8.log similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_002/DeepCrossing_avazu_x4_040_aa3259c8.log rename to ranking/ctr/DeepCrossing/DeepCross_avazu_x4_002/DeepCrossing_avazu_x4_040_aa3259c8.log diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_002/DeepCrossing_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/DeepCrossing/DeepCross_avazu_x4_002/DeepCrossing_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_002/DeepCrossing_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DeepCrossing/DeepCross_avazu_x4_002/DeepCrossing_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_002/DeepCrossing_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/DeepCrossing/DeepCross_avazu_x4_002/DeepCrossing_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_002/DeepCrossing_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/DeepCrossing/DeepCross_avazu_x4_002/DeepCrossing_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_002/README.md b/ranking/ctr/DeepCrossing/DeepCross_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_002/README.md rename to ranking/ctr/DeepCrossing/DeepCross_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_002/environments.txt b/ranking/ctr/DeepCrossing/DeepCross_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_002/environments.txt rename to ranking/ctr/DeepCrossing/DeepCross_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_002/results.csv b/ranking/ctr/DeepCrossing/DeepCross_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_002/results.csv rename to ranking/ctr/DeepCrossing/DeepCross_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_002/run_expid.py b/ranking/ctr/DeepCrossing/DeepCross_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_avazu_x4_002/run_expid.py rename to ranking/ctr/DeepCrossing/DeepCross_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_001/DeepCrossing_criteo_x4_018_3638c8fb.log b/ranking/ctr/DeepCrossing/DeepCross_criteo_x4_001/DeepCrossing_criteo_x4_018_3638c8fb.log similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_001/DeepCrossing_criteo_x4_018_3638c8fb.log rename to ranking/ctr/DeepCrossing/DeepCross_criteo_x4_001/DeepCrossing_criteo_x4_018_3638c8fb.log diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_001/DeepCrossing_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/DeepCrossing/DeepCross_criteo_x4_001/DeepCrossing_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_001/DeepCrossing_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DeepCrossing/DeepCross_criteo_x4_001/DeepCrossing_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_001/DeepCrossing_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/DeepCrossing/DeepCross_criteo_x4_001/DeepCrossing_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_001/DeepCrossing_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/DeepCrossing/DeepCross_criteo_x4_001/DeepCrossing_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_001/README.md b/ranking/ctr/DeepCrossing/DeepCross_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_001/README.md rename to ranking/ctr/DeepCrossing/DeepCross_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_001/environments.txt b/ranking/ctr/DeepCrossing/DeepCross_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_001/environments.txt rename to ranking/ctr/DeepCrossing/DeepCross_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_001/results.csv b/ranking/ctr/DeepCrossing/DeepCross_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_001/results.csv rename to ranking/ctr/DeepCrossing/DeepCross_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_001/run_expid.py b/ranking/ctr/DeepCrossing/DeepCross_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_001/run_expid.py rename to ranking/ctr/DeepCrossing/DeepCross_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_002/DeepCrossing_criteo_x4_056_0f92ea50.log b/ranking/ctr/DeepCrossing/DeepCross_criteo_x4_002/DeepCrossing_criteo_x4_056_0f92ea50.log similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_002/DeepCrossing_criteo_x4_056_0f92ea50.log rename to ranking/ctr/DeepCrossing/DeepCross_criteo_x4_002/DeepCrossing_criteo_x4_056_0f92ea50.log diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_002/DeepCrossing_criteo_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/DeepCrossing/DeepCross_criteo_x4_002/DeepCrossing_criteo_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_002/DeepCrossing_criteo_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DeepCrossing/DeepCross_criteo_x4_002/DeepCrossing_criteo_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_002/DeepCrossing_criteo_x4_tuner_config_02/model_config.yaml b/ranking/ctr/DeepCrossing/DeepCross_criteo_x4_002/DeepCrossing_criteo_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_002/DeepCrossing_criteo_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/DeepCrossing/DeepCross_criteo_x4_002/DeepCrossing_criteo_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_002/README.md b/ranking/ctr/DeepCrossing/DeepCross_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_002/README.md rename to ranking/ctr/DeepCrossing/DeepCross_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_002/environments.txt b/ranking/ctr/DeepCrossing/DeepCross_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_002/environments.txt rename to ranking/ctr/DeepCrossing/DeepCross_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_002/results.csv b/ranking/ctr/DeepCrossing/DeepCross_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_002/results.csv rename to ranking/ctr/DeepCrossing/DeepCross_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_002/run_expid.py b/ranking/ctr/DeepCrossing/DeepCross_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_criteo_x4_002/run_expid.py rename to ranking/ctr/DeepCrossing/DeepCross_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_kkbox_x1/DeepCrossing_kkbox_x1_022_34237390.log b/ranking/ctr/DeepCrossing/DeepCross_kkbox_x1/DeepCrossing_kkbox_x1_022_34237390.log similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_kkbox_x1/DeepCrossing_kkbox_x1_022_34237390.log rename to ranking/ctr/DeepCrossing/DeepCross_kkbox_x1/DeepCrossing_kkbox_x1_022_34237390.log diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_kkbox_x1/DeepCrossing_kkbox_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/DeepCrossing/DeepCross_kkbox_x1/DeepCrossing_kkbox_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_kkbox_x1/DeepCrossing_kkbox_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DeepCrossing/DeepCross_kkbox_x1/DeepCrossing_kkbox_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_kkbox_x1/DeepCrossing_kkbox_x1_tuner_config_02/model_config.yaml b/ranking/ctr/DeepCrossing/DeepCross_kkbox_x1/DeepCrossing_kkbox_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_kkbox_x1/DeepCrossing_kkbox_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/DeepCrossing/DeepCross_kkbox_x1/DeepCrossing_kkbox_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_kkbox_x1/README.md b/ranking/ctr/DeepCrossing/DeepCross_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_kkbox_x1/README.md rename to ranking/ctr/DeepCrossing/DeepCross_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_kkbox_x1/environments.txt b/ranking/ctr/DeepCrossing/DeepCross_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_kkbox_x1/environments.txt rename to ranking/ctr/DeepCrossing/DeepCross_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_kkbox_x1/results.csv b/ranking/ctr/DeepCrossing/DeepCross_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_kkbox_x1/results.csv rename to ranking/ctr/DeepCrossing/DeepCross_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/DeepCrossing/DeepCross_kkbox_x1/run_expid.py b/ranking/ctr/DeepCrossing/DeepCross_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepCrossing/DeepCross_kkbox_x1/run_expid.py rename to ranking/ctr/DeepCrossing/DeepCross_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_014_68eadb7e.log b/ranking/ctr/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_014_68eadb7e.log similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_014_68eadb7e.log rename to ranking/ctr/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_014_68eadb7e.log diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_tuner_config_02.csv b/ranking/ctr/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_tuner_config_02.csv rename to ranking/ctr/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_tuner_config_02.yaml b/ranking/ctr/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_tuner_config_02.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_tuner_config_02.yaml rename to ranking/ctr/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_tuner_config_02.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_tuner_config_02/model_config.yaml b/ranking/ctr/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/DeepFM/DeepFM_amazonelectronics_x1/DeepFM_amazonelectronics_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_amazonelectronics_x1/README.md b/ranking/ctr/DeepFM/DeepFM_amazonelectronics_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_amazonelectronics_x1/README.md rename to ranking/ctr/DeepFM/DeepFM_amazonelectronics_x1/README.md diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_amazonelectronics_x1/environments.txt b/ranking/ctr/DeepFM/DeepFM_amazonelectronics_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_amazonelectronics_x1/environments.txt rename to ranking/ctr/DeepFM/DeepFM_amazonelectronics_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_amazonelectronics_x1/results.csv b/ranking/ctr/DeepFM/DeepFM_amazonelectronics_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_amazonelectronics_x1/results.csv rename to ranking/ctr/DeepFM/DeepFM_amazonelectronics_x1/results.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x1/DeepFM_avazu_x1_004_514a2b87.log b/ranking/ctr/DeepFM/DeepFM_avazu_x1/DeepFM_avazu_x1_004_514a2b87.log similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x1/DeepFM_avazu_x1_004_514a2b87.log rename to ranking/ctr/DeepFM/DeepFM_avazu_x1/DeepFM_avazu_x1_004_514a2b87.log diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x1/DeepFM_avazu_x1_tuner_config_02.csv b/ranking/ctr/DeepFM/DeepFM_avazu_x1/DeepFM_avazu_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x1/DeepFM_avazu_x1_tuner_config_02.csv rename to ranking/ctr/DeepFM/DeepFM_avazu_x1/DeepFM_avazu_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x1/DeepFM_avazu_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/DeepFM/DeepFM_avazu_x1/DeepFM_avazu_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x1/DeepFM_avazu_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DeepFM/DeepFM_avazu_x1/DeepFM_avazu_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x1/DeepFM_avazu_x1_tuner_config_02/model_config.yaml b/ranking/ctr/DeepFM/DeepFM_avazu_x1/DeepFM_avazu_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x1/DeepFM_avazu_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/DeepFM/DeepFM_avazu_x1/DeepFM_avazu_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x1/README.md b/ranking/ctr/DeepFM/DeepFM_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x1/README.md rename to ranking/ctr/DeepFM/DeepFM_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x1/environments.txt b/ranking/ctr/DeepFM/DeepFM_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x1/environments.txt rename to ranking/ctr/DeepFM/DeepFM_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x1/results.csv b/ranking/ctr/DeepFM/DeepFM_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x1/results.csv rename to ranking/ctr/DeepFM/DeepFM_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x1/run_expid.py b/ranking/ctr/DeepFM/DeepFM_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x1/run_expid.py rename to ranking/ctr/DeepFM/DeepFM_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_001/DeepFM_avazu_x4_019_9e93795e.log b/ranking/ctr/DeepFM/DeepFM_avazu_x4_001/DeepFM_avazu_x4_019_9e93795e.log similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_001/DeepFM_avazu_x4_019_9e93795e.log rename to ranking/ctr/DeepFM/DeepFM_avazu_x4_001/DeepFM_avazu_x4_019_9e93795e.log diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_001/DeepFM_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/DeepFM/DeepFM_avazu_x4_001/DeepFM_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_001/DeepFM_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DeepFM/DeepFM_avazu_x4_001/DeepFM_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_001/DeepFM_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/DeepFM/DeepFM_avazu_x4_001/DeepFM_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_001/DeepFM_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/DeepFM/DeepFM_avazu_x4_001/DeepFM_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_001/README.md b/ranking/ctr/DeepFM/DeepFM_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_001/README.md rename to ranking/ctr/DeepFM/DeepFM_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_001/environments.txt b/ranking/ctr/DeepFM/DeepFM_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_001/environments.txt rename to ranking/ctr/DeepFM/DeepFM_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_001/results.csv b/ranking/ctr/DeepFM/DeepFM_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_001/results.csv rename to ranking/ctr/DeepFM/DeepFM_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_001/run_expid.py b/ranking/ctr/DeepFM/DeepFM_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_001/run_expid.py rename to ranking/ctr/DeepFM/DeepFM_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_002/DeepFM_avazu_x4_003_f37a42a1.log b/ranking/ctr/DeepFM/DeepFM_avazu_x4_002/DeepFM_avazu_x4_003_f37a42a1.log similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_002/DeepFM_avazu_x4_003_f37a42a1.log rename to ranking/ctr/DeepFM/DeepFM_avazu_x4_002/DeepFM_avazu_x4_003_f37a42a1.log diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_002/DeepFM_avazu_x4_tuner_config_07/dataset_config.yaml b/ranking/ctr/DeepFM/DeepFM_avazu_x4_002/DeepFM_avazu_x4_tuner_config_07/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_002/DeepFM_avazu_x4_tuner_config_07/dataset_config.yaml rename to ranking/ctr/DeepFM/DeepFM_avazu_x4_002/DeepFM_avazu_x4_tuner_config_07/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_002/DeepFM_avazu_x4_tuner_config_07/model_config.yaml b/ranking/ctr/DeepFM/DeepFM_avazu_x4_002/DeepFM_avazu_x4_tuner_config_07/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_002/DeepFM_avazu_x4_tuner_config_07/model_config.yaml rename to ranking/ctr/DeepFM/DeepFM_avazu_x4_002/DeepFM_avazu_x4_tuner_config_07/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_002/README.md b/ranking/ctr/DeepFM/DeepFM_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_002/README.md rename to ranking/ctr/DeepFM/DeepFM_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_002/environments.txt b/ranking/ctr/DeepFM/DeepFM_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_002/environments.txt rename to ranking/ctr/DeepFM/DeepFM_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_002/results.csv b/ranking/ctr/DeepFM/DeepFM_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_002/results.csv rename to ranking/ctr/DeepFM/DeepFM_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_002/run_expid.py b/ranking/ctr/DeepFM/DeepFM_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_avazu_x4_002/run_expid.py rename to ranking/ctr/DeepFM/DeepFM_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x1/DeepFM_criteo_x1_001_4b788fed.log b/ranking/ctr/DeepFM/DeepFM_criteo_x1/DeepFM_criteo_x1_001_4b788fed.log similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x1/DeepFM_criteo_x1_001_4b788fed.log rename to ranking/ctr/DeepFM/DeepFM_criteo_x1/DeepFM_criteo_x1_001_4b788fed.log diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x1/DeepFM_criteo_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DeepFM/DeepFM_criteo_x1/DeepFM_criteo_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x1/DeepFM_criteo_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DeepFM/DeepFM_criteo_x1/DeepFM_criteo_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x1/DeepFM_criteo_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DeepFM/DeepFM_criteo_x1/DeepFM_criteo_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x1/DeepFM_criteo_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DeepFM/DeepFM_criteo_x1/DeepFM_criteo_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x1/README.md b/ranking/ctr/DeepFM/DeepFM_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x1/README.md rename to ranking/ctr/DeepFM/DeepFM_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x1/environments.txt b/ranking/ctr/DeepFM/DeepFM_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x1/environments.txt rename to ranking/ctr/DeepFM/DeepFM_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x1/results.csv b/ranking/ctr/DeepFM/DeepFM_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x1/results.csv rename to ranking/ctr/DeepFM/DeepFM_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x1/run_expid.py b/ranking/ctr/DeepFM/DeepFM_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x1/run_expid.py rename to ranking/ctr/DeepFM/DeepFM_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_001/DeepFM_criteo_x4_024_626165ea.log b/ranking/ctr/DeepFM/DeepFM_criteo_x4_001/DeepFM_criteo_x4_024_626165ea.log similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_001/DeepFM_criteo_x4_024_626165ea.log rename to ranking/ctr/DeepFM/DeepFM_criteo_x4_001/DeepFM_criteo_x4_024_626165ea.log diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_001/DeepFM_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/DeepFM/DeepFM_criteo_x4_001/DeepFM_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_001/DeepFM_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DeepFM/DeepFM_criteo_x4_001/DeepFM_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_001/DeepFM_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/DeepFM/DeepFM_criteo_x4_001/DeepFM_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_001/DeepFM_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/DeepFM/DeepFM_criteo_x4_001/DeepFM_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_001/README.md b/ranking/ctr/DeepFM/DeepFM_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_001/README.md rename to ranking/ctr/DeepFM/DeepFM_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_001/environments.txt b/ranking/ctr/DeepFM/DeepFM_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_001/environments.txt rename to ranking/ctr/DeepFM/DeepFM_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_001/results.csv b/ranking/ctr/DeepFM/DeepFM_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_001/results.csv rename to ranking/ctr/DeepFM/DeepFM_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_001/run_expid.py b/ranking/ctr/DeepFM/DeepFM_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_001/run_expid.py rename to ranking/ctr/DeepFM/DeepFM_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_002/DeepFM_criteo_x4_033_eb777c4c.log b/ranking/ctr/DeepFM/DeepFM_criteo_x4_002/DeepFM_criteo_x4_033_eb777c4c.log similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_002/DeepFM_criteo_x4_033_eb777c4c.log rename to ranking/ctr/DeepFM/DeepFM_criteo_x4_002/DeepFM_criteo_x4_033_eb777c4c.log diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_002/DeepFM_criteo_x4_tuner_config_03/dataset_config.yaml b/ranking/ctr/DeepFM/DeepFM_criteo_x4_002/DeepFM_criteo_x4_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_002/DeepFM_criteo_x4_tuner_config_03/dataset_config.yaml rename to ranking/ctr/DeepFM/DeepFM_criteo_x4_002/DeepFM_criteo_x4_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_002/DeepFM_criteo_x4_tuner_config_03/model_config.yaml b/ranking/ctr/DeepFM/DeepFM_criteo_x4_002/DeepFM_criteo_x4_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_002/DeepFM_criteo_x4_tuner_config_03/model_config.yaml rename to ranking/ctr/DeepFM/DeepFM_criteo_x4_002/DeepFM_criteo_x4_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_002/README.md b/ranking/ctr/DeepFM/DeepFM_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_002/README.md rename to ranking/ctr/DeepFM/DeepFM_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_002/environments.txt b/ranking/ctr/DeepFM/DeepFM_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_002/environments.txt rename to ranking/ctr/DeepFM/DeepFM_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_002/results.csv b/ranking/ctr/DeepFM/DeepFM_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_002/results.csv rename to ranking/ctr/DeepFM/DeepFM_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_002/run_expid.py b/ranking/ctr/DeepFM/DeepFM_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_criteo_x4_002/run_expid.py rename to ranking/ctr/DeepFM/DeepFM_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_frappe_x1/DeepFM_frappe_x1_001_4ae3a56e.log b/ranking/ctr/DeepFM/DeepFM_frappe_x1/DeepFM_frappe_x1_001_4ae3a56e.log similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_frappe_x1/DeepFM_frappe_x1_001_4ae3a56e.log rename to ranking/ctr/DeepFM/DeepFM_frappe_x1/DeepFM_frappe_x1_001_4ae3a56e.log diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_frappe_x1/DeepFM_frappe_x1_tuner_config_02.csv b/ranking/ctr/DeepFM/DeepFM_frappe_x1/DeepFM_frappe_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_frappe_x1/DeepFM_frappe_x1_tuner_config_02.csv rename to ranking/ctr/DeepFM/DeepFM_frappe_x1/DeepFM_frappe_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_frappe_x1/DeepFM_frappe_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/DeepFM/DeepFM_frappe_x1/DeepFM_frappe_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_frappe_x1/DeepFM_frappe_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DeepFM/DeepFM_frappe_x1/DeepFM_frappe_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_frappe_x1/DeepFM_frappe_x1_tuner_config_02/model_config.yaml b/ranking/ctr/DeepFM/DeepFM_frappe_x1/DeepFM_frappe_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_frappe_x1/DeepFM_frappe_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/DeepFM/DeepFM_frappe_x1/DeepFM_frappe_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_frappe_x1/README.md b/ranking/ctr/DeepFM/DeepFM_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_frappe_x1/README.md rename to ranking/ctr/DeepFM/DeepFM_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_frappe_x1/environments.txt b/ranking/ctr/DeepFM/DeepFM_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_frappe_x1/environments.txt rename to ranking/ctr/DeepFM/DeepFM_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_frappe_x1/results.csv b/ranking/ctr/DeepFM/DeepFM_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_frappe_x1/results.csv rename to ranking/ctr/DeepFM/DeepFM_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_frappe_x1/run_expid.py b/ranking/ctr/DeepFM/DeepFM_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_frappe_x1/run_expid.py rename to ranking/ctr/DeepFM/DeepFM_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_kkbox_x1/DeepFM_kkbox_x1_005_32cb2ca8.log b/ranking/ctr/DeepFM/DeepFM_kkbox_x1/DeepFM_kkbox_x1_005_32cb2ca8.log similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_kkbox_x1/DeepFM_kkbox_x1_005_32cb2ca8.log rename to ranking/ctr/DeepFM/DeepFM_kkbox_x1/DeepFM_kkbox_x1_005_32cb2ca8.log diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_kkbox_x1/DeepFM_kkbox_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/DeepFM/DeepFM_kkbox_x1/DeepFM_kkbox_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_kkbox_x1/DeepFM_kkbox_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/DeepFM/DeepFM_kkbox_x1/DeepFM_kkbox_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_kkbox_x1/DeepFM_kkbox_x1_tuner_config_03/model_config.yaml b/ranking/ctr/DeepFM/DeepFM_kkbox_x1/DeepFM_kkbox_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_kkbox_x1/DeepFM_kkbox_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/DeepFM/DeepFM_kkbox_x1/DeepFM_kkbox_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_kkbox_x1/README.md b/ranking/ctr/DeepFM/DeepFM_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_kkbox_x1/README.md rename to ranking/ctr/DeepFM/DeepFM_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_kkbox_x1/environments.txt b/ranking/ctr/DeepFM/DeepFM_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_kkbox_x1/environments.txt rename to ranking/ctr/DeepFM/DeepFM_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_kkbox_x1/results.csv b/ranking/ctr/DeepFM/DeepFM_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_kkbox_x1/results.csv rename to ranking/ctr/DeepFM/DeepFM_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_kkbox_x1/run_expid.py b/ranking/ctr/DeepFM/DeepFM_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_kkbox_x1/run_expid.py rename to ranking/ctr/DeepFM/DeepFM_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_003_a7784cdb.log b/ranking/ctr/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_003_a7784cdb.log similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_003_a7784cdb.log rename to ranking/ctr/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_003_a7784cdb.log diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_tuner_config_01.csv b/ranking/ctr/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_tuner_config_01.csv rename to ranking/ctr/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_tuner_config_01.yaml b/ranking/ctr/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_tuner_config_01.yaml rename to ranking/ctr/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DeepFM/DeepFM_kuaivideo_x1/DeepFM_kuaivideo_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_kuaivideo_x1/README.md b/ranking/ctr/DeepFM/DeepFM_kuaivideo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_kuaivideo_x1/README.md rename to ranking/ctr/DeepFM/DeepFM_kuaivideo_x1/README.md diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_kuaivideo_x1/environments.txt b/ranking/ctr/DeepFM/DeepFM_kuaivideo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_kuaivideo_x1/environments.txt rename to ranking/ctr/DeepFM/DeepFM_kuaivideo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_kuaivideo_x1/results.csv b/ranking/ctr/DeepFM/DeepFM_kuaivideo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_kuaivideo_x1/results.csv rename to ranking/ctr/DeepFM/DeepFM_kuaivideo_x1/results.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_023_bda67d29.log b/ranking/ctr/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_023_bda67d29.log similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_023_bda67d29.log rename to ranking/ctr/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_023_bda67d29.log diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_tuner_config_04.csv b/ranking/ctr/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_tuner_config_04.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_tuner_config_04.csv rename to ranking/ctr/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_tuner_config_04.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_tuner_config_04.yaml b/ranking/ctr/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_tuner_config_04.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_tuner_config_04.yaml rename to ranking/ctr/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_tuner_config_04.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_tuner_config_04/model_config.yaml b/ranking/ctr/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/DeepFM/DeepFM_microvideo1.7m_x1/DeepFM_microvideo1.7m_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_microvideo1.7m_x1/README.md b/ranking/ctr/DeepFM/DeepFM_microvideo1.7m_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_microvideo1.7m_x1/README.md rename to ranking/ctr/DeepFM/DeepFM_microvideo1.7m_x1/README.md diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_microvideo1.7m_x1/environments.txt b/ranking/ctr/DeepFM/DeepFM_microvideo1.7m_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_microvideo1.7m_x1/environments.txt rename to ranking/ctr/DeepFM/DeepFM_microvideo1.7m_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_microvideo1.7m_x1/results.csv b/ranking/ctr/DeepFM/DeepFM_microvideo1.7m_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_microvideo1.7m_x1/results.csv rename to ranking/ctr/DeepFM/DeepFM_microvideo1.7m_x1/results.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_movielenslatest_x1/DeepFM_movielenslatest_x1_005_0f6d2e8e.log b/ranking/ctr/DeepFM/DeepFM_movielenslatest_x1/DeepFM_movielenslatest_x1_005_0f6d2e8e.log similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_movielenslatest_x1/DeepFM_movielenslatest_x1_005_0f6d2e8e.log rename to ranking/ctr/DeepFM/DeepFM_movielenslatest_x1/DeepFM_movielenslatest_x1_005_0f6d2e8e.log diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_movielenslatest_x1/DeepFM_movielenslatest_x1_tuner_config_02.csv b/ranking/ctr/DeepFM/DeepFM_movielenslatest_x1/DeepFM_movielenslatest_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_movielenslatest_x1/DeepFM_movielenslatest_x1_tuner_config_02.csv rename to ranking/ctr/DeepFM/DeepFM_movielenslatest_x1/DeepFM_movielenslatest_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_movielenslatest_x1/DeepFM_movielenslatest_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/DeepFM/DeepFM_movielenslatest_x1/DeepFM_movielenslatest_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_movielenslatest_x1/DeepFM_movielenslatest_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/DeepFM/DeepFM_movielenslatest_x1/DeepFM_movielenslatest_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_movielenslatest_x1/DeepFM_movielenslatest_x1_tuner_config_02/model_config.yaml b/ranking/ctr/DeepFM/DeepFM_movielenslatest_x1/DeepFM_movielenslatest_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_movielenslatest_x1/DeepFM_movielenslatest_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/DeepFM/DeepFM_movielenslatest_x1/DeepFM_movielenslatest_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_movielenslatest_x1/README.md b/ranking/ctr/DeepFM/DeepFM_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_movielenslatest_x1/README.md rename to ranking/ctr/DeepFM/DeepFM_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_movielenslatest_x1/environments.txt b/ranking/ctr/DeepFM/DeepFM_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_movielenslatest_x1/environments.txt rename to ranking/ctr/DeepFM/DeepFM_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_movielenslatest_x1/results.csv b/ranking/ctr/DeepFM/DeepFM_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_movielenslatest_x1/results.csv rename to ranking/ctr/DeepFM/DeepFM_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_movielenslatest_x1/run_expid.py b/ranking/ctr/DeepFM/DeepFM_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_movielenslatest_x1/run_expid.py rename to ranking/ctr/DeepFM/DeepFM_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_009_afab5940.log b/ranking/ctr/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_009_afab5940.log similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_009_afab5940.log rename to ranking/ctr/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_009_afab5940.log diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_tuner_config_01.csv b/ranking/ctr/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_tuner_config_01.csv rename to ranking/ctr/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_tuner_config_01.yaml b/ranking/ctr/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_tuner_config_01.yaml rename to ranking/ctr/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DeepFM/DeepFM_taobaoad_x1/DeepFM_taobaoad_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_taobaoad_x1/README.md b/ranking/ctr/DeepFM/DeepFM_taobaoad_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_taobaoad_x1/README.md rename to ranking/ctr/DeepFM/DeepFM_taobaoad_x1/README.md diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_taobaoad_x1/environments.txt b/ranking/ctr/DeepFM/DeepFM_taobaoad_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_taobaoad_x1/environments.txt rename to ranking/ctr/DeepFM/DeepFM_taobaoad_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DeepFM/DeepFM_taobaoad_x1/results.csv b/ranking/ctr/DeepFM/DeepFM_taobaoad_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepFM/DeepFM_taobaoad_x1/results.csv rename to ranking/ctr/DeepFM/DeepFM_taobaoad_x1/results.csv diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_avazu_x1/DeepIM_avazu_x1_001_ce22770f.log b/ranking/ctr/DeepIM/DeepIM_avazu_x1/DeepIM_avazu_x1_001_ce22770f.log similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_avazu_x1/DeepIM_avazu_x1_001_ce22770f.log rename to ranking/ctr/DeepIM/DeepIM_avazu_x1/DeepIM_avazu_x1_001_ce22770f.log diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_avazu_x1/DeepIM_avazu_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/DeepIM/DeepIM_avazu_x1/DeepIM_avazu_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_avazu_x1/DeepIM_avazu_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/DeepIM/DeepIM_avazu_x1/DeepIM_avazu_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_avazu_x1/DeepIM_avazu_x1_tuner_config_03/model_config.yaml b/ranking/ctr/DeepIM/DeepIM_avazu_x1/DeepIM_avazu_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_avazu_x1/DeepIM_avazu_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/DeepIM/DeepIM_avazu_x1/DeepIM_avazu_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_avazu_x1/README.md b/ranking/ctr/DeepIM/DeepIM_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_avazu_x1/README.md rename to ranking/ctr/DeepIM/DeepIM_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_avazu_x1/environments.txt b/ranking/ctr/DeepIM/DeepIM_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_avazu_x1/environments.txt rename to ranking/ctr/DeepIM/DeepIM_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_avazu_x1/results.csv b/ranking/ctr/DeepIM/DeepIM_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_avazu_x1/results.csv rename to ranking/ctr/DeepIM/DeepIM_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_avazu_x1/run_expid.py b/ranking/ctr/DeepIM/DeepIM_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_avazu_x1/run_expid.py rename to ranking/ctr/DeepIM/DeepIM_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_criteo_x1/DeepIM_criteo_x1_001_6de9e773.log b/ranking/ctr/DeepIM/DeepIM_criteo_x1/DeepIM_criteo_x1_001_6de9e773.log similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_criteo_x1/DeepIM_criteo_x1_001_6de9e773.log rename to ranking/ctr/DeepIM/DeepIM_criteo_x1/DeepIM_criteo_x1_001_6de9e773.log diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_criteo_x1/DeepIM_criteo_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DeepIM/DeepIM_criteo_x1/DeepIM_criteo_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_criteo_x1/DeepIM_criteo_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DeepIM/DeepIM_criteo_x1/DeepIM_criteo_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_criteo_x1/DeepIM_criteo_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DeepIM/DeepIM_criteo_x1/DeepIM_criteo_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_criteo_x1/DeepIM_criteo_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DeepIM/DeepIM_criteo_x1/DeepIM_criteo_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_criteo_x1/README.md b/ranking/ctr/DeepIM/DeepIM_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_criteo_x1/README.md rename to ranking/ctr/DeepIM/DeepIM_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_criteo_x1/environments.txt b/ranking/ctr/DeepIM/DeepIM_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_criteo_x1/environments.txt rename to ranking/ctr/DeepIM/DeepIM_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_criteo_x1/results.csv b/ranking/ctr/DeepIM/DeepIM_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_criteo_x1/results.csv rename to ranking/ctr/DeepIM/DeepIM_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_criteo_x1/run_expid.py b/ranking/ctr/DeepIM/DeepIM_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_criteo_x1/run_expid.py rename to ranking/ctr/DeepIM/DeepIM_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_frappe_x1/DeepIM_frappe_x1_005_a11ff117.log b/ranking/ctr/DeepIM/DeepIM_frappe_x1/DeepIM_frappe_x1_005_a11ff117.log similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_frappe_x1/DeepIM_frappe_x1_005_a11ff117.log rename to ranking/ctr/DeepIM/DeepIM_frappe_x1/DeepIM_frappe_x1_005_a11ff117.log diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_frappe_x1/DeepIM_frappe_x1_tuner_config_01.csv b/ranking/ctr/DeepIM/DeepIM_frappe_x1/DeepIM_frappe_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_frappe_x1/DeepIM_frappe_x1_tuner_config_01.csv rename to ranking/ctr/DeepIM/DeepIM_frappe_x1/DeepIM_frappe_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_frappe_x1/DeepIM_frappe_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DeepIM/DeepIM_frappe_x1/DeepIM_frappe_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_frappe_x1/DeepIM_frappe_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DeepIM/DeepIM_frappe_x1/DeepIM_frappe_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_frappe_x1/DeepIM_frappe_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DeepIM/DeepIM_frappe_x1/DeepIM_frappe_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_frappe_x1/DeepIM_frappe_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DeepIM/DeepIM_frappe_x1/DeepIM_frappe_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_frappe_x1/README.md b/ranking/ctr/DeepIM/DeepIM_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_frappe_x1/README.md rename to ranking/ctr/DeepIM/DeepIM_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_frappe_x1/environments.txt b/ranking/ctr/DeepIM/DeepIM_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_frappe_x1/environments.txt rename to ranking/ctr/DeepIM/DeepIM_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_frappe_x1/results.csv b/ranking/ctr/DeepIM/DeepIM_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_frappe_x1/results.csv rename to ranking/ctr/DeepIM/DeepIM_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_frappe_x1/run_expid.py b/ranking/ctr/DeepIM/DeepIM_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_frappe_x1/run_expid.py rename to ranking/ctr/DeepIM/DeepIM_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_movielenslatest_x1/DeepIM_movielenslatest_x1_022_eb1c9e99.log b/ranking/ctr/DeepIM/DeepIM_movielenslatest_x1/DeepIM_movielenslatest_x1_022_eb1c9e99.log similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_movielenslatest_x1/DeepIM_movielenslatest_x1_022_eb1c9e99.log rename to ranking/ctr/DeepIM/DeepIM_movielenslatest_x1/DeepIM_movielenslatest_x1_022_eb1c9e99.log diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_movielenslatest_x1/DeepIM_movielenslatest_x1_tuner_config_01.csv b/ranking/ctr/DeepIM/DeepIM_movielenslatest_x1/DeepIM_movielenslatest_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_movielenslatest_x1/DeepIM_movielenslatest_x1_tuner_config_01.csv rename to ranking/ctr/DeepIM/DeepIM_movielenslatest_x1/DeepIM_movielenslatest_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_movielenslatest_x1/DeepIM_movielenslatest_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/DeepIM/DeepIM_movielenslatest_x1/DeepIM_movielenslatest_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_movielenslatest_x1/DeepIM_movielenslatest_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/DeepIM/DeepIM_movielenslatest_x1/DeepIM_movielenslatest_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_movielenslatest_x1/DeepIM_movielenslatest_x1_tuner_config_01/model_config.yaml b/ranking/ctr/DeepIM/DeepIM_movielenslatest_x1/DeepIM_movielenslatest_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_movielenslatest_x1/DeepIM_movielenslatest_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/DeepIM/DeepIM_movielenslatest_x1/DeepIM_movielenslatest_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_movielenslatest_x1/README.md b/ranking/ctr/DeepIM/DeepIM_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_movielenslatest_x1/README.md rename to ranking/ctr/DeepIM/DeepIM_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_movielenslatest_x1/environments.txt b/ranking/ctr/DeepIM/DeepIM_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_movielenslatest_x1/environments.txt rename to ranking/ctr/DeepIM/DeepIM_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_movielenslatest_x1/results.csv b/ranking/ctr/DeepIM/DeepIM_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_movielenslatest_x1/results.csv rename to ranking/ctr/DeepIM/DeepIM_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/DeepIM/DeepIM_movielenslatest_x1/run_expid.py b/ranking/ctr/DeepIM/DeepIM_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/DeepIM/DeepIM_movielenslatest_x1/run_expid.py rename to ranking/ctr/DeepIM/DeepIM_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/EDCN_avazu_x1_030_97df3f6c.log b/ranking/ctr/EDCN/EDCN_avazu_x1/EDCN_avazu_x1_030_97df3f6c.log similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/EDCN_avazu_x1_030_97df3f6c.log rename to ranking/ctr/EDCN/EDCN_avazu_x1/EDCN_avazu_x1_030_97df3f6c.log diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/EDCN_avazu_x1_tuner_config_03.csv b/ranking/ctr/EDCN/EDCN_avazu_x1/EDCN_avazu_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/EDCN_avazu_x1_tuner_config_03.csv rename to ranking/ctr/EDCN/EDCN_avazu_x1/EDCN_avazu_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/EDCN_avazu_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/EDCN/EDCN_avazu_x1/EDCN_avazu_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/EDCN_avazu_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/EDCN/EDCN_avazu_x1/EDCN_avazu_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/EDCN_avazu_x1_tuner_config_03/model_config.yaml b/ranking/ctr/EDCN/EDCN_avazu_x1/EDCN_avazu_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/EDCN_avazu_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/EDCN/EDCN_avazu_x1/EDCN_avazu_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/README.md b/ranking/ctr/EDCN/EDCN_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/README.md rename to ranking/ctr/EDCN/EDCN_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/environments.txt b/ranking/ctr/EDCN/EDCN_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/environments.txt rename to ranking/ctr/EDCN/EDCN_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/fuxictr_version.py b/ranking/ctr/EDCN/EDCN_avazu_x1/fuxictr_version.py similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/fuxictr_version.py rename to ranking/ctr/EDCN/EDCN_avazu_x1/fuxictr_version.py diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/results.csv b/ranking/ctr/EDCN/EDCN_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/results.csv rename to ranking/ctr/EDCN/EDCN_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/run_expid.py b/ranking/ctr/EDCN/EDCN_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_avazu_x1/run_expid.py rename to ranking/ctr/EDCN/EDCN_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/EDCN_criteo_x1_004_4023a363.log b/ranking/ctr/EDCN/EDCN_criteo_x1/EDCN_criteo_x1_004_4023a363.log similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/EDCN_criteo_x1_004_4023a363.log rename to ranking/ctr/EDCN/EDCN_criteo_x1/EDCN_criteo_x1_004_4023a363.log diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/EDCN_criteo_x1_tuner_config_02.csv b/ranking/ctr/EDCN/EDCN_criteo_x1/EDCN_criteo_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/EDCN_criteo_x1_tuner_config_02.csv rename to ranking/ctr/EDCN/EDCN_criteo_x1/EDCN_criteo_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/EDCN_criteo_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/EDCN/EDCN_criteo_x1/EDCN_criteo_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/EDCN_criteo_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/EDCN/EDCN_criteo_x1/EDCN_criteo_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/EDCN_criteo_x1_tuner_config_02/model_config.yaml b/ranking/ctr/EDCN/EDCN_criteo_x1/EDCN_criteo_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/EDCN_criteo_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/EDCN/EDCN_criteo_x1/EDCN_criteo_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/README.md b/ranking/ctr/EDCN/EDCN_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/README.md rename to ranking/ctr/EDCN/EDCN_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/environments.txt b/ranking/ctr/EDCN/EDCN_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/environments.txt rename to ranking/ctr/EDCN/EDCN_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/fuxictr_version.py b/ranking/ctr/EDCN/EDCN_criteo_x1/fuxictr_version.py similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/fuxictr_version.py rename to ranking/ctr/EDCN/EDCN_criteo_x1/fuxictr_version.py diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/results.csv b/ranking/ctr/EDCN/EDCN_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/results.csv rename to ranking/ctr/EDCN/EDCN_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/run_expid.py b/ranking/ctr/EDCN/EDCN_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_criteo_x1/run_expid.py rename to ranking/ctr/EDCN/EDCN_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/EDCN_frappe_x1_006_5e8b9617.log b/ranking/ctr/EDCN/EDCN_frappe_x1/EDCN_frappe_x1_006_5e8b9617.log similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/EDCN_frappe_x1_006_5e8b9617.log rename to ranking/ctr/EDCN/EDCN_frappe_x1/EDCN_frappe_x1_006_5e8b9617.log diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/EDCN_frappe_x1_tuner_config_02.csv b/ranking/ctr/EDCN/EDCN_frappe_x1/EDCN_frappe_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/EDCN_frappe_x1_tuner_config_02.csv rename to ranking/ctr/EDCN/EDCN_frappe_x1/EDCN_frappe_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/EDCN_frappe_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/EDCN/EDCN_frappe_x1/EDCN_frappe_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/EDCN_frappe_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/EDCN/EDCN_frappe_x1/EDCN_frappe_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/EDCN_frappe_x1_tuner_config_02/model_config.yaml b/ranking/ctr/EDCN/EDCN_frappe_x1/EDCN_frappe_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/EDCN_frappe_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/EDCN/EDCN_frappe_x1/EDCN_frappe_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/README.md b/ranking/ctr/EDCN/EDCN_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/README.md rename to ranking/ctr/EDCN/EDCN_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/environments.txt b/ranking/ctr/EDCN/EDCN_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/environments.txt rename to ranking/ctr/EDCN/EDCN_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/fuxictr_version.py b/ranking/ctr/EDCN/EDCN_frappe_x1/fuxictr_version.py similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/fuxictr_version.py rename to ranking/ctr/EDCN/EDCN_frappe_x1/fuxictr_version.py diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/results.csv b/ranking/ctr/EDCN/EDCN_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/results.csv rename to ranking/ctr/EDCN/EDCN_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/run_expid.py b/ranking/ctr/EDCN/EDCN_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_frappe_x1/run_expid.py rename to ranking/ctr/EDCN/EDCN_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/EDCN_movielenslatest_x1_003_b42bf948.log b/ranking/ctr/EDCN/EDCN_movielenslatest_x1/EDCN_movielenslatest_x1_003_b42bf948.log similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/EDCN_movielenslatest_x1_003_b42bf948.log rename to ranking/ctr/EDCN/EDCN_movielenslatest_x1/EDCN_movielenslatest_x1_003_b42bf948.log diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/EDCN_movielenslatest_x1_tuner_config_02.csv b/ranking/ctr/EDCN/EDCN_movielenslatest_x1/EDCN_movielenslatest_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/EDCN_movielenslatest_x1_tuner_config_02.csv rename to ranking/ctr/EDCN/EDCN_movielenslatest_x1/EDCN_movielenslatest_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/EDCN_movielenslatest_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/EDCN/EDCN_movielenslatest_x1/EDCN_movielenslatest_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/EDCN_movielenslatest_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/EDCN/EDCN_movielenslatest_x1/EDCN_movielenslatest_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/EDCN_movielenslatest_x1_tuner_config_02/model_config.yaml b/ranking/ctr/EDCN/EDCN_movielenslatest_x1/EDCN_movielenslatest_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/EDCN_movielenslatest_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/EDCN/EDCN_movielenslatest_x1/EDCN_movielenslatest_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/README.md b/ranking/ctr/EDCN/EDCN_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/README.md rename to ranking/ctr/EDCN/EDCN_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/environments.txt b/ranking/ctr/EDCN/EDCN_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/environments.txt rename to ranking/ctr/EDCN/EDCN_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/fuxictr_version.py b/ranking/ctr/EDCN/EDCN_movielenslatest_x1/fuxictr_version.py similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/fuxictr_version.py rename to ranking/ctr/EDCN/EDCN_movielenslatest_x1/fuxictr_version.py diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/results.csv b/ranking/ctr/EDCN/EDCN_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/results.csv rename to ranking/ctr/EDCN/EDCN_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/run_expid.py b/ranking/ctr/EDCN/EDCN_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/EDCN/EDCN_movielenslatest_x1/run_expid.py rename to ranking/ctr/EDCN/EDCN_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x1/FFM_avazu_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FFM/FFM_avazu_x1/FFM_avazu_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x1/FFM_avazu_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FFM/FFM_avazu_x1/FFM_avazu_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x1/FFM_avazu_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FFM/FFM_avazu_x1/FFM_avazu_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x1/FFM_avazu_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FFM/FFM_avazu_x1/FFM_avazu_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x1/FFMv2_avazu_x1_003_2c82d4bd.log b/ranking/ctr/FFM/FFM_avazu_x1/FFMv2_avazu_x1_003_2c82d4bd.log similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x1/FFMv2_avazu_x1_003_2c82d4bd.log rename to ranking/ctr/FFM/FFM_avazu_x1/FFMv2_avazu_x1_003_2c82d4bd.log diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x1/README.md b/ranking/ctr/FFM/FFM_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x1/README.md rename to ranking/ctr/FFM/FFM_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x1/environments.txt b/ranking/ctr/FFM/FFM_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x1/environments.txt rename to ranking/ctr/FFM/FFM_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x1/results.csv b/ranking/ctr/FFM/FFM_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x1/results.csv rename to ranking/ctr/FFM/FFM_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x1/run_expid.py b/ranking/ctr/FFM/FFM_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x1/run_expid.py rename to ranking/ctr/FFM/FFM_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x4_001/FFM_avazu_x4_003_792ee3b7.log b/ranking/ctr/FFM/FFM_avazu_x4_001/FFM_avazu_x4_003_792ee3b7.log similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x4_001/FFM_avazu_x4_003_792ee3b7.log rename to ranking/ctr/FFM/FFM_avazu_x4_001/FFM_avazu_x4_003_792ee3b7.log diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x4_001/FFM_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/FFM/FFM_avazu_x4_001/FFM_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x4_001/FFM_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FFM/FFM_avazu_x4_001/FFM_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x4_001/FFM_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/FFM/FFM_avazu_x4_001/FFM_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x4_001/FFM_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/FFM/FFM_avazu_x4_001/FFM_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x4_001/README.md b/ranking/ctr/FFM/FFM_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x4_001/README.md rename to ranking/ctr/FFM/FFM_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x4_001/environments.txt b/ranking/ctr/FFM/FFM_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x4_001/environments.txt rename to ranking/ctr/FFM/FFM_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x4_001/results.csv b/ranking/ctr/FFM/FFM_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x4_001/results.csv rename to ranking/ctr/FFM/FFM_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x4_001/run_expid.py b/ranking/ctr/FFM/FFM_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x4_001/run_expid.py rename to ranking/ctr/FFM/FFM_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x4_002/FFM_avazu_x4_004_0fd87979.log b/ranking/ctr/FFM/FFM_avazu_x4_002/FFM_avazu_x4_004_0fd87979.log similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x4_002/FFM_avazu_x4_004_0fd87979.log rename to ranking/ctr/FFM/FFM_avazu_x4_002/FFM_avazu_x4_004_0fd87979.log diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x4_002/FFM_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/FFM/FFM_avazu_x4_002/FFM_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x4_002/FFM_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FFM/FFM_avazu_x4_002/FFM_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x4_002/FFM_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/FFM/FFM_avazu_x4_002/FFM_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x4_002/FFM_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/FFM/FFM_avazu_x4_002/FFM_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x4_002/README.md b/ranking/ctr/FFM/FFM_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x4_002/README.md rename to ranking/ctr/FFM/FFM_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x4_002/environments.txt b/ranking/ctr/FFM/FFM_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x4_002/environments.txt rename to ranking/ctr/FFM/FFM_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x4_002/results.csv b/ranking/ctr/FFM/FFM_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x4_002/results.csv rename to ranking/ctr/FFM/FFM_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/FFM/FFM_avazu_x4_002/run_expid.py b/ranking/ctr/FFM/FFM_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_avazu_x4_002/run_expid.py rename to ranking/ctr/FFM/FFM_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x1/FFM_criteo_x1_tuner_config_01.csv b/ranking/ctr/FFM/FFM_criteo_x1/FFM_criteo_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x1/FFM_criteo_x1_tuner_config_01.csv rename to ranking/ctr/FFM/FFM_criteo_x1/FFM_criteo_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x1/FFM_criteo_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FFM/FFM_criteo_x1/FFM_criteo_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x1/FFM_criteo_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FFM/FFM_criteo_x1/FFM_criteo_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x1/FFM_criteo_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FFM/FFM_criteo_x1/FFM_criteo_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x1/FFM_criteo_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FFM/FFM_criteo_x1/FFM_criteo_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x1/FFMv2_criteo_x1_004_9c79983e.log b/ranking/ctr/FFM/FFM_criteo_x1/FFMv2_criteo_x1_004_9c79983e.log similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x1/FFMv2_criteo_x1_004_9c79983e.log rename to ranking/ctr/FFM/FFM_criteo_x1/FFMv2_criteo_x1_004_9c79983e.log diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x1/README.md b/ranking/ctr/FFM/FFM_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x1/README.md rename to ranking/ctr/FFM/FFM_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x1/environments.txt b/ranking/ctr/FFM/FFM_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x1/environments.txt rename to ranking/ctr/FFM/FFM_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x1/results.csv b/ranking/ctr/FFM/FFM_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x1/results.csv rename to ranking/ctr/FFM/FFM_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x1/run_expid.py b/ranking/ctr/FFM/FFM_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x1/run_expid.py rename to ranking/ctr/FFM/FFM_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x4_001/FFM_criteo_x4_010_d090ef3d.log b/ranking/ctr/FFM/FFM_criteo_x4_001/FFM_criteo_x4_010_d090ef3d.log similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x4_001/FFM_criteo_x4_010_d090ef3d.log rename to ranking/ctr/FFM/FFM_criteo_x4_001/FFM_criteo_x4_010_d090ef3d.log diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x4_001/FFM_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/FFM/FFM_criteo_x4_001/FFM_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x4_001/FFM_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FFM/FFM_criteo_x4_001/FFM_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x4_001/FFM_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/FFM/FFM_criteo_x4_001/FFM_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x4_001/FFM_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/FFM/FFM_criteo_x4_001/FFM_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x4_001/README.md b/ranking/ctr/FFM/FFM_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x4_001/README.md rename to ranking/ctr/FFM/FFM_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x4_001/environments.txt b/ranking/ctr/FFM/FFM_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x4_001/environments.txt rename to ranking/ctr/FFM/FFM_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x4_001/results.csv b/ranking/ctr/FFM/FFM_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x4_001/results.csv rename to ranking/ctr/FFM/FFM_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x4_001/run_expid.py b/ranking/ctr/FFM/FFM_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x4_001/run_expid.py rename to ranking/ctr/FFM/FFM_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x4_002/FFM_criteo_x4_023_0b654e65.log b/ranking/ctr/FFM/FFM_criteo_x4_002/FFM_criteo_x4_023_0b654e65.log similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x4_002/FFM_criteo_x4_023_0b654e65.log rename to ranking/ctr/FFM/FFM_criteo_x4_002/FFM_criteo_x4_023_0b654e65.log diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x4_002/FFM_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/FFM/FFM_criteo_x4_002/FFM_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x4_002/FFM_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FFM/FFM_criteo_x4_002/FFM_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x4_002/FFM_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/FFM/FFM_criteo_x4_002/FFM_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x4_002/FFM_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/FFM/FFM_criteo_x4_002/FFM_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x4_002/README.md b/ranking/ctr/FFM/FFM_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x4_002/README.md rename to ranking/ctr/FFM/FFM_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x4_002/environments.txt b/ranking/ctr/FFM/FFM_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x4_002/environments.txt rename to ranking/ctr/FFM/FFM_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x4_002/results.csv b/ranking/ctr/FFM/FFM_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x4_002/results.csv rename to ranking/ctr/FFM/FFM_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/FFM/FFM_criteo_x4_002/run_expid.py b/ranking/ctr/FFM/FFM_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_criteo_x4_002/run_expid.py rename to ranking/ctr/FFM/FFM_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/FFM/FFM_frappe_x1/FFM_frappe_x1_007_b8a83e47.log b/ranking/ctr/FFM/FFM_frappe_x1/FFM_frappe_x1_007_b8a83e47.log similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_frappe_x1/FFM_frappe_x1_007_b8a83e47.log rename to ranking/ctr/FFM/FFM_frappe_x1/FFM_frappe_x1_007_b8a83e47.log diff --git a/ctr_prediction/benchmarks/FFM/FFM_frappe_x1/FFM_frappe_x1_tuner_config_01.csv b/ranking/ctr/FFM/FFM_frappe_x1/FFM_frappe_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_frappe_x1/FFM_frappe_x1_tuner_config_01.csv rename to ranking/ctr/FFM/FFM_frappe_x1/FFM_frappe_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/FFM/FFM_frappe_x1/FFM_frappe_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FFM/FFM_frappe_x1/FFM_frappe_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_frappe_x1/FFM_frappe_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FFM/FFM_frappe_x1/FFM_frappe_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_frappe_x1/FFM_frappe_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FFM/FFM_frappe_x1/FFM_frappe_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_frappe_x1/FFM_frappe_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FFM/FFM_frappe_x1/FFM_frappe_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_frappe_x1/README.md b/ranking/ctr/FFM/FFM_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_frappe_x1/README.md rename to ranking/ctr/FFM/FFM_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/FFM/FFM_frappe_x1/environments.txt b/ranking/ctr/FFM/FFM_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_frappe_x1/environments.txt rename to ranking/ctr/FFM/FFM_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FFM/FFM_frappe_x1/results.csv b/ranking/ctr/FFM/FFM_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_frappe_x1/results.csv rename to ranking/ctr/FFM/FFM_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/FFM/FFM_frappe_x1/run_expid.py b/ranking/ctr/FFM/FFM_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_frappe_x1/run_expid.py rename to ranking/ctr/FFM/FFM_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FFM/FFM_kkbox_x1/FFM_kkbox_x1_017_852fd9b3.log b/ranking/ctr/FFM/FFM_kkbox_x1/FFM_kkbox_x1_017_852fd9b3.log similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_kkbox_x1/FFM_kkbox_x1_017_852fd9b3.log rename to ranking/ctr/FFM/FFM_kkbox_x1/FFM_kkbox_x1_017_852fd9b3.log diff --git a/ctr_prediction/benchmarks/FFM/FFM_kkbox_x1/FFM_kkbox_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/FFM/FFM_kkbox_x1/FFM_kkbox_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_kkbox_x1/FFM_kkbox_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/FFM/FFM_kkbox_x1/FFM_kkbox_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_kkbox_x1/FFM_kkbox_x1_tuner_config_03/model_config.yaml b/ranking/ctr/FFM/FFM_kkbox_x1/FFM_kkbox_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_kkbox_x1/FFM_kkbox_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/FFM/FFM_kkbox_x1/FFM_kkbox_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_kkbox_x1/README.md b/ranking/ctr/FFM/FFM_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_kkbox_x1/README.md rename to ranking/ctr/FFM/FFM_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/FFM/FFM_kkbox_x1/environments.txt b/ranking/ctr/FFM/FFM_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_kkbox_x1/environments.txt rename to ranking/ctr/FFM/FFM_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FFM/FFM_kkbox_x1/results.csv b/ranking/ctr/FFM/FFM_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_kkbox_x1/results.csv rename to ranking/ctr/FFM/FFM_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/FFM/FFM_kkbox_x1/run_expid.py b/ranking/ctr/FFM/FFM_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_kkbox_x1/run_expid.py rename to ranking/ctr/FFM/FFM_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FFM/FFM_movielenslatest_x1/FFM_movielenslatest_x1_001_79d12e82.log b/ranking/ctr/FFM/FFM_movielenslatest_x1/FFM_movielenslatest_x1_001_79d12e82.log similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_movielenslatest_x1/FFM_movielenslatest_x1_001_79d12e82.log rename to ranking/ctr/FFM/FFM_movielenslatest_x1/FFM_movielenslatest_x1_001_79d12e82.log diff --git a/ctr_prediction/benchmarks/FFM/FFM_movielenslatest_x1/FFM_movielenslatest_x1_tuner_config_03.csv b/ranking/ctr/FFM/FFM_movielenslatest_x1/FFM_movielenslatest_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_movielenslatest_x1/FFM_movielenslatest_x1_tuner_config_03.csv rename to ranking/ctr/FFM/FFM_movielenslatest_x1/FFM_movielenslatest_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/FFM/FFM_movielenslatest_x1/FFM_movielenslatest_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/FFM/FFM_movielenslatest_x1/FFM_movielenslatest_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_movielenslatest_x1/FFM_movielenslatest_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/FFM/FFM_movielenslatest_x1/FFM_movielenslatest_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_movielenslatest_x1/FFM_movielenslatest_x1_tuner_config_03/model_config.yaml b/ranking/ctr/FFM/FFM_movielenslatest_x1/FFM_movielenslatest_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_movielenslatest_x1/FFM_movielenslatest_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/FFM/FFM_movielenslatest_x1/FFM_movielenslatest_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/FFM/FFM_movielenslatest_x1/README.md b/ranking/ctr/FFM/FFM_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_movielenslatest_x1/README.md rename to ranking/ctr/FFM/FFM_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/FFM/FFM_movielenslatest_x1/environments.txt b/ranking/ctr/FFM/FFM_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_movielenslatest_x1/environments.txt rename to ranking/ctr/FFM/FFM_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FFM/FFM_movielenslatest_x1/results.csv b/ranking/ctr/FFM/FFM_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_movielenslatest_x1/results.csv rename to ranking/ctr/FFM/FFM_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/FFM/FFM_movielenslatest_x1/run_expid.py b/ranking/ctr/FFM/FFM_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FFM/FFM_movielenslatest_x1/run_expid.py rename to ranking/ctr/FFM/FFM_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_001/FGCNN_avazu_x4_010_06d6ae32.log b/ranking/ctr/FGCNN/FGCNN_avazu_x4_001/FGCNN_avazu_x4_010_06d6ae32.log similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_001/FGCNN_avazu_x4_010_06d6ae32.log rename to ranking/ctr/FGCNN/FGCNN_avazu_x4_001/FGCNN_avazu_x4_010_06d6ae32.log diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_001/FGCNN_avazu_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/FGCNN/FGCNN_avazu_x4_001/FGCNN_avazu_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_001/FGCNN_avazu_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FGCNN/FGCNN_avazu_x4_001/FGCNN_avazu_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_001/FGCNN_avazu_x4_tuner_config_02/model_config.yaml b/ranking/ctr/FGCNN/FGCNN_avazu_x4_001/FGCNN_avazu_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_001/FGCNN_avazu_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/FGCNN/FGCNN_avazu_x4_001/FGCNN_avazu_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_001/README.md b/ranking/ctr/FGCNN/FGCNN_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_001/README.md rename to ranking/ctr/FGCNN/FGCNN_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_001/environments.txt b/ranking/ctr/FGCNN/FGCNN_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_001/environments.txt rename to ranking/ctr/FGCNN/FGCNN_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_001/results.csv b/ranking/ctr/FGCNN/FGCNN_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_001/results.csv rename to ranking/ctr/FGCNN/FGCNN_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_001/run_expid.py b/ranking/ctr/FGCNN/FGCNN_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_001/run_expid.py rename to ranking/ctr/FGCNN/FGCNN_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_002/FGCNN_avazu_x4_043_958cbd01.log b/ranking/ctr/FGCNN/FGCNN_avazu_x4_002/FGCNN_avazu_x4_043_958cbd01.log similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_002/FGCNN_avazu_x4_043_958cbd01.log rename to ranking/ctr/FGCNN/FGCNN_avazu_x4_002/FGCNN_avazu_x4_043_958cbd01.log diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_002/FGCNN_avazu_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/FGCNN/FGCNN_avazu_x4_002/FGCNN_avazu_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_002/FGCNN_avazu_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FGCNN/FGCNN_avazu_x4_002/FGCNN_avazu_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_002/FGCNN_avazu_x4_tuner_config_02/model_config.yaml b/ranking/ctr/FGCNN/FGCNN_avazu_x4_002/FGCNN_avazu_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_002/FGCNN_avazu_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/FGCNN/FGCNN_avazu_x4_002/FGCNN_avazu_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_002/README.md b/ranking/ctr/FGCNN/FGCNN_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_002/README.md rename to ranking/ctr/FGCNN/FGCNN_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_002/environments.txt b/ranking/ctr/FGCNN/FGCNN_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_002/environments.txt rename to ranking/ctr/FGCNN/FGCNN_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_002/results.csv b/ranking/ctr/FGCNN/FGCNN_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_002/results.csv rename to ranking/ctr/FGCNN/FGCNN_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_002/run_expid.py b/ranking/ctr/FGCNN/FGCNN_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_avazu_x4_002/run_expid.py rename to ranking/ctr/FGCNN/FGCNN_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_001/FGCNN_criteo_x4_001_4832442e.log b/ranking/ctr/FGCNN/FGCNN_criteo_x4_001/FGCNN_criteo_x4_001_4832442e.log similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_001/FGCNN_criteo_x4_001_4832442e.log rename to ranking/ctr/FGCNN/FGCNN_criteo_x4_001/FGCNN_criteo_x4_001_4832442e.log diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_001/FGCNN_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/FGCNN/FGCNN_criteo_x4_001/FGCNN_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_001/FGCNN_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FGCNN/FGCNN_criteo_x4_001/FGCNN_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_001/FGCNN_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/FGCNN/FGCNN_criteo_x4_001/FGCNN_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_001/FGCNN_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/FGCNN/FGCNN_criteo_x4_001/FGCNN_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_001/README.md b/ranking/ctr/FGCNN/FGCNN_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_001/README.md rename to ranking/ctr/FGCNN/FGCNN_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_001/environments.txt b/ranking/ctr/FGCNN/FGCNN_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_001/environments.txt rename to ranking/ctr/FGCNN/FGCNN_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_001/results.csv b/ranking/ctr/FGCNN/FGCNN_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_001/results.csv rename to ranking/ctr/FGCNN/FGCNN_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_002/FGCNN_criteo_x4_004_50ff06b8.log b/ranking/ctr/FGCNN/FGCNN_criteo_x4_002/FGCNN_criteo_x4_004_50ff06b8.log similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_002/FGCNN_criteo_x4_004_50ff06b8.log rename to ranking/ctr/FGCNN/FGCNN_criteo_x4_002/FGCNN_criteo_x4_004_50ff06b8.log diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_002/FGCNN_criteo_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/FGCNN/FGCNN_criteo_x4_002/FGCNN_criteo_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_002/FGCNN_criteo_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FGCNN/FGCNN_criteo_x4_002/FGCNN_criteo_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_002/FGCNN_criteo_x4_tuner_config_02/model_config.yaml b/ranking/ctr/FGCNN/FGCNN_criteo_x4_002/FGCNN_criteo_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_002/FGCNN_criteo_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/FGCNN/FGCNN_criteo_x4_002/FGCNN_criteo_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_002/README.md b/ranking/ctr/FGCNN/FGCNN_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_002/README.md rename to ranking/ctr/FGCNN/FGCNN_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_002/environments.txt b/ranking/ctr/FGCNN/FGCNN_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_002/environments.txt rename to ranking/ctr/FGCNN/FGCNN_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_002/results.csv b/ranking/ctr/FGCNN/FGCNN_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_002/results.csv rename to ranking/ctr/FGCNN/FGCNN_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_002/run_expid.py b/ranking/ctr/FGCNN/FGCNN_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_criteo_x4_002/run_expid.py rename to ranking/ctr/FGCNN/FGCNN_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_kkbox_x1/FGCNN_kkbox_x1_014_48888fb8.log b/ranking/ctr/FGCNN/FGCNN_kkbox_x1/FGCNN_kkbox_x1_014_48888fb8.log similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_kkbox_x1/FGCNN_kkbox_x1_014_48888fb8.log rename to ranking/ctr/FGCNN/FGCNN_kkbox_x1/FGCNN_kkbox_x1_014_48888fb8.log diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_kkbox_x1/FGCNN_kkbox_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/FGCNN/FGCNN_kkbox_x1/FGCNN_kkbox_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_kkbox_x1/FGCNN_kkbox_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FGCNN/FGCNN_kkbox_x1/FGCNN_kkbox_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_kkbox_x1/FGCNN_kkbox_x1_tuner_config_02/model_config.yaml b/ranking/ctr/FGCNN/FGCNN_kkbox_x1/FGCNN_kkbox_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_kkbox_x1/FGCNN_kkbox_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/FGCNN/FGCNN_kkbox_x1/FGCNN_kkbox_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_kkbox_x1/README.md b/ranking/ctr/FGCNN/FGCNN_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_kkbox_x1/README.md rename to ranking/ctr/FGCNN/FGCNN_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_kkbox_x1/environments.txt b/ranking/ctr/FGCNN/FGCNN_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_kkbox_x1/environments.txt rename to ranking/ctr/FGCNN/FGCNN_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_kkbox_x1/results.csv b/ranking/ctr/FGCNN/FGCNN_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_kkbox_x1/results.csv rename to ranking/ctr/FGCNN/FGCNN_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/FGCNN/FGCNN_kkbox_x1/run_expid.py b/ranking/ctr/FGCNN/FGCNN_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FGCNN/FGCNN_kkbox_x1/run_expid.py rename to ranking/ctr/FGCNN/FGCNN_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FLEN/FLEN_avazu_x4_001/FLEN_avazu_x4_006_1e50e8f0.log b/ranking/ctr/FLEN/FLEN_avazu_x4_001/FLEN_avazu_x4_006_1e50e8f0.log similarity index 100% rename from ctr_prediction/benchmarks/FLEN/FLEN_avazu_x4_001/FLEN_avazu_x4_006_1e50e8f0.log rename to ranking/ctr/FLEN/FLEN_avazu_x4_001/FLEN_avazu_x4_006_1e50e8f0.log diff --git a/ctr_prediction/benchmarks/FLEN/FLEN_avazu_x4_001/FLEN_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/FLEN/FLEN_avazu_x4_001/FLEN_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FLEN/FLEN_avazu_x4_001/FLEN_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FLEN/FLEN_avazu_x4_001/FLEN_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FLEN/FLEN_avazu_x4_001/FLEN_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/FLEN/FLEN_avazu_x4_001/FLEN_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FLEN/FLEN_avazu_x4_001/FLEN_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/FLEN/FLEN_avazu_x4_001/FLEN_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FLEN/FLEN_avazu_x4_001/README.md b/ranking/ctr/FLEN/FLEN_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/FLEN/FLEN_avazu_x4_001/README.md rename to ranking/ctr/FLEN/FLEN_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/FLEN/FLEN_avazu_x4_001/environments.txt b/ranking/ctr/FLEN/FLEN_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FLEN/FLEN_avazu_x4_001/environments.txt rename to ranking/ctr/FLEN/FLEN_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/FLEN/FLEN_avazu_x4_001/results.csv b/ranking/ctr/FLEN/FLEN_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FLEN/FLEN_avazu_x4_001/results.csv rename to ranking/ctr/FLEN/FLEN_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/FLEN/FLEN_avazu_x4_001/run_expid.py b/ranking/ctr/FLEN/FLEN_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FLEN/FLEN_avazu_x4_001/run_expid.py rename to ranking/ctr/FLEN/FLEN_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_009_a87f4871.log b/ranking/ctr/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_009_a87f4871.log similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_009_a87f4871.log rename to ranking/ctr/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_009_a87f4871.log diff --git a/ctr_prediction/benchmarks/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_tuner_config_02.csv b/ranking/ctr/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_tuner_config_02.csv rename to ranking/ctr/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_tuner_config_02.yaml b/ranking/ctr/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_tuner_config_02.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_tuner_config_02.yaml rename to ranking/ctr/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_tuner_config_02.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_tuner_config_02/model_config.yaml b/ranking/ctr/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/FM/FM_amazonelectronics_x1/FM_amazonelectronics_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_amazonelectronics_x1/README.md b/ranking/ctr/FM/FM_amazonelectronics_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_amazonelectronics_x1/README.md rename to ranking/ctr/FM/FM_amazonelectronics_x1/README.md diff --git a/ctr_prediction/benchmarks/FM/FM_amazonelectronics_x1/environments.txt b/ranking/ctr/FM/FM_amazonelectronics_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_amazonelectronics_x1/environments.txt rename to ranking/ctr/FM/FM_amazonelectronics_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FM/FM_amazonelectronics_x1/results.csv b/ranking/ctr/FM/FM_amazonelectronics_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_amazonelectronics_x1/results.csv rename to ranking/ctr/FM/FM_amazonelectronics_x1/results.csv diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x1/FM_avazu_x1_004_814f7d09.log b/ranking/ctr/FM/FM_avazu_x1/FM_avazu_x1_004_814f7d09.log similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x1/FM_avazu_x1_004_814f7d09.log rename to ranking/ctr/FM/FM_avazu_x1/FM_avazu_x1_004_814f7d09.log diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x1/FM_avazu_x1_tuner_config_01.csv b/ranking/ctr/FM/FM_avazu_x1/FM_avazu_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x1/FM_avazu_x1_tuner_config_01.csv rename to ranking/ctr/FM/FM_avazu_x1/FM_avazu_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x1/FM_avazu_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FM/FM_avazu_x1/FM_avazu_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x1/FM_avazu_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FM/FM_avazu_x1/FM_avazu_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x1/FM_avazu_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FM/FM_avazu_x1/FM_avazu_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x1/FM_avazu_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FM/FM_avazu_x1/FM_avazu_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x1/README.md b/ranking/ctr/FM/FM_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x1/README.md rename to ranking/ctr/FM/FM_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x1/environments.txt b/ranking/ctr/FM/FM_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x1/environments.txt rename to ranking/ctr/FM/FM_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x1/results.csv b/ranking/ctr/FM/FM_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x1/results.csv rename to ranking/ctr/FM/FM_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x1/run_expid.py b/ranking/ctr/FM/FM_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x1/run_expid.py rename to ranking/ctr/FM/FM_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x4_001/FM_avazu_x4_003_a62c6000.log b/ranking/ctr/FM/FM_avazu_x4_001/FM_avazu_x4_003_a62c6000.log similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x4_001/FM_avazu_x4_003_a62c6000.log rename to ranking/ctr/FM/FM_avazu_x4_001/FM_avazu_x4_003_a62c6000.log diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x4_001/FM_avazu_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/FM/FM_avazu_x4_001/FM_avazu_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x4_001/FM_avazu_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FM/FM_avazu_x4_001/FM_avazu_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x4_001/FM_avazu_x4_tuner_config_02/model_config.yaml b/ranking/ctr/FM/FM_avazu_x4_001/FM_avazu_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x4_001/FM_avazu_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/FM/FM_avazu_x4_001/FM_avazu_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x4_001/README.md b/ranking/ctr/FM/FM_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x4_001/README.md rename to ranking/ctr/FM/FM_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x4_001/environments.txt b/ranking/ctr/FM/FM_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x4_001/environments.txt rename to ranking/ctr/FM/FM_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x4_001/results.csv b/ranking/ctr/FM/FM_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x4_001/results.csv rename to ranking/ctr/FM/FM_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x4_001/run_expid.py b/ranking/ctr/FM/FM_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x4_001/run_expid.py rename to ranking/ctr/FM/FM_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x4_002/FM_avazu_x4_003_6dd622eb.log b/ranking/ctr/FM/FM_avazu_x4_002/FM_avazu_x4_003_6dd622eb.log similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x4_002/FM_avazu_x4_003_6dd622eb.log rename to ranking/ctr/FM/FM_avazu_x4_002/FM_avazu_x4_003_6dd622eb.log diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x4_002/FM_avazu_x4_tuner_config_04/dataset_config.yaml b/ranking/ctr/FM/FM_avazu_x4_002/FM_avazu_x4_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x4_002/FM_avazu_x4_tuner_config_04/dataset_config.yaml rename to ranking/ctr/FM/FM_avazu_x4_002/FM_avazu_x4_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x4_002/FM_avazu_x4_tuner_config_04/model_config.yaml b/ranking/ctr/FM/FM_avazu_x4_002/FM_avazu_x4_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x4_002/FM_avazu_x4_tuner_config_04/model_config.yaml rename to ranking/ctr/FM/FM_avazu_x4_002/FM_avazu_x4_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x4_002/README.md b/ranking/ctr/FM/FM_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x4_002/README.md rename to ranking/ctr/FM/FM_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x4_002/environments.txt b/ranking/ctr/FM/FM_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x4_002/environments.txt rename to ranking/ctr/FM/FM_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x4_002/results.csv b/ranking/ctr/FM/FM_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x4_002/results.csv rename to ranking/ctr/FM/FM_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/FM/FM_avazu_x4_002/run_expid.py b/ranking/ctr/FM/FM_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_avazu_x4_002/run_expid.py rename to ranking/ctr/FM/FM_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x1/FM_criteo_x1_001_bea53d16.log b/ranking/ctr/FM/FM_criteo_x1/FM_criteo_x1_001_bea53d16.log similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x1/FM_criteo_x1_001_bea53d16.log rename to ranking/ctr/FM/FM_criteo_x1/FM_criteo_x1_001_bea53d16.log diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x1/FM_criteo_x1_tuner_config_02.csv b/ranking/ctr/FM/FM_criteo_x1/FM_criteo_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x1/FM_criteo_x1_tuner_config_02.csv rename to ranking/ctr/FM/FM_criteo_x1/FM_criteo_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x1/FM_criteo_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/FM/FM_criteo_x1/FM_criteo_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x1/FM_criteo_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FM/FM_criteo_x1/FM_criteo_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x1/FM_criteo_x1_tuner_config_02/model_config.yaml b/ranking/ctr/FM/FM_criteo_x1/FM_criteo_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x1/FM_criteo_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/FM/FM_criteo_x1/FM_criteo_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x1/README.md b/ranking/ctr/FM/FM_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x1/README.md rename to ranking/ctr/FM/FM_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x1/environments.txt b/ranking/ctr/FM/FM_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x1/environments.txt rename to ranking/ctr/FM/FM_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x1/results.csv b/ranking/ctr/FM/FM_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x1/results.csv rename to ranking/ctr/FM/FM_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x1/run_expid.py b/ranking/ctr/FM/FM_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x1/run_expid.py rename to ranking/ctr/FM/FM_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x4_001/FM_criteo_x4_003_c13a1abf.log b/ranking/ctr/FM/FM_criteo_x4_001/FM_criteo_x4_003_c13a1abf.log similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x4_001/FM_criteo_x4_003_c13a1abf.log rename to ranking/ctr/FM/FM_criteo_x4_001/FM_criteo_x4_003_c13a1abf.log diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x4_001/FM_criteo_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/FM/FM_criteo_x4_001/FM_criteo_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x4_001/FM_criteo_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FM/FM_criteo_x4_001/FM_criteo_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x4_001/FM_criteo_x4_tuner_config_02/model_config.yaml b/ranking/ctr/FM/FM_criteo_x4_001/FM_criteo_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x4_001/FM_criteo_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/FM/FM_criteo_x4_001/FM_criteo_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x4_001/README.md b/ranking/ctr/FM/FM_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x4_001/README.md rename to ranking/ctr/FM/FM_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x4_001/environments.txt b/ranking/ctr/FM/FM_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x4_001/environments.txt rename to ranking/ctr/FM/FM_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x4_001/results.csv b/ranking/ctr/FM/FM_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x4_001/results.csv rename to ranking/ctr/FM/FM_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x4_001/run_expid.py b/ranking/ctr/FM/FM_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x4_001/run_expid.py rename to ranking/ctr/FM/FM_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x4_002/FM_criteo_x4_003_608b63aa.log b/ranking/ctr/FM/FM_criteo_x4_002/FM_criteo_x4_003_608b63aa.log similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x4_002/FM_criteo_x4_003_608b63aa.log rename to ranking/ctr/FM/FM_criteo_x4_002/FM_criteo_x4_003_608b63aa.log diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x4_002/FM_criteo_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/FM/FM_criteo_x4_002/FM_criteo_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x4_002/FM_criteo_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FM/FM_criteo_x4_002/FM_criteo_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x4_002/FM_criteo_x4_tuner_config_02/model_config.yaml b/ranking/ctr/FM/FM_criteo_x4_002/FM_criteo_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x4_002/FM_criteo_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/FM/FM_criteo_x4_002/FM_criteo_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x4_002/README.md b/ranking/ctr/FM/FM_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x4_002/README.md rename to ranking/ctr/FM/FM_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x4_002/environments.txt b/ranking/ctr/FM/FM_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x4_002/environments.txt rename to ranking/ctr/FM/FM_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x4_002/results.csv b/ranking/ctr/FM/FM_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x4_002/results.csv rename to ranking/ctr/FM/FM_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/FM/FM_criteo_x4_002/run_expid.py b/ranking/ctr/FM/FM_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_criteo_x4_002/run_expid.py rename to ranking/ctr/FM/FM_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/FM/FM_frappe_x1/FM_frappe_x1_001_6234f1e3.log b/ranking/ctr/FM/FM_frappe_x1/FM_frappe_x1_001_6234f1e3.log similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_frappe_x1/FM_frappe_x1_001_6234f1e3.log rename to ranking/ctr/FM/FM_frappe_x1/FM_frappe_x1_001_6234f1e3.log diff --git a/ctr_prediction/benchmarks/FM/FM_frappe_x1/FM_frappe_x1_tuner_config_03.csv b/ranking/ctr/FM/FM_frappe_x1/FM_frappe_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_frappe_x1/FM_frappe_x1_tuner_config_03.csv rename to ranking/ctr/FM/FM_frappe_x1/FM_frappe_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/FM/FM_frappe_x1/FM_frappe_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/FM/FM_frappe_x1/FM_frappe_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_frappe_x1/FM_frappe_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/FM/FM_frappe_x1/FM_frappe_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_frappe_x1/FM_frappe_x1_tuner_config_03/model_config.yaml b/ranking/ctr/FM/FM_frappe_x1/FM_frappe_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_frappe_x1/FM_frappe_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/FM/FM_frappe_x1/FM_frappe_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_frappe_x1/README.md b/ranking/ctr/FM/FM_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_frappe_x1/README.md rename to ranking/ctr/FM/FM_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/FM/FM_frappe_x1/environments.txt b/ranking/ctr/FM/FM_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_frappe_x1/environments.txt rename to ranking/ctr/FM/FM_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FM/FM_frappe_x1/results.csv b/ranking/ctr/FM/FM_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_frappe_x1/results.csv rename to ranking/ctr/FM/FM_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/FM/FM_frappe_x1/run_expid.py b/ranking/ctr/FM/FM_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_frappe_x1/run_expid.py rename to ranking/ctr/FM/FM_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FM/FM_kkbox_x1/FM_kkbox_x1_002_c55e9f15.log b/ranking/ctr/FM/FM_kkbox_x1/FM_kkbox_x1_002_c55e9f15.log similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_kkbox_x1/FM_kkbox_x1_002_c55e9f15.log rename to ranking/ctr/FM/FM_kkbox_x1/FM_kkbox_x1_002_c55e9f15.log diff --git a/ctr_prediction/benchmarks/FM/FM_kkbox_x1/FM_kkbox_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FM/FM_kkbox_x1/FM_kkbox_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_kkbox_x1/FM_kkbox_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FM/FM_kkbox_x1/FM_kkbox_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_kkbox_x1/FM_kkbox_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FM/FM_kkbox_x1/FM_kkbox_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_kkbox_x1/FM_kkbox_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FM/FM_kkbox_x1/FM_kkbox_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_kkbox_x1/README.md b/ranking/ctr/FM/FM_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_kkbox_x1/README.md rename to ranking/ctr/FM/FM_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/FM/FM_kkbox_x1/environments.txt b/ranking/ctr/FM/FM_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_kkbox_x1/environments.txt rename to ranking/ctr/FM/FM_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FM/FM_kkbox_x1/results.csv b/ranking/ctr/FM/FM_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_kkbox_x1/results.csv rename to ranking/ctr/FM/FM_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/FM/FM_kkbox_x1/run_expid.py b/ranking/ctr/FM/FM_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_kkbox_x1/run_expid.py rename to ranking/ctr/FM/FM_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_012_35007647.log b/ranking/ctr/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_012_35007647.log similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_012_35007647.log rename to ranking/ctr/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_012_35007647.log diff --git a/ctr_prediction/benchmarks/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_tuner_config_02.csv b/ranking/ctr/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_tuner_config_02.csv rename to ranking/ctr/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_tuner_config_02.yaml b/ranking/ctr/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_tuner_config_02.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_tuner_config_02.yaml rename to ranking/ctr/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_tuner_config_02.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_tuner_config_02/model_config.yaml b/ranking/ctr/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/FM/FM_kuaivideo_x1/FM_kuaivideo_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_kuaivideo_x1/README.md b/ranking/ctr/FM/FM_kuaivideo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_kuaivideo_x1/README.md rename to ranking/ctr/FM/FM_kuaivideo_x1/README.md diff --git a/ctr_prediction/benchmarks/FM/FM_kuaivideo_x1/environments.txt b/ranking/ctr/FM/FM_kuaivideo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_kuaivideo_x1/environments.txt rename to ranking/ctr/FM/FM_kuaivideo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FM/FM_kuaivideo_x1/results.csv b/ranking/ctr/FM/FM_kuaivideo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_kuaivideo_x1/results.csv rename to ranking/ctr/FM/FM_kuaivideo_x1/results.csv diff --git a/ctr_prediction/benchmarks/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_003_ffc302aa.log b/ranking/ctr/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_003_ffc302aa.log similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_003_ffc302aa.log rename to ranking/ctr/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_003_ffc302aa.log diff --git a/ctr_prediction/benchmarks/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_tuner_config_05.csv b/ranking/ctr/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_tuner_config_05.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_tuner_config_05.csv rename to ranking/ctr/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_tuner_config_05.csv diff --git a/ctr_prediction/benchmarks/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_tuner_config_05.yaml b/ranking/ctr/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_tuner_config_05.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_tuner_config_05.yaml rename to ranking/ctr/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_tuner_config_05.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_tuner_config_05/dataset_config.yaml b/ranking/ctr/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_tuner_config_05/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_tuner_config_05/dataset_config.yaml rename to ranking/ctr/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_tuner_config_05/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_tuner_config_05/model_config.yaml b/ranking/ctr/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_tuner_config_05/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_tuner_config_05/model_config.yaml rename to ranking/ctr/FM/FM_microvideo1.7m_x1/FM_microvideo1.7m_x1_tuner_config_05/model_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_microvideo1.7m_x1/README.md b/ranking/ctr/FM/FM_microvideo1.7m_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_microvideo1.7m_x1/README.md rename to ranking/ctr/FM/FM_microvideo1.7m_x1/README.md diff --git a/ctr_prediction/benchmarks/FM/FM_microvideo1.7m_x1/environments.txt b/ranking/ctr/FM/FM_microvideo1.7m_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_microvideo1.7m_x1/environments.txt rename to ranking/ctr/FM/FM_microvideo1.7m_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FM/FM_microvideo1.7m_x1/results.csv b/ranking/ctr/FM/FM_microvideo1.7m_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_microvideo1.7m_x1/results.csv rename to ranking/ctr/FM/FM_microvideo1.7m_x1/results.csv diff --git a/ctr_prediction/benchmarks/FM/FM_movielenslatest_x1/FM_movielenslatest_x1_003_3e63baf6.log b/ranking/ctr/FM/FM_movielenslatest_x1/FM_movielenslatest_x1_003_3e63baf6.log similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_movielenslatest_x1/FM_movielenslatest_x1_003_3e63baf6.log rename to ranking/ctr/FM/FM_movielenslatest_x1/FM_movielenslatest_x1_003_3e63baf6.log diff --git a/ctr_prediction/benchmarks/FM/FM_movielenslatest_x1/FM_movielenslatest_x1_tuner_config_03.csv b/ranking/ctr/FM/FM_movielenslatest_x1/FM_movielenslatest_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_movielenslatest_x1/FM_movielenslatest_x1_tuner_config_03.csv rename to ranking/ctr/FM/FM_movielenslatest_x1/FM_movielenslatest_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/FM/FM_movielenslatest_x1/FM_movielenslatest_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/FM/FM_movielenslatest_x1/FM_movielenslatest_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_movielenslatest_x1/FM_movielenslatest_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/FM/FM_movielenslatest_x1/FM_movielenslatest_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_movielenslatest_x1/FM_movielenslatest_x1_tuner_config_03/model_config.yaml b/ranking/ctr/FM/FM_movielenslatest_x1/FM_movielenslatest_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_movielenslatest_x1/FM_movielenslatest_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/FM/FM_movielenslatest_x1/FM_movielenslatest_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_movielenslatest_x1/README.md b/ranking/ctr/FM/FM_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_movielenslatest_x1/README.md rename to ranking/ctr/FM/FM_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/FM/FM_movielenslatest_x1/environments.txt b/ranking/ctr/FM/FM_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_movielenslatest_x1/environments.txt rename to ranking/ctr/FM/FM_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FM/FM_movielenslatest_x1/results.csv b/ranking/ctr/FM/FM_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_movielenslatest_x1/results.csv rename to ranking/ctr/FM/FM_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/FM/FM_movielenslatest_x1/run_expid.py b/ranking/ctr/FM/FM_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_movielenslatest_x1/run_expid.py rename to ranking/ctr/FM/FM_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FM/FM_taobaoad_x1/FM_taobaoad_x1_009_a28e6634.log b/ranking/ctr/FM/FM_taobaoad_x1/FM_taobaoad_x1_009_a28e6634.log similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_taobaoad_x1/FM_taobaoad_x1_009_a28e6634.log rename to ranking/ctr/FM/FM_taobaoad_x1/FM_taobaoad_x1_009_a28e6634.log diff --git a/ctr_prediction/benchmarks/FM/FM_taobaoad_x1/FM_taobaoad_x1_tuner_config_01.csv b/ranking/ctr/FM/FM_taobaoad_x1/FM_taobaoad_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_taobaoad_x1/FM_taobaoad_x1_tuner_config_01.csv rename to ranking/ctr/FM/FM_taobaoad_x1/FM_taobaoad_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/FM/FM_taobaoad_x1/FM_taobaoad_x1_tuner_config_01.yaml b/ranking/ctr/FM/FM_taobaoad_x1/FM_taobaoad_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_taobaoad_x1/FM_taobaoad_x1_tuner_config_01.yaml rename to ranking/ctr/FM/FM_taobaoad_x1/FM_taobaoad_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_taobaoad_x1/FM_taobaoad_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FM/FM_taobaoad_x1/FM_taobaoad_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_taobaoad_x1/FM_taobaoad_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FM/FM_taobaoad_x1/FM_taobaoad_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_taobaoad_x1/FM_taobaoad_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FM/FM_taobaoad_x1/FM_taobaoad_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_taobaoad_x1/FM_taobaoad_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FM/FM_taobaoad_x1/FM_taobaoad_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FM/FM_taobaoad_x1/README.md b/ranking/ctr/FM/FM_taobaoad_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_taobaoad_x1/README.md rename to ranking/ctr/FM/FM_taobaoad_x1/README.md diff --git a/ctr_prediction/benchmarks/FM/FM_taobaoad_x1/environments.txt b/ranking/ctr/FM/FM_taobaoad_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_taobaoad_x1/environments.txt rename to ranking/ctr/FM/FM_taobaoad_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FM/FM_taobaoad_x1/results.csv b/ranking/ctr/FM/FM_taobaoad_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FM/FM_taobaoad_x1/results.csv rename to ranking/ctr/FM/FM_taobaoad_x1/results.csv diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x1/FiBiNET_avazu_x1_002_f09a284c.log b/ranking/ctr/FiBiNET/FiBiNET_avazu_x1/FiBiNET_avazu_x1_002_f09a284c.log similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x1/FiBiNET_avazu_x1_002_f09a284c.log rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x1/FiBiNET_avazu_x1_002_f09a284c.log diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x1/FiBiNET_avazu_x1_tuner_config_06.csv b/ranking/ctr/FiBiNET/FiBiNET_avazu_x1/FiBiNET_avazu_x1_tuner_config_06.csv similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x1/FiBiNET_avazu_x1_tuner_config_06.csv rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x1/FiBiNET_avazu_x1_tuner_config_06.csv diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x1/FiBiNET_avazu_x1_tuner_config_06/dataset_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_avazu_x1/FiBiNET_avazu_x1_tuner_config_06/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x1/FiBiNET_avazu_x1_tuner_config_06/dataset_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x1/FiBiNET_avazu_x1_tuner_config_06/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x1/FiBiNET_avazu_x1_tuner_config_06/model_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_avazu_x1/FiBiNET_avazu_x1_tuner_config_06/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x1/FiBiNET_avazu_x1_tuner_config_06/model_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x1/FiBiNET_avazu_x1_tuner_config_06/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x1/README.md b/ranking/ctr/FiBiNET/FiBiNET_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x1/README.md rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x1/environments.txt b/ranking/ctr/FiBiNET/FiBiNET_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x1/environments.txt rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x1/results.csv b/ranking/ctr/FiBiNET/FiBiNET_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x1/results.csv rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x1/run_expid.py b/ranking/ctr/FiBiNET/FiBiNET_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x1/run_expid.py rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_001/FiBiNET_avazu_x4_002_3a9a2fa0.log b/ranking/ctr/FiBiNET/FiBiNET_avazu_x4_001/FiBiNET_avazu_x4_002_3a9a2fa0.log similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_001/FiBiNET_avazu_x4_002_3a9a2fa0.log rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x4_001/FiBiNET_avazu_x4_002_3a9a2fa0.log diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_001/FiBiNET_avazu_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_avazu_x4_001/FiBiNET_avazu_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_001/FiBiNET_avazu_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x4_001/FiBiNET_avazu_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_001/FiBiNET_avazu_x4_tuner_config_02/model_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_avazu_x4_001/FiBiNET_avazu_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_001/FiBiNET_avazu_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x4_001/FiBiNET_avazu_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_001/README.md b/ranking/ctr/FiBiNET/FiBiNET_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_001/README.md rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_001/environments.txt b/ranking/ctr/FiBiNET/FiBiNET_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_001/environments.txt rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_001/results.csv b/ranking/ctr/FiBiNET/FiBiNET_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_001/results.csv rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_001/run_expid.py b/ranking/ctr/FiBiNET/FiBiNET_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_001/run_expid.py rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_002/FiBiNET_avazu_x4_012_fe1312f5.log b/ranking/ctr/FiBiNET/FiBiNET_avazu_x4_002/FiBiNET_avazu_x4_012_fe1312f5.log similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_002/FiBiNET_avazu_x4_012_fe1312f5.log rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x4_002/FiBiNET_avazu_x4_012_fe1312f5.log diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_002/FiBiNET_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_avazu_x4_002/FiBiNET_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_002/FiBiNET_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x4_002/FiBiNET_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_002/FiBiNET_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_avazu_x4_002/FiBiNET_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_002/FiBiNET_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x4_002/FiBiNET_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_002/README.md b/ranking/ctr/FiBiNET/FiBiNET_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_002/README.md rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_002/environments.txt b/ranking/ctr/FiBiNET/FiBiNET_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_002/environments.txt rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_002/results.csv b/ranking/ctr/FiBiNET/FiBiNET_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_002/results.csv rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_002/run_expid.py b/ranking/ctr/FiBiNET/FiBiNET_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_avazu_x4_002/run_expid.py rename to ranking/ctr/FiBiNET/FiBiNET_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x1/FiBiNET_criteo_x1_022_19ca3145.log b/ranking/ctr/FiBiNET/FiBiNET_criteo_x1/FiBiNET_criteo_x1_022_19ca3145.log similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x1/FiBiNET_criteo_x1_022_19ca3145.log rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x1/FiBiNET_criteo_x1_022_19ca3145.log diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x1/FiBiNET_criteo_x1_tuner_config_03.csv b/ranking/ctr/FiBiNET/FiBiNET_criteo_x1/FiBiNET_criteo_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x1/FiBiNET_criteo_x1_tuner_config_03.csv rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x1/FiBiNET_criteo_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x1/FiBiNET_criteo_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_criteo_x1/FiBiNET_criteo_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x1/FiBiNET_criteo_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x1/FiBiNET_criteo_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x1/FiBiNET_criteo_x1_tuner_config_03/model_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_criteo_x1/FiBiNET_criteo_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x1/FiBiNET_criteo_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x1/FiBiNET_criteo_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x1/README.md b/ranking/ctr/FiBiNET/FiBiNET_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x1/README.md rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x1/environments.txt b/ranking/ctr/FiBiNET/FiBiNET_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x1/environments.txt rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x1/results.csv b/ranking/ctr/FiBiNET/FiBiNET_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x1/results.csv rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x1/run_expid.py b/ranking/ctr/FiBiNET/FiBiNET_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x1/run_expid.py rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_001/FiBiNET_criteo_x4_004_73513faa.log b/ranking/ctr/FiBiNET/FiBiNET_criteo_x4_001/FiBiNET_criteo_x4_004_73513faa.log similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_001/FiBiNET_criteo_x4_004_73513faa.log rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x4_001/FiBiNET_criteo_x4_004_73513faa.log diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_001/FiBiNET_criteo_x4_tuner_config_05/dataset_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_criteo_x4_001/FiBiNET_criteo_x4_tuner_config_05/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_001/FiBiNET_criteo_x4_tuner_config_05/dataset_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x4_001/FiBiNET_criteo_x4_tuner_config_05/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_001/FiBiNET_criteo_x4_tuner_config_05/model_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_criteo_x4_001/FiBiNET_criteo_x4_tuner_config_05/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_001/FiBiNET_criteo_x4_tuner_config_05/model_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x4_001/FiBiNET_criteo_x4_tuner_config_05/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_001/README.md b/ranking/ctr/FiBiNET/FiBiNET_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_001/README.md rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_001/environments.txt b/ranking/ctr/FiBiNET/FiBiNET_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_001/environments.txt rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_001/results.csv b/ranking/ctr/FiBiNET/FiBiNET_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_001/results.csv rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_001/run_expid.py b/ranking/ctr/FiBiNET/FiBiNET_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_001/run_expid.py rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_002/FiBiNET_criteo_x4_012_faef5b65.log b/ranking/ctr/FiBiNET/FiBiNET_criteo_x4_002/FiBiNET_criteo_x4_012_faef5b65.log similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_002/FiBiNET_criteo_x4_012_faef5b65.log rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x4_002/FiBiNET_criteo_x4_012_faef5b65.log diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_002/FiBiNET_criteo_x4_tuner_config_03/dataset_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_criteo_x4_002/FiBiNET_criteo_x4_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_002/FiBiNET_criteo_x4_tuner_config_03/dataset_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x4_002/FiBiNET_criteo_x4_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_002/FiBiNET_criteo_x4_tuner_config_03/model_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_criteo_x4_002/FiBiNET_criteo_x4_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_002/FiBiNET_criteo_x4_tuner_config_03/model_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x4_002/FiBiNET_criteo_x4_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_002/README.md b/ranking/ctr/FiBiNET/FiBiNET_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_002/README.md rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_002/environments.txt b/ranking/ctr/FiBiNET/FiBiNET_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_002/environments.txt rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_002/results.csv b/ranking/ctr/FiBiNET/FiBiNET_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_002/results.csv rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_002/run_expid.py b/ranking/ctr/FiBiNET/FiBiNET_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_criteo_x4_002/run_expid.py rename to ranking/ctr/FiBiNET/FiBiNET_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_frappe_x1/FiBiNET_frappe_x1_002_aedd06a6.log b/ranking/ctr/FiBiNET/FiBiNET_frappe_x1/FiBiNET_frappe_x1_002_aedd06a6.log similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_frappe_x1/FiBiNET_frappe_x1_002_aedd06a6.log rename to ranking/ctr/FiBiNET/FiBiNET_frappe_x1/FiBiNET_frappe_x1_002_aedd06a6.log diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_frappe_x1/FiBiNET_frappe_x1_tuner_config_03.csv b/ranking/ctr/FiBiNET/FiBiNET_frappe_x1/FiBiNET_frappe_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_frappe_x1/FiBiNET_frappe_x1_tuner_config_03.csv rename to ranking/ctr/FiBiNET/FiBiNET_frappe_x1/FiBiNET_frappe_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_frappe_x1/FiBiNET_frappe_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_frappe_x1/FiBiNET_frappe_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_frappe_x1/FiBiNET_frappe_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_frappe_x1/FiBiNET_frappe_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_frappe_x1/FiBiNET_frappe_x1_tuner_config_03/model_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_frappe_x1/FiBiNET_frappe_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_frappe_x1/FiBiNET_frappe_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_frappe_x1/FiBiNET_frappe_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_frappe_x1/README.md b/ranking/ctr/FiBiNET/FiBiNET_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_frappe_x1/README.md rename to ranking/ctr/FiBiNET/FiBiNET_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_frappe_x1/environments.txt b/ranking/ctr/FiBiNET/FiBiNET_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_frappe_x1/environments.txt rename to ranking/ctr/FiBiNET/FiBiNET_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_frappe_x1/results.csv b/ranking/ctr/FiBiNET/FiBiNET_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_frappe_x1/results.csv rename to ranking/ctr/FiBiNET/FiBiNET_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_frappe_x1/run_expid.py b/ranking/ctr/FiBiNET/FiBiNET_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_frappe_x1/run_expid.py rename to ranking/ctr/FiBiNET/FiBiNET_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_kkbox_x1/FiBiNET_kkbox_x1_007_f8ea597b.log b/ranking/ctr/FiBiNET/FiBiNET_kkbox_x1/FiBiNET_kkbox_x1_007_f8ea597b.log similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_kkbox_x1/FiBiNET_kkbox_x1_007_f8ea597b.log rename to ranking/ctr/FiBiNET/FiBiNET_kkbox_x1/FiBiNET_kkbox_x1_007_f8ea597b.log diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_kkbox_x1/FiBiNET_kkbox_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_kkbox_x1/FiBiNET_kkbox_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_kkbox_x1/FiBiNET_kkbox_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_kkbox_x1/FiBiNET_kkbox_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_kkbox_x1/FiBiNET_kkbox_x1_tuner_config_03/model_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_kkbox_x1/FiBiNET_kkbox_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_kkbox_x1/FiBiNET_kkbox_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_kkbox_x1/FiBiNET_kkbox_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_kkbox_x1/README.md b/ranking/ctr/FiBiNET/FiBiNET_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_kkbox_x1/README.md rename to ranking/ctr/FiBiNET/FiBiNET_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_kkbox_x1/environments.txt b/ranking/ctr/FiBiNET/FiBiNET_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_kkbox_x1/environments.txt rename to ranking/ctr/FiBiNET/FiBiNET_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_kkbox_x1/results.csv b/ranking/ctr/FiBiNET/FiBiNET_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_kkbox_x1/results.csv rename to ranking/ctr/FiBiNET/FiBiNET_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_kkbox_x1/run_expid.py b/ranking/ctr/FiBiNET/FiBiNET_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_kkbox_x1/run_expid.py rename to ranking/ctr/FiBiNET/FiBiNET_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_movielenslatest_x1/FiBiNET_movielenslatest_x1_017_24eff5cf.log b/ranking/ctr/FiBiNET/FiBiNET_movielenslatest_x1/FiBiNET_movielenslatest_x1_017_24eff5cf.log similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_movielenslatest_x1/FiBiNET_movielenslatest_x1_017_24eff5cf.log rename to ranking/ctr/FiBiNET/FiBiNET_movielenslatest_x1/FiBiNET_movielenslatest_x1_017_24eff5cf.log diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_movielenslatest_x1/FiBiNET_movielenslatest_x1_tuner_config_03.csv b/ranking/ctr/FiBiNET/FiBiNET_movielenslatest_x1/FiBiNET_movielenslatest_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_movielenslatest_x1/FiBiNET_movielenslatest_x1_tuner_config_03.csv rename to ranking/ctr/FiBiNET/FiBiNET_movielenslatest_x1/FiBiNET_movielenslatest_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_movielenslatest_x1/FiBiNET_movielenslatest_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_movielenslatest_x1/FiBiNET_movielenslatest_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_movielenslatest_x1/FiBiNET_movielenslatest_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_movielenslatest_x1/FiBiNET_movielenslatest_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_movielenslatest_x1/FiBiNET_movielenslatest_x1_tuner_config_03/model_config.yaml b/ranking/ctr/FiBiNET/FiBiNET_movielenslatest_x1/FiBiNET_movielenslatest_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_movielenslatest_x1/FiBiNET_movielenslatest_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/FiBiNET/FiBiNET_movielenslatest_x1/FiBiNET_movielenslatest_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_movielenslatest_x1/README.md b/ranking/ctr/FiBiNET/FiBiNET_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_movielenslatest_x1/README.md rename to ranking/ctr/FiBiNET/FiBiNET_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_movielenslatest_x1/environments.txt b/ranking/ctr/FiBiNET/FiBiNET_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_movielenslatest_x1/environments.txt rename to ranking/ctr/FiBiNET/FiBiNET_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_movielenslatest_x1/results.csv b/ranking/ctr/FiBiNET/FiBiNET_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_movielenslatest_x1/results.csv rename to ranking/ctr/FiBiNET/FiBiNET_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/FiBiNET/FiBiNET_movielenslatest_x1/run_expid.py b/ranking/ctr/FiBiNET/FiBiNET_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiBiNET/FiBiNET_movielenslatest_x1/run_expid.py rename to ranking/ctr/FiBiNET/FiBiNET_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x1/FiGNN_avazu_x1_015_9c5d8df1.log b/ranking/ctr/FiGNN/FiGNN_avazu_x1/FiGNN_avazu_x1_015_9c5d8df1.log similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x1/FiGNN_avazu_x1_015_9c5d8df1.log rename to ranking/ctr/FiGNN/FiGNN_avazu_x1/FiGNN_avazu_x1_015_9c5d8df1.log diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x1/FiGNN_avazu_x1_tuner_config_02.csv b/ranking/ctr/FiGNN/FiGNN_avazu_x1/FiGNN_avazu_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x1/FiGNN_avazu_x1_tuner_config_02.csv rename to ranking/ctr/FiGNN/FiGNN_avazu_x1/FiGNN_avazu_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x1/FiGNN_avazu_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/FiGNN/FiGNN_avazu_x1/FiGNN_avazu_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x1/FiGNN_avazu_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FiGNN/FiGNN_avazu_x1/FiGNN_avazu_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x1/FiGNN_avazu_x1_tuner_config_02/model_config.yaml b/ranking/ctr/FiGNN/FiGNN_avazu_x1/FiGNN_avazu_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x1/FiGNN_avazu_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/FiGNN/FiGNN_avazu_x1/FiGNN_avazu_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x1/README.md b/ranking/ctr/FiGNN/FiGNN_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x1/README.md rename to ranking/ctr/FiGNN/FiGNN_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x1/environments.txt b/ranking/ctr/FiGNN/FiGNN_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x1/environments.txt rename to ranking/ctr/FiGNN/FiGNN_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x1/results.csv b/ranking/ctr/FiGNN/FiGNN_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x1/results.csv rename to ranking/ctr/FiGNN/FiGNN_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x1/run_expid.py b/ranking/ctr/FiGNN/FiGNN_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x1/run_expid.py rename to ranking/ctr/FiGNN/FiGNN_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_001/FiGNN_avazu_x4_002_c8515c76.log b/ranking/ctr/FiGNN/FiGNN_avazu_x4_001/FiGNN_avazu_x4_002_c8515c76.log similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_001/FiGNN_avazu_x4_002_c8515c76.log rename to ranking/ctr/FiGNN/FiGNN_avazu_x4_001/FiGNN_avazu_x4_002_c8515c76.log diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_001/FiGNN_avazu_x4_tuner_config_03/dataset_config.yaml b/ranking/ctr/FiGNN/FiGNN_avazu_x4_001/FiGNN_avazu_x4_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_001/FiGNN_avazu_x4_tuner_config_03/dataset_config.yaml rename to ranking/ctr/FiGNN/FiGNN_avazu_x4_001/FiGNN_avazu_x4_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_001/FiGNN_avazu_x4_tuner_config_03/model_config.yaml b/ranking/ctr/FiGNN/FiGNN_avazu_x4_001/FiGNN_avazu_x4_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_001/FiGNN_avazu_x4_tuner_config_03/model_config.yaml rename to ranking/ctr/FiGNN/FiGNN_avazu_x4_001/FiGNN_avazu_x4_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_001/README.md b/ranking/ctr/FiGNN/FiGNN_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_001/README.md rename to ranking/ctr/FiGNN/FiGNN_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_001/environments.txt b/ranking/ctr/FiGNN/FiGNN_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_001/environments.txt rename to ranking/ctr/FiGNN/FiGNN_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_001/results.csv b/ranking/ctr/FiGNN/FiGNN_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_001/results.csv rename to ranking/ctr/FiGNN/FiGNN_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_001/run_expid.py b/ranking/ctr/FiGNN/FiGNN_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_001/run_expid.py rename to ranking/ctr/FiGNN/FiGNN_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_002/FiGNN_avazu_x4_005_918ee532.log b/ranking/ctr/FiGNN/FiGNN_avazu_x4_002/FiGNN_avazu_x4_005_918ee532.log similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_002/FiGNN_avazu_x4_005_918ee532.log rename to ranking/ctr/FiGNN/FiGNN_avazu_x4_002/FiGNN_avazu_x4_005_918ee532.log diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_002/FiGNN_avazu_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/FiGNN/FiGNN_avazu_x4_002/FiGNN_avazu_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_002/FiGNN_avazu_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FiGNN/FiGNN_avazu_x4_002/FiGNN_avazu_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_002/FiGNN_avazu_x4_tuner_config_02/model_config.yaml b/ranking/ctr/FiGNN/FiGNN_avazu_x4_002/FiGNN_avazu_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_002/FiGNN_avazu_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/FiGNN/FiGNN_avazu_x4_002/FiGNN_avazu_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_002/README.md b/ranking/ctr/FiGNN/FiGNN_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_002/README.md rename to ranking/ctr/FiGNN/FiGNN_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_002/environments.txt b/ranking/ctr/FiGNN/FiGNN_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_002/environments.txt rename to ranking/ctr/FiGNN/FiGNN_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_002/results.csv b/ranking/ctr/FiGNN/FiGNN_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_002/results.csv rename to ranking/ctr/FiGNN/FiGNN_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_002/run_expid.py b/ranking/ctr/FiGNN/FiGNN_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_avazu_x4_002/run_expid.py rename to ranking/ctr/FiGNN/FiGNN_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x1/FiGNN_criteo_x1_003_c6718583.log b/ranking/ctr/FiGNN/FiGNN_criteo_x1/FiGNN_criteo_x1_003_c6718583.log similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x1/FiGNN_criteo_x1_003_c6718583.log rename to ranking/ctr/FiGNN/FiGNN_criteo_x1/FiGNN_criteo_x1_003_c6718583.log diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x1/FiGNN_criteo_x1_tuner_config_01.csv b/ranking/ctr/FiGNN/FiGNN_criteo_x1/FiGNN_criteo_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x1/FiGNN_criteo_x1_tuner_config_01.csv rename to ranking/ctr/FiGNN/FiGNN_criteo_x1/FiGNN_criteo_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x1/FiGNN_criteo_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FiGNN/FiGNN_criteo_x1/FiGNN_criteo_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x1/FiGNN_criteo_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FiGNN/FiGNN_criteo_x1/FiGNN_criteo_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x1/FiGNN_criteo_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FiGNN/FiGNN_criteo_x1/FiGNN_criteo_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x1/FiGNN_criteo_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FiGNN/FiGNN_criteo_x1/FiGNN_criteo_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x1/README.md b/ranking/ctr/FiGNN/FiGNN_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x1/README.md rename to ranking/ctr/FiGNN/FiGNN_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x1/environments.txt b/ranking/ctr/FiGNN/FiGNN_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x1/environments.txt rename to ranking/ctr/FiGNN/FiGNN_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x1/results.csv b/ranking/ctr/FiGNN/FiGNN_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x1/results.csv rename to ranking/ctr/FiGNN/FiGNN_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x1/run_expid.py b/ranking/ctr/FiGNN/FiGNN_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x1/run_expid.py rename to ranking/ctr/FiGNN/FiGNN_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_001/FiGNN_criteo_x4_006_097e8622.log b/ranking/ctr/FiGNN/FiGNN_criteo_x4_001/FiGNN_criteo_x4_006_097e8622.log similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_001/FiGNN_criteo_x4_006_097e8622.log rename to ranking/ctr/FiGNN/FiGNN_criteo_x4_001/FiGNN_criteo_x4_006_097e8622.log diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_001/FiGNN_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/FiGNN/FiGNN_criteo_x4_001/FiGNN_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_001/FiGNN_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FiGNN/FiGNN_criteo_x4_001/FiGNN_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_001/FiGNN_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/FiGNN/FiGNN_criteo_x4_001/FiGNN_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_001/FiGNN_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/FiGNN/FiGNN_criteo_x4_001/FiGNN_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_001/README.md b/ranking/ctr/FiGNN/FiGNN_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_001/README.md rename to ranking/ctr/FiGNN/FiGNN_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_001/environments.txt b/ranking/ctr/FiGNN/FiGNN_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_001/environments.txt rename to ranking/ctr/FiGNN/FiGNN_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_001/results.csv b/ranking/ctr/FiGNN/FiGNN_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_001/results.csv rename to ranking/ctr/FiGNN/FiGNN_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_001/run_expid.py b/ranking/ctr/FiGNN/FiGNN_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_001/run_expid.py rename to ranking/ctr/FiGNN/FiGNN_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_002/FiGNN_criteo_x4_005_033c41f1.log b/ranking/ctr/FiGNN/FiGNN_criteo_x4_002/FiGNN_criteo_x4_005_033c41f1.log similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_002/FiGNN_criteo_x4_005_033c41f1.log rename to ranking/ctr/FiGNN/FiGNN_criteo_x4_002/FiGNN_criteo_x4_005_033c41f1.log diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_002/FiGNN_criteo_x4_tuner_config_03/dataset_config.yaml b/ranking/ctr/FiGNN/FiGNN_criteo_x4_002/FiGNN_criteo_x4_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_002/FiGNN_criteo_x4_tuner_config_03/dataset_config.yaml rename to ranking/ctr/FiGNN/FiGNN_criteo_x4_002/FiGNN_criteo_x4_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_002/FiGNN_criteo_x4_tuner_config_03/model_config.yaml b/ranking/ctr/FiGNN/FiGNN_criteo_x4_002/FiGNN_criteo_x4_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_002/FiGNN_criteo_x4_tuner_config_03/model_config.yaml rename to ranking/ctr/FiGNN/FiGNN_criteo_x4_002/FiGNN_criteo_x4_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_002/README.md b/ranking/ctr/FiGNN/FiGNN_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_002/README.md rename to ranking/ctr/FiGNN/FiGNN_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_002/environments.txt b/ranking/ctr/FiGNN/FiGNN_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_002/environments.txt rename to ranking/ctr/FiGNN/FiGNN_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_002/results.csv b/ranking/ctr/FiGNN/FiGNN_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_002/results.csv rename to ranking/ctr/FiGNN/FiGNN_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_002/run_expid.py b/ranking/ctr/FiGNN/FiGNN_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_criteo_x4_002/run_expid.py rename to ranking/ctr/FiGNN/FiGNN_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_frappe_x1/FiGNN_frappe_x1_003_57243619.log b/ranking/ctr/FiGNN/FiGNN_frappe_x1/FiGNN_frappe_x1_003_57243619.log similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_frappe_x1/FiGNN_frappe_x1_003_57243619.log rename to ranking/ctr/FiGNN/FiGNN_frappe_x1/FiGNN_frappe_x1_003_57243619.log diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_frappe_x1/FiGNN_frappe_x1_tuner_config_04.csv b/ranking/ctr/FiGNN/FiGNN_frappe_x1/FiGNN_frappe_x1_tuner_config_04.csv similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_frappe_x1/FiGNN_frappe_x1_tuner_config_04.csv rename to ranking/ctr/FiGNN/FiGNN_frappe_x1/FiGNN_frappe_x1_tuner_config_04.csv diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_frappe_x1/FiGNN_frappe_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/FiGNN/FiGNN_frappe_x1/FiGNN_frappe_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_frappe_x1/FiGNN_frappe_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/FiGNN/FiGNN_frappe_x1/FiGNN_frappe_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_frappe_x1/FiGNN_frappe_x1_tuner_config_04/model_config.yaml b/ranking/ctr/FiGNN/FiGNN_frappe_x1/FiGNN_frappe_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_frappe_x1/FiGNN_frappe_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/FiGNN/FiGNN_frappe_x1/FiGNN_frappe_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_frappe_x1/README.md b/ranking/ctr/FiGNN/FiGNN_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_frappe_x1/README.md rename to ranking/ctr/FiGNN/FiGNN_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_frappe_x1/environments.txt b/ranking/ctr/FiGNN/FiGNN_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_frappe_x1/environments.txt rename to ranking/ctr/FiGNN/FiGNN_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_frappe_x1/results.csv b/ranking/ctr/FiGNN/FiGNN_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_frappe_x1/results.csv rename to ranking/ctr/FiGNN/FiGNN_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_frappe_x1/run_expid.py b/ranking/ctr/FiGNN/FiGNN_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_frappe_x1/run_expid.py rename to ranking/ctr/FiGNN/FiGNN_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_kkbox_x1/FiGNN_kkbox_x1_003_92b4cef8.log b/ranking/ctr/FiGNN/FiGNN_kkbox_x1/FiGNN_kkbox_x1_003_92b4cef8.log similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_kkbox_x1/FiGNN_kkbox_x1_003_92b4cef8.log rename to ranking/ctr/FiGNN/FiGNN_kkbox_x1/FiGNN_kkbox_x1_003_92b4cef8.log diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_kkbox_x1/FiGNN_kkbox_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/FiGNN/FiGNN_kkbox_x1/FiGNN_kkbox_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_kkbox_x1/FiGNN_kkbox_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/FiGNN/FiGNN_kkbox_x1/FiGNN_kkbox_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_kkbox_x1/FiGNN_kkbox_x1_tuner_config_04/model_config.yaml b/ranking/ctr/FiGNN/FiGNN_kkbox_x1/FiGNN_kkbox_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_kkbox_x1/FiGNN_kkbox_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/FiGNN/FiGNN_kkbox_x1/FiGNN_kkbox_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_kkbox_x1/README.md b/ranking/ctr/FiGNN/FiGNN_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_kkbox_x1/README.md rename to ranking/ctr/FiGNN/FiGNN_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_kkbox_x1/environments.txt b/ranking/ctr/FiGNN/FiGNN_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_kkbox_x1/environments.txt rename to ranking/ctr/FiGNN/FiGNN_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_kkbox_x1/results.csv b/ranking/ctr/FiGNN/FiGNN_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_kkbox_x1/results.csv rename to ranking/ctr/FiGNN/FiGNN_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_kkbox_x1/run_expid.py b/ranking/ctr/FiGNN/FiGNN_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_kkbox_x1/run_expid.py rename to ranking/ctr/FiGNN/FiGNN_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_movielenslatest_x1/FiGNN_movielenslatest_x1_005_73cc20f9.log b/ranking/ctr/FiGNN/FiGNN_movielenslatest_x1/FiGNN_movielenslatest_x1_005_73cc20f9.log similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_movielenslatest_x1/FiGNN_movielenslatest_x1_005_73cc20f9.log rename to ranking/ctr/FiGNN/FiGNN_movielenslatest_x1/FiGNN_movielenslatest_x1_005_73cc20f9.log diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_movielenslatest_x1/FiGNN_movielenslatest_x1_tuner_config_02.csv b/ranking/ctr/FiGNN/FiGNN_movielenslatest_x1/FiGNN_movielenslatest_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_movielenslatest_x1/FiGNN_movielenslatest_x1_tuner_config_02.csv rename to ranking/ctr/FiGNN/FiGNN_movielenslatest_x1/FiGNN_movielenslatest_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_movielenslatest_x1/FiGNN_movielenslatest_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/FiGNN/FiGNN_movielenslatest_x1/FiGNN_movielenslatest_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_movielenslatest_x1/FiGNN_movielenslatest_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FiGNN/FiGNN_movielenslatest_x1/FiGNN_movielenslatest_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_movielenslatest_x1/FiGNN_movielenslatest_x1_tuner_config_02/model_config.yaml b/ranking/ctr/FiGNN/FiGNN_movielenslatest_x1/FiGNN_movielenslatest_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_movielenslatest_x1/FiGNN_movielenslatest_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/FiGNN/FiGNN_movielenslatest_x1/FiGNN_movielenslatest_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_movielenslatest_x1/README.md b/ranking/ctr/FiGNN/FiGNN_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_movielenslatest_x1/README.md rename to ranking/ctr/FiGNN/FiGNN_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_movielenslatest_x1/environments.txt b/ranking/ctr/FiGNN/FiGNN_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_movielenslatest_x1/environments.txt rename to ranking/ctr/FiGNN/FiGNN_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_movielenslatest_x1/results.csv b/ranking/ctr/FiGNN/FiGNN_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_movielenslatest_x1/results.csv rename to ranking/ctr/FiGNN/FiGNN_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/FiGNN/FiGNN_movielenslatest_x1/run_expid.py b/ranking/ctr/FiGNN/FiGNN_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FiGNN/FiGNN_movielenslatest_x1/run_expid.py rename to ranking/ctr/FiGNN/FiGNN_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_018_488deefd.log b/ranking/ctr/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_018_488deefd.log similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_018_488deefd.log rename to ranking/ctr/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_018_488deefd.log diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_tuner_config_03.csv b/ranking/ctr/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_tuner_config_03.csv rename to ranking/ctr/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_tuner_config_03.yaml b/ranking/ctr/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_tuner_config_03.yaml similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_tuner_config_03.yaml rename to ranking/ctr/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_tuner_config_03.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_1B_avazu_x1/FINAL_1B_avazu_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_tuner_config_03/FinalMLP_avazu_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_1B_avazu_x1/FINAL_1B_avazu_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_tuner_config_03/FinalMLP_avazu_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_tuner_config_03/FinalMLP_avazu_x1_tuner_config_03/model_config.yaml b/ranking/ctr/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_tuner_config_03/FinalMLP_avazu_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_tuner_config_03/FinalMLP_avazu_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_tuner_config_03/FinalMLP_avazu_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_avazu_x1/README.md b/ranking/ctr/FinalMLP/FinalMLP_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_avazu_x1/README.md rename to ranking/ctr/FinalMLP/FinalMLP_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_avazu_x1/environments.txt b/ranking/ctr/FinalMLP/FinalMLP_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_avazu_x1/environments.txt rename to ranking/ctr/FinalMLP/FinalMLP_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_avazu_x1/results.csv b/ranking/ctr/FinalMLP/FinalMLP_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_avazu_x1/results.csv rename to ranking/ctr/FinalMLP/FinalMLP_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_004_d5d36917.log b/ranking/ctr/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_004_d5d36917.log similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_004_d5d36917.log rename to ranking/ctr/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_004_d5d36917.log diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_tuner_config_07.csv b/ranking/ctr/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_tuner_config_07.csv similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_tuner_config_07.csv rename to ranking/ctr/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_tuner_config_07.csv diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_tuner_config_07.yaml b/ranking/ctr/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_tuner_config_07.yaml similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_tuner_config_07.yaml rename to ranking/ctr/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_tuner_config_07.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_1B_criteo_x1/FINAL_1B_criteo_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_tuner_config_07/FinalMLP_criteo_x1_tuner_config_07/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_1B_criteo_x1/FINAL_1B_criteo_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_tuner_config_07/FinalMLP_criteo_x1_tuner_config_07/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_tuner_config_07/FinalMLP_criteo_x1_tuner_config_07/model_config.yaml b/ranking/ctr/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_tuner_config_07/FinalMLP_criteo_x1_tuner_config_07/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_tuner_config_07/FinalMLP_criteo_x1_tuner_config_07/model_config.yaml rename to ranking/ctr/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_tuner_config_07/FinalMLP_criteo_x1_tuner_config_07/model_config.yaml diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_criteo_x1/README.md b/ranking/ctr/FinalMLP/FinalMLP_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_criteo_x1/README.md rename to ranking/ctr/FinalMLP/FinalMLP_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_criteo_x1/environments.txt b/ranking/ctr/FinalMLP/FinalMLP_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_criteo_x1/environments.txt rename to ranking/ctr/FinalMLP/FinalMLP_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_criteo_x1/results.csv b/ranking/ctr/FinalMLP/FinalMLP_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_criteo_x1/results.csv rename to ranking/ctr/FinalMLP/FinalMLP_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_004_e1ab402f.log b/ranking/ctr/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_004_e1ab402f.log similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_004_e1ab402f.log rename to ranking/ctr/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_004_e1ab402f.log diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_tuner_config_16.csv b/ranking/ctr/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_tuner_config_16.csv similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_tuner_config_16.csv rename to ranking/ctr/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_tuner_config_16.csv diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_tuner_config_16.yaml b/ranking/ctr/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_tuner_config_16.yaml similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_tuner_config_16.yaml rename to ranking/ctr/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_tuner_config_16.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_1B_frappe_x1/FINAL_1B_frappe_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_tuner_config_16/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_1B_frappe_x1/FINAL_1B_frappe_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_tuner_config_16/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_tuner_config_16/model_config.yaml b/ranking/ctr/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_tuner_config_16/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_tuner_config_16/model_config.yaml rename to ranking/ctr/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_tuner_config_16/model_config.yaml diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_frappe_x1/README.md b/ranking/ctr/FinalMLP/FinalMLP_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_frappe_x1/README.md rename to ranking/ctr/FinalMLP/FinalMLP_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_frappe_x1/environments.txt b/ranking/ctr/FinalMLP/FinalMLP_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_frappe_x1/environments.txt rename to ranking/ctr/FinalMLP/FinalMLP_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_frappe_x1/results.csv b/ranking/ctr/FinalMLP/FinalMLP_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_frappe_x1/results.csv rename to ranking/ctr/FinalMLP/FinalMLP_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_004_498f3e4f.log b/ranking/ctr/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_004_498f3e4f.log similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_004_498f3e4f.log rename to ranking/ctr/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_004_498f3e4f.log diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_tuner_config_06.csv b/ranking/ctr/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_tuner_config_06.csv similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_tuner_config_06.csv rename to ranking/ctr/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_tuner_config_06.csv diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_tuner_config_06.yaml b/ranking/ctr/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_tuner_config_06.yaml similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_tuner_config_06.yaml rename to ranking/ctr/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_tuner_config_06.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_1B_movielenslatest_x1/FINAL_1B_movielenslatest_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_tuner_config_06/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_1B_movielenslatest_x1/FINAL_1B_movielenslatest_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_tuner_config_06/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_tuner_config_06/model_config.yaml b/ranking/ctr/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_tuner_config_06/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_tuner_config_06/model_config.yaml rename to ranking/ctr/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_tuner_config_06/model_config.yaml diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_movielenslatest_x1/README.md b/ranking/ctr/FinalMLP/FinalMLP_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_movielenslatest_x1/README.md rename to ranking/ctr/FinalMLP/FinalMLP_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_movielenslatest_x1/environments.txt b/ranking/ctr/FinalMLP/FinalMLP_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_movielenslatest_x1/environments.txt rename to ranking/ctr/FinalMLP/FinalMLP_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_movielenslatest_x1/results.csv b/ranking/ctr/FinalMLP/FinalMLP_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_movielenslatest_x1/results.csv rename to ranking/ctr/FinalMLP/FinalMLP_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_1B_avazu_x1/FINAL_1B_avazu_x1_tuner_config_01.csv b/ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_1B_avazu_x1/FINAL_1B_avazu_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_1B_avazu_x1/FINAL_1B_avazu_x1_tuner_config_01.csv rename to ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_1B_avazu_x1/FINAL_1B_avazu_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_1B_avazu_x1/FINAL_1B_avazu_x1_tuner_config_01.yaml b/ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_1B_avazu_x1/FINAL_1B_avazu_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_1B_avazu_x1/FINAL_1B_avazu_x1_tuner_config_01.yaml rename to ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_1B_avazu_x1/FINAL_1B_avazu_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_2B_avazu_x1/FINAL_2B_avazu_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_1B_avazu_x1/FINAL_1B_avazu_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_2B_avazu_x1/FINAL_2B_avazu_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_1B_avazu_x1/FINAL_1B_avazu_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_1B_avazu_x1/FINAL_1B_avazu_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_1B_avazu_x1/FINAL_1B_avazu_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_1B_avazu_x1/FINAL_1B_avazu_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_1B_avazu_x1/FINAL_1B_avazu_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_1B_avazu_x1/FINAL_avazu_x1_030_3ad8933e.log b/ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_1B_avazu_x1/FINAL_avazu_x1_030_3ad8933e.log similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_1B_avazu_x1/FINAL_avazu_x1_030_3ad8933e.log rename to ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_1B_avazu_x1/FINAL_avazu_x1_030_3ad8933e.log diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_1B_avazu_x1/README.md b/ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_1B_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_1B_avazu_x1/README.md rename to ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_1B_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_1B_avazu_x1/environments.txt b/ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_1B_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_1B_avazu_x1/environments.txt rename to ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_1B_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_1B_avazu_x1/results.csv b/ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_1B_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_1B_avazu_x1/results.csv rename to ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_1B_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_2B_avazu_x1/FINAL_2B_avazu_x1_tuner_config_04.csv b/ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_2B_avazu_x1/FINAL_2B_avazu_x1_tuner_config_04.csv similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_2B_avazu_x1/FINAL_2B_avazu_x1_tuner_config_04.csv rename to ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_2B_avazu_x1/FINAL_2B_avazu_x1_tuner_config_04.csv diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_2B_avazu_x1/FINAL_2B_avazu_x1_tuner_config_04.yaml b/ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_2B_avazu_x1/FINAL_2B_avazu_x1_tuner_config_04.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_2B_avazu_x1/FINAL_2B_avazu_x1_tuner_config_04.yaml rename to ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_2B_avazu_x1/FINAL_2B_avazu_x1_tuner_config_04.yaml diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_tuner_config_03/FinalMLP_avazu_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_2B_avazu_x1/FINAL_2B_avazu_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_avazu_x1/FinalMLP_avazu_x1_tuner_config_03/FinalMLP_avazu_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_2B_avazu_x1/FINAL_2B_avazu_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_2B_avazu_x1/FINAL_2B_avazu_x1_tuner_config_04/model_config.yaml b/ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_2B_avazu_x1/FINAL_2B_avazu_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_2B_avazu_x1/FINAL_2B_avazu_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_2B_avazu_x1/FINAL_2B_avazu_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_2B_avazu_x1/FINAL_avazu_x1_023_9d3d37bd.log b/ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_2B_avazu_x1/FINAL_avazu_x1_023_9d3d37bd.log similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_2B_avazu_x1/FINAL_avazu_x1_023_9d3d37bd.log rename to ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_2B_avazu_x1/FINAL_avazu_x1_023_9d3d37bd.log diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_2B_avazu_x1/README.md b/ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_2B_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_2B_avazu_x1/README.md rename to ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_2B_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_2B_avazu_x1/environments.txt b/ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_2B_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_2B_avazu_x1/environments.txt rename to ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_2B_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_2B_avazu_x1/results.csv b/ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_2B_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_avazu_x1/FINAL_2B_avazu_x1/results.csv rename to ranking/ctr/FinalNet/FinalNet_avazu_x1/FinalNet_2B_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_1B_criteo_x1/FINAL_1B_criteo_x1_tuner_config_04.csv b/ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_1B_criteo_x1/FINAL_1B_criteo_x1_tuner_config_04.csv similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_1B_criteo_x1/FINAL_1B_criteo_x1_tuner_config_04.csv rename to ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_1B_criteo_x1/FINAL_1B_criteo_x1_tuner_config_04.csv diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_1B_criteo_x1/FINAL_1B_criteo_x1_tuner_config_04.yaml b/ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_1B_criteo_x1/FINAL_1B_criteo_x1_tuner_config_04.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_1B_criteo_x1/FINAL_1B_criteo_x1_tuner_config_04.yaml rename to ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_1B_criteo_x1/FINAL_1B_criteo_x1_tuner_config_04.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_2B_criteo_x1/FINAL_2B_criteo_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_1B_criteo_x1/FINAL_1B_criteo_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_2B_criteo_x1/FINAL_2B_criteo_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_1B_criteo_x1/FINAL_1B_criteo_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_1B_criteo_x1/FINAL_1B_criteo_x1_tuner_config_04/model_config.yaml b/ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_1B_criteo_x1/FINAL_1B_criteo_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_1B_criteo_x1/FINAL_1B_criteo_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_1B_criteo_x1/FINAL_1B_criteo_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_1B_criteo_x1/FINAL_criteo_x1_007_b4783421.log b/ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_1B_criteo_x1/FINAL_criteo_x1_007_b4783421.log similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_1B_criteo_x1/FINAL_criteo_x1_007_b4783421.log rename to ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_1B_criteo_x1/FINAL_criteo_x1_007_b4783421.log diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_1B_criteo_x1/README.md b/ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_1B_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_1B_criteo_x1/README.md rename to ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_1B_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_1B_criteo_x1/environments.txt b/ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_1B_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_1B_criteo_x1/environments.txt rename to ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_1B_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_1B_criteo_x1/results.csv b/ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_1B_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_1B_criteo_x1/results.csv rename to ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_1B_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_2B_criteo_x1/FINAL_2B_criteo_x1_tuner_config_01.csv b/ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_2B_criteo_x1/FINAL_2B_criteo_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_2B_criteo_x1/FINAL_2B_criteo_x1_tuner_config_01.csv rename to ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_2B_criteo_x1/FINAL_2B_criteo_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_2B_criteo_x1/FINAL_2B_criteo_x1_tuner_config_01.yaml b/ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_2B_criteo_x1/FINAL_2B_criteo_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_2B_criteo_x1/FINAL_2B_criteo_x1_tuner_config_01.yaml rename to ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_2B_criteo_x1/FINAL_2B_criteo_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_tuner_config_07/FinalMLP_criteo_x1_tuner_config_07/dataset_config.yaml b/ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_2B_criteo_x1/FINAL_2B_criteo_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_criteo_x1/FinalMLP_criteo_x1_tuner_config_07/FinalMLP_criteo_x1_tuner_config_07/dataset_config.yaml rename to ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_2B_criteo_x1/FINAL_2B_criteo_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_2B_criteo_x1/FINAL_2B_criteo_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_2B_criteo_x1/FINAL_2B_criteo_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_2B_criteo_x1/FINAL_2B_criteo_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_2B_criteo_x1/FINAL_2B_criteo_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_2B_criteo_x1/FINAL_criteo_x1_028_b6c861e4.log b/ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_2B_criteo_x1/FINAL_criteo_x1_028_b6c861e4.log similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_2B_criteo_x1/FINAL_criteo_x1_028_b6c861e4.log rename to ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_2B_criteo_x1/FINAL_criteo_x1_028_b6c861e4.log diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_2B_criteo_x1/README.md b/ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_2B_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_2B_criteo_x1/README.md rename to ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_2B_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_2B_criteo_x1/environments.txt b/ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_2B_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_2B_criteo_x1/environments.txt rename to ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_2B_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_2B_criteo_x1/results.csv b/ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_2B_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_criteo_x1/FINAL_2B_criteo_x1/results.csv rename to ranking/ctr/FinalNet/FinalNet_criteo_x1/FinalNet_2B_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_1B_frappe_x1/FINAL_1B_frappe_x1_tuner_config_01.csv b/ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_1B_frappe_x1/FINAL_1B_frappe_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_1B_frappe_x1/FINAL_1B_frappe_x1_tuner_config_01.csv rename to ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_1B_frappe_x1/FINAL_1B_frappe_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_1B_frappe_x1/FINAL_1B_frappe_x1_tuner_config_01.yaml b/ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_1B_frappe_x1/FINAL_1B_frappe_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_1B_frappe_x1/FINAL_1B_frappe_x1_tuner_config_01.yaml rename to ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_1B_frappe_x1/FINAL_1B_frappe_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_2B_frappe_x1/FINAL_2B_frappe_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_1B_frappe_x1/FINAL_1B_frappe_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_2B_frappe_x1/FINAL_2B_frappe_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_1B_frappe_x1/FINAL_1B_frappe_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_1B_frappe_x1/FINAL_1B_frappe_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_1B_frappe_x1/FINAL_1B_frappe_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_1B_frappe_x1/FINAL_1B_frappe_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_1B_frappe_x1/FINAL_1B_frappe_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_1B_frappe_x1/FINAL_frappe_x1_006_75be0578.log b/ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_1B_frappe_x1/FINAL_frappe_x1_006_75be0578.log similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_1B_frappe_x1/FINAL_frappe_x1_006_75be0578.log rename to ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_1B_frappe_x1/FINAL_frappe_x1_006_75be0578.log diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_1B_frappe_x1/README.md b/ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_1B_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_1B_frappe_x1/README.md rename to ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_1B_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_1B_frappe_x1/environments.txt b/ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_1B_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_1B_frappe_x1/environments.txt rename to ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_1B_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_1B_frappe_x1/results.csv b/ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_1B_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_1B_frappe_x1/results.csv rename to ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_1B_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_2B_frappe_x1/FINAL_2B_frappe_x1_tuner_config_02.csv b/ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_2B_frappe_x1/FINAL_2B_frappe_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_2B_frappe_x1/FINAL_2B_frappe_x1_tuner_config_02.csv rename to ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_2B_frappe_x1/FINAL_2B_frappe_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_2B_frappe_x1/FINAL_2B_frappe_x1_tuner_config_02.yaml b/ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_2B_frappe_x1/FINAL_2B_frappe_x1_tuner_config_02.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_2B_frappe_x1/FINAL_2B_frappe_x1_tuner_config_02.yaml rename to ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_2B_frappe_x1/FINAL_2B_frappe_x1_tuner_config_02.yaml diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_tuner_config_16/dataset_config.yaml b/ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_2B_frappe_x1/FINAL_2B_frappe_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_frappe_x1/FinalMLP_frappe_x1_tuner_config_16/dataset_config.yaml rename to ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_2B_frappe_x1/FINAL_2B_frappe_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_2B_frappe_x1/FINAL_2B_frappe_x1_tuner_config_02/model_config.yaml b/ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_2B_frappe_x1/FINAL_2B_frappe_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_2B_frappe_x1/FINAL_2B_frappe_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_2B_frappe_x1/FINAL_2B_frappe_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_2B_frappe_x1/FINAL_frappe_x1_003_c3722b71.log b/ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_2B_frappe_x1/FINAL_frappe_x1_003_c3722b71.log similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_2B_frappe_x1/FINAL_frappe_x1_003_c3722b71.log rename to ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_2B_frappe_x1/FINAL_frappe_x1_003_c3722b71.log diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_2B_frappe_x1/README.md b/ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_2B_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_2B_frappe_x1/README.md rename to ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_2B_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_2B_frappe_x1/environments.txt b/ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_2B_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_2B_frappe_x1/environments.txt rename to ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_2B_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_2B_frappe_x1/results.csv b/ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_2B_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_frappe_x1/FINAL_2B_frappe_x1/results.csv rename to ranking/ctr/FinalNet/FinalNet_frappe_x1/FinalNet_2B_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_1B_movielenslatest_x1/FINAL_1B_movielenslatest_x1_tuner_config_02.csv b/ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_1B_movielenslatest_x1/FINAL_1B_movielenslatest_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_1B_movielenslatest_x1/FINAL_1B_movielenslatest_x1_tuner_config_02.csv rename to ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_1B_movielenslatest_x1/FINAL_1B_movielenslatest_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_1B_movielenslatest_x1/FINAL_1B_movielenslatest_x1_tuner_config_02.yaml b/ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_1B_movielenslatest_x1/FINAL_1B_movielenslatest_x1_tuner_config_02.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_1B_movielenslatest_x1/FINAL_1B_movielenslatest_x1_tuner_config_02.yaml rename to ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_1B_movielenslatest_x1/FINAL_1B_movielenslatest_x1_tuner_config_02.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_2B_movielenslatest_x1/FINAL_2B_movielenslatest_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_1B_movielenslatest_x1/FINAL_1B_movielenslatest_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_2B_movielenslatest_x1/FINAL_2B_movielenslatest_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_1B_movielenslatest_x1/FINAL_1B_movielenslatest_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_1B_movielenslatest_x1/FINAL_1B_movielenslatest_x1_tuner_config_02/model_config.yaml b/ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_1B_movielenslatest_x1/FINAL_1B_movielenslatest_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_1B_movielenslatest_x1/FINAL_1B_movielenslatest_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_1B_movielenslatest_x1/FINAL_1B_movielenslatest_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_1B_movielenslatest_x1/FINAL_movielenslatest_x1_008_f8e20ee7.log b/ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_1B_movielenslatest_x1/FINAL_movielenslatest_x1_008_f8e20ee7.log similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_1B_movielenslatest_x1/FINAL_movielenslatest_x1_008_f8e20ee7.log rename to ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_1B_movielenslatest_x1/FINAL_movielenslatest_x1_008_f8e20ee7.log diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_1B_movielenslatest_x1/README.md b/ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_1B_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_1B_movielenslatest_x1/README.md rename to ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_1B_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_1B_movielenslatest_x1/environments.txt b/ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_1B_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_1B_movielenslatest_x1/environments.txt rename to ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_1B_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_1B_movielenslatest_x1/results.csv b/ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_1B_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_1B_movielenslatest_x1/results.csv rename to ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_1B_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_2B_movielenslatest_x1/FINAL_2B_movielenslatest_x1_tuner_config_03.csv b/ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_2B_movielenslatest_x1/FINAL_2B_movielenslatest_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_2B_movielenslatest_x1/FINAL_2B_movielenslatest_x1_tuner_config_03.csv rename to ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_2B_movielenslatest_x1/FINAL_2B_movielenslatest_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_2B_movielenslatest_x1/FINAL_2B_movielenslatest_x1_tuner_config_03.yaml b/ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_2B_movielenslatest_x1/FINAL_2B_movielenslatest_x1_tuner_config_03.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_2B_movielenslatest_x1/FINAL_2B_movielenslatest_x1_tuner_config_03.yaml rename to ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_2B_movielenslatest_x1/FINAL_2B_movielenslatest_x1_tuner_config_03.yaml diff --git a/ctr_prediction/benchmarks/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_tuner_config_06/dataset_config.yaml b/ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_2B_movielenslatest_x1/FINAL_2B_movielenslatest_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FinalMLP/FinalMLP_movielenslatest_x1/FinalMLP_movielenslatest_x1_tuner_config_06/dataset_config.yaml rename to ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_2B_movielenslatest_x1/FINAL_2B_movielenslatest_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_2B_movielenslatest_x1/FINAL_2B_movielenslatest_x1_tuner_config_03/model_config.yaml b/ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_2B_movielenslatest_x1/FINAL_2B_movielenslatest_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_2B_movielenslatest_x1/FINAL_2B_movielenslatest_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_2B_movielenslatest_x1/FINAL_2B_movielenslatest_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_2B_movielenslatest_x1/FINAL_movielenslatest_x1_014_50fc5f7f.log b/ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_2B_movielenslatest_x1/FINAL_movielenslatest_x1_014_50fc5f7f.log similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_2B_movielenslatest_x1/FINAL_movielenslatest_x1_014_50fc5f7f.log rename to ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_2B_movielenslatest_x1/FINAL_movielenslatest_x1_014_50fc5f7f.log diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_2B_movielenslatest_x1/README.md b/ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_2B_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_2B_movielenslatest_x1/README.md rename to ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_2B_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_2B_movielenslatest_x1/environments.txt b/ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_2B_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_2B_movielenslatest_x1/environments.txt rename to ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_2B_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_2B_movielenslatest_x1/results.csv b/ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_2B_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FINAL/FINAL_movielenslatest_x1/FINAL_2B_movielenslatest_x1/results.csv rename to ranking/ctr/FinalNet/FinalNet_movielenslatest_x1/FinalNet_2B_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_011_eb015129.log b/ranking/ctr/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_011_eb015129.log similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_011_eb015129.log rename to ranking/ctr/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_011_eb015129.log diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_tuner_config_01.csv b/ranking/ctr/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_tuner_config_01.csv rename to ranking/ctr/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_tuner_config_01.yaml b/ranking/ctr/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_tuner_config_01.yaml rename to ranking/ctr/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FmFM/FmFM_amazonelectronics_x1/FmFM_amazonelectronics_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_amazonelectronics_x1/README.md b/ranking/ctr/FmFM/FmFM_amazonelectronics_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_amazonelectronics_x1/README.md rename to ranking/ctr/FmFM/FmFM_amazonelectronics_x1/README.md diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_amazonelectronics_x1/environments.txt b/ranking/ctr/FmFM/FmFM_amazonelectronics_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_amazonelectronics_x1/environments.txt rename to ranking/ctr/FmFM/FmFM_amazonelectronics_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_amazonelectronics_x1/results.csv b/ranking/ctr/FmFM/FmFM_amazonelectronics_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_amazonelectronics_x1/results.csv rename to ranking/ctr/FmFM/FmFM_amazonelectronics_x1/results.csv diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_avazu_x1/FmFM_avazu_x1_002_a09a4da9.log b/ranking/ctr/FmFM/FmFM_avazu_x1/FmFM_avazu_x1_002_a09a4da9.log similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_avazu_x1/FmFM_avazu_x1_002_a09a4da9.log rename to ranking/ctr/FmFM/FmFM_avazu_x1/FmFM_avazu_x1_002_a09a4da9.log diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_avazu_x1/FmFM_avazu_x1_tuner_config_01.csv b/ranking/ctr/FmFM/FmFM_avazu_x1/FmFM_avazu_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_avazu_x1/FmFM_avazu_x1_tuner_config_01.csv rename to ranking/ctr/FmFM/FmFM_avazu_x1/FmFM_avazu_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_avazu_x1/FmFM_avazu_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FmFM/FmFM_avazu_x1/FmFM_avazu_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_avazu_x1/FmFM_avazu_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FmFM/FmFM_avazu_x1/FmFM_avazu_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_avazu_x1/FmFM_avazu_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FmFM/FmFM_avazu_x1/FmFM_avazu_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_avazu_x1/FmFM_avazu_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FmFM/FmFM_avazu_x1/FmFM_avazu_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_avazu_x1/README.md b/ranking/ctr/FmFM/FmFM_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_avazu_x1/README.md rename to ranking/ctr/FmFM/FmFM_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_avazu_x1/environments.txt b/ranking/ctr/FmFM/FmFM_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_avazu_x1/environments.txt rename to ranking/ctr/FmFM/FmFM_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_avazu_x1/results.csv b/ranking/ctr/FmFM/FmFM_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_avazu_x1/results.csv rename to ranking/ctr/FmFM/FmFM_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_avazu_x1/run_expid.py b/ranking/ctr/FmFM/FmFM_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_avazu_x1/run_expid.py rename to ranking/ctr/FmFM/FmFM_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_criteo_x1/FmFM_criteo_x1_001_9ba4938b.log b/ranking/ctr/FmFM/FmFM_criteo_x1/FmFM_criteo_x1_001_9ba4938b.log similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_criteo_x1/FmFM_criteo_x1_001_9ba4938b.log rename to ranking/ctr/FmFM/FmFM_criteo_x1/FmFM_criteo_x1_001_9ba4938b.log diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_criteo_x1/FmFM_criteo_x1_tuner_config_01.csv b/ranking/ctr/FmFM/FmFM_criteo_x1/FmFM_criteo_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_criteo_x1/FmFM_criteo_x1_tuner_config_01.csv rename to ranking/ctr/FmFM/FmFM_criteo_x1/FmFM_criteo_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_criteo_x1/FmFM_criteo_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FmFM/FmFM_criteo_x1/FmFM_criteo_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_criteo_x1/FmFM_criteo_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FmFM/FmFM_criteo_x1/FmFM_criteo_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_criteo_x1/FmFM_criteo_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FmFM/FmFM_criteo_x1/FmFM_criteo_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_criteo_x1/FmFM_criteo_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FmFM/FmFM_criteo_x1/FmFM_criteo_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_criteo_x1/README.md b/ranking/ctr/FmFM/FmFM_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_criteo_x1/README.md rename to ranking/ctr/FmFM/FmFM_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_criteo_x1/environments.txt b/ranking/ctr/FmFM/FmFM_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_criteo_x1/environments.txt rename to ranking/ctr/FmFM/FmFM_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_criteo_x1/results.csv b/ranking/ctr/FmFM/FmFM_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_criteo_x1/results.csv rename to ranking/ctr/FmFM/FmFM_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_criteo_x1/run_expid.py b/ranking/ctr/FmFM/FmFM_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_criteo_x1/run_expid.py rename to ranking/ctr/FmFM/FmFM_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_frappe_x1/FmFM_frappe_x1_004_2445bd6f.log b/ranking/ctr/FmFM/FmFM_frappe_x1/FmFM_frappe_x1_004_2445bd6f.log similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_frappe_x1/FmFM_frappe_x1_004_2445bd6f.log rename to ranking/ctr/FmFM/FmFM_frappe_x1/FmFM_frappe_x1_004_2445bd6f.log diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_frappe_x1/FmFM_frappe_x1_tuner_config_02.csv b/ranking/ctr/FmFM/FmFM_frappe_x1/FmFM_frappe_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_frappe_x1/FmFM_frappe_x1_tuner_config_02.csv rename to ranking/ctr/FmFM/FmFM_frappe_x1/FmFM_frappe_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_frappe_x1/FmFM_frappe_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/FmFM/FmFM_frappe_x1/FmFM_frappe_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_frappe_x1/FmFM_frappe_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FmFM/FmFM_frappe_x1/FmFM_frappe_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_frappe_x1/FmFM_frappe_x1_tuner_config_02/model_config.yaml b/ranking/ctr/FmFM/FmFM_frappe_x1/FmFM_frappe_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_frappe_x1/FmFM_frappe_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/FmFM/FmFM_frappe_x1/FmFM_frappe_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_frappe_x1/README.md b/ranking/ctr/FmFM/FmFM_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_frappe_x1/README.md rename to ranking/ctr/FmFM/FmFM_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_frappe_x1/environments.txt b/ranking/ctr/FmFM/FmFM_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_frappe_x1/environments.txt rename to ranking/ctr/FmFM/FmFM_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_frappe_x1/results.csv b/ranking/ctr/FmFM/FmFM_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_frappe_x1/results.csv rename to ranking/ctr/FmFM/FmFM_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_frappe_x1/run_expid.py b/ranking/ctr/FmFM/FmFM_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_frappe_x1/run_expid.py rename to ranking/ctr/FmFM/FmFM_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_007_a04f176e.log b/ranking/ctr/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_007_a04f176e.log similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_007_a04f176e.log rename to ranking/ctr/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_007_a04f176e.log diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_tuner_config_02.csv b/ranking/ctr/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_tuner_config_02.csv rename to ranking/ctr/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_tuner_config_02.yaml b/ranking/ctr/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_tuner_config_02.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_tuner_config_02.yaml rename to ranking/ctr/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_tuner_config_02.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_tuner_config_02/model_config.yaml b/ranking/ctr/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/FmFM/FmFM_kuaivideo_x1/FmFM_kuaivideo_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_kuaivideo_x1/README.md b/ranking/ctr/FmFM/FmFM_kuaivideo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_kuaivideo_x1/README.md rename to ranking/ctr/FmFM/FmFM_kuaivideo_x1/README.md diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_kuaivideo_x1/environments.txt b/ranking/ctr/FmFM/FmFM_kuaivideo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_kuaivideo_x1/environments.txt rename to ranking/ctr/FmFM/FmFM_kuaivideo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_kuaivideo_x1/results.csv b/ranking/ctr/FmFM/FmFM_kuaivideo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_kuaivideo_x1/results.csv rename to ranking/ctr/FmFM/FmFM_kuaivideo_x1/results.csv diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_008_b522838a.log b/ranking/ctr/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_008_b522838a.log similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_008_b522838a.log rename to ranking/ctr/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_008_b522838a.log diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_tuner_config_03.csv b/ranking/ctr/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_tuner_config_03.csv rename to ranking/ctr/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_tuner_config_03.yaml b/ranking/ctr/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_tuner_config_03.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_tuner_config_03.yaml rename to ranking/ctr/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_tuner_config_03.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_tuner_config_03/model_config.yaml b/ranking/ctr/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/FmFM/FmFM_microvideo1.7m_x1/FmFM_microvideo1.7m_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_microvideo1.7m_x1/README.md b/ranking/ctr/FmFM/FmFM_microvideo1.7m_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_microvideo1.7m_x1/README.md rename to ranking/ctr/FmFM/FmFM_microvideo1.7m_x1/README.md diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_microvideo1.7m_x1/environments.txt b/ranking/ctr/FmFM/FmFM_microvideo1.7m_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_microvideo1.7m_x1/environments.txt rename to ranking/ctr/FmFM/FmFM_microvideo1.7m_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_microvideo1.7m_x1/results.csv b/ranking/ctr/FmFM/FmFM_microvideo1.7m_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_microvideo1.7m_x1/results.csv rename to ranking/ctr/FmFM/FmFM_microvideo1.7m_x1/results.csv diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_movielenslatest_x1/FmFM_movielenslatest_x1_003_28d4fd9f.log b/ranking/ctr/FmFM/FmFM_movielenslatest_x1/FmFM_movielenslatest_x1_003_28d4fd9f.log similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_movielenslatest_x1/FmFM_movielenslatest_x1_003_28d4fd9f.log rename to ranking/ctr/FmFM/FmFM_movielenslatest_x1/FmFM_movielenslatest_x1_003_28d4fd9f.log diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_movielenslatest_x1/FmFM_movielenslatest_x1_tuner_config_03.csv b/ranking/ctr/FmFM/FmFM_movielenslatest_x1/FmFM_movielenslatest_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_movielenslatest_x1/FmFM_movielenslatest_x1_tuner_config_03.csv rename to ranking/ctr/FmFM/FmFM_movielenslatest_x1/FmFM_movielenslatest_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_movielenslatest_x1/FmFM_movielenslatest_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/FmFM/FmFM_movielenslatest_x1/FmFM_movielenslatest_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_movielenslatest_x1/FmFM_movielenslatest_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/FmFM/FmFM_movielenslatest_x1/FmFM_movielenslatest_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_movielenslatest_x1/FmFM_movielenslatest_x1_tuner_config_03/model_config.yaml b/ranking/ctr/FmFM/FmFM_movielenslatest_x1/FmFM_movielenslatest_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_movielenslatest_x1/FmFM_movielenslatest_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/FmFM/FmFM_movielenslatest_x1/FmFM_movielenslatest_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_movielenslatest_x1/README.md b/ranking/ctr/FmFM/FmFM_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_movielenslatest_x1/README.md rename to ranking/ctr/FmFM/FmFM_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_movielenslatest_x1/environments.txt b/ranking/ctr/FmFM/FmFM_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_movielenslatest_x1/environments.txt rename to ranking/ctr/FmFM/FmFM_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_movielenslatest_x1/results.csv b/ranking/ctr/FmFM/FmFM_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_movielenslatest_x1/results.csv rename to ranking/ctr/FmFM/FmFM_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_movielenslatest_x1/run_expid.py b/ranking/ctr/FmFM/FmFM_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_movielenslatest_x1/run_expid.py rename to ranking/ctr/FmFM/FmFM_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_012_d3d0ca94.log b/ranking/ctr/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_012_d3d0ca94.log similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_012_d3d0ca94.log rename to ranking/ctr/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_012_d3d0ca94.log diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_tuner_config_01.csv b/ranking/ctr/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_tuner_config_01.csv rename to ranking/ctr/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_tuner_config_01.yaml b/ranking/ctr/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_tuner_config_01.yaml rename to ranking/ctr/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FmFM/FmFM_taobaoad_x1/FmFM_taobaoad_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_taobaoad_x1/README.md b/ranking/ctr/FmFM/FmFM_taobaoad_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_taobaoad_x1/README.md rename to ranking/ctr/FmFM/FmFM_taobaoad_x1/README.md diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_taobaoad_x1/environments.txt b/ranking/ctr/FmFM/FmFM_taobaoad_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_taobaoad_x1/environments.txt rename to ranking/ctr/FmFM/FmFM_taobaoad_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FmFM/FmFM_taobaoad_x1/results.csv b/ranking/ctr/FmFM/FmFM_taobaoad_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FmFM/FmFM_taobaoad_x1/results.csv rename to ranking/ctr/FmFM/FmFM_taobaoad_x1/results.csv diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x1/FwFM_avazu_x1_009_b8820aaa.log b/ranking/ctr/FwFM/FwFM_avazu_x1/FwFM_avazu_x1_009_b8820aaa.log similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x1/FwFM_avazu_x1_009_b8820aaa.log rename to ranking/ctr/FwFM/FwFM_avazu_x1/FwFM_avazu_x1_009_b8820aaa.log diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x1/FwFM_avazu_x1_tuner_config_01.csv b/ranking/ctr/FwFM/FwFM_avazu_x1/FwFM_avazu_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x1/FwFM_avazu_x1_tuner_config_01.csv rename to ranking/ctr/FwFM/FwFM_avazu_x1/FwFM_avazu_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x1/FwFM_avazu_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FwFM/FwFM_avazu_x1/FwFM_avazu_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x1/FwFM_avazu_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FwFM/FwFM_avazu_x1/FwFM_avazu_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x1/FwFM_avazu_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FwFM/FwFM_avazu_x1/FwFM_avazu_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x1/FwFM_avazu_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FwFM/FwFM_avazu_x1/FwFM_avazu_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x1/README.md b/ranking/ctr/FwFM/FwFM_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x1/README.md rename to ranking/ctr/FwFM/FwFM_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x1/environments.txt b/ranking/ctr/FwFM/FwFM_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x1/environments.txt rename to ranking/ctr/FwFM/FwFM_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x1/results.csv b/ranking/ctr/FwFM/FwFM_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x1/results.csv rename to ranking/ctr/FwFM/FwFM_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x1/run_expid.py b/ranking/ctr/FwFM/FwFM_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x1/run_expid.py rename to ranking/ctr/FwFM/FwFM_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_001/FwFM_avazu_x4_012_3a5f1b67.log b/ranking/ctr/FwFM/FwFM_avazu_x4_001/FwFM_avazu_x4_012_3a5f1b67.log similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_001/FwFM_avazu_x4_012_3a5f1b67.log rename to ranking/ctr/FwFM/FwFM_avazu_x4_001/FwFM_avazu_x4_012_3a5f1b67.log diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_001/FwFM_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/FwFM/FwFM_avazu_x4_001/FwFM_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_001/FwFM_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FwFM/FwFM_avazu_x4_001/FwFM_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_001/FwFM_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/FwFM/FwFM_avazu_x4_001/FwFM_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_001/FwFM_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/FwFM/FwFM_avazu_x4_001/FwFM_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_001/README.md b/ranking/ctr/FwFM/FwFM_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_001/README.md rename to ranking/ctr/FwFM/FwFM_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_001/environments.txt b/ranking/ctr/FwFM/FwFM_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_001/environments.txt rename to ranking/ctr/FwFM/FwFM_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_001/results.csv b/ranking/ctr/FwFM/FwFM_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_001/results.csv rename to ranking/ctr/FwFM/FwFM_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_001/run_expid.py b/ranking/ctr/FwFM/FwFM_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_001/run_expid.py rename to ranking/ctr/FwFM/FwFM_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_002/FwFM_avazu_x4_003_9c0d0ea9.log b/ranking/ctr/FwFM/FwFM_avazu_x4_002/FwFM_avazu_x4_003_9c0d0ea9.log similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_002/FwFM_avazu_x4_003_9c0d0ea9.log rename to ranking/ctr/FwFM/FwFM_avazu_x4_002/FwFM_avazu_x4_003_9c0d0ea9.log diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_002/FwFM_avazu_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/FwFM/FwFM_avazu_x4_002/FwFM_avazu_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_002/FwFM_avazu_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FwFM/FwFM_avazu_x4_002/FwFM_avazu_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_002/FwFM_avazu_x4_tuner_config_02/model_config.yaml b/ranking/ctr/FwFM/FwFM_avazu_x4_002/FwFM_avazu_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_002/FwFM_avazu_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/FwFM/FwFM_avazu_x4_002/FwFM_avazu_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_002/README.md b/ranking/ctr/FwFM/FwFM_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_002/README.md rename to ranking/ctr/FwFM/FwFM_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_002/environments.txt b/ranking/ctr/FwFM/FwFM_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_002/environments.txt rename to ranking/ctr/FwFM/FwFM_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_002/results.csv b/ranking/ctr/FwFM/FwFM_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_002/results.csv rename to ranking/ctr/FwFM/FwFM_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_002/run_expid.py b/ranking/ctr/FwFM/FwFM_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_avazu_x4_002/run_expid.py rename to ranking/ctr/FwFM/FwFM_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x1/FwFM_criteo_x1_009_ff52575c.log b/ranking/ctr/FwFM/FwFM_criteo_x1/FwFM_criteo_x1_009_ff52575c.log similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x1/FwFM_criteo_x1_009_ff52575c.log rename to ranking/ctr/FwFM/FwFM_criteo_x1/FwFM_criteo_x1_009_ff52575c.log diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x1/FwFM_criteo_x1_tuner_config_01.csv b/ranking/ctr/FwFM/FwFM_criteo_x1/FwFM_criteo_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x1/FwFM_criteo_x1_tuner_config_01.csv rename to ranking/ctr/FwFM/FwFM_criteo_x1/FwFM_criteo_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x1/FwFM_criteo_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FwFM/FwFM_criteo_x1/FwFM_criteo_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x1/FwFM_criteo_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FwFM/FwFM_criteo_x1/FwFM_criteo_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x1/FwFM_criteo_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FwFM/FwFM_criteo_x1/FwFM_criteo_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x1/FwFM_criteo_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FwFM/FwFM_criteo_x1/FwFM_criteo_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x1/README.md b/ranking/ctr/FwFM/FwFM_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x1/README.md rename to ranking/ctr/FwFM/FwFM_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x1/environments.txt b/ranking/ctr/FwFM/FwFM_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x1/environments.txt rename to ranking/ctr/FwFM/FwFM_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x1/results.csv b/ranking/ctr/FwFM/FwFM_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x1/results.csv rename to ranking/ctr/FwFM/FwFM_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x1/run_expid.py b/ranking/ctr/FwFM/FwFM_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x1/run_expid.py rename to ranking/ctr/FwFM/FwFM_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_001/FwFM_criteo_x4_002_3519edbe.log b/ranking/ctr/FwFM/FwFM_criteo_x4_001/FwFM_criteo_x4_002_3519edbe.log similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_001/FwFM_criteo_x4_002_3519edbe.log rename to ranking/ctr/FwFM/FwFM_criteo_x4_001/FwFM_criteo_x4_002_3519edbe.log diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_001/FwFM_criteo_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/FwFM/FwFM_criteo_x4_001/FwFM_criteo_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_001/FwFM_criteo_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FwFM/FwFM_criteo_x4_001/FwFM_criteo_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_001/FwFM_criteo_x4_tuner_config_02/model_config.yaml b/ranking/ctr/FwFM/FwFM_criteo_x4_001/FwFM_criteo_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_001/FwFM_criteo_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/FwFM/FwFM_criteo_x4_001/FwFM_criteo_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_001/README.md b/ranking/ctr/FwFM/FwFM_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_001/README.md rename to ranking/ctr/FwFM/FwFM_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_001/environments.txt b/ranking/ctr/FwFM/FwFM_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_001/environments.txt rename to ranking/ctr/FwFM/FwFM_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_001/results.csv b/ranking/ctr/FwFM/FwFM_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_001/results.csv rename to ranking/ctr/FwFM/FwFM_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_001/run_expid.py b/ranking/ctr/FwFM/FwFM_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_001/run_expid.py rename to ranking/ctr/FwFM/FwFM_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_002/FwFM_criteo_x4_001_fc1f0a5d.log b/ranking/ctr/FwFM/FwFM_criteo_x4_002/FwFM_criteo_x4_001_fc1f0a5d.log similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_002/FwFM_criteo_x4_001_fc1f0a5d.log rename to ranking/ctr/FwFM/FwFM_criteo_x4_002/FwFM_criteo_x4_001_fc1f0a5d.log diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_002/FwFM_criteo_x4_tuner_config_01.yaml b/ranking/ctr/FwFM/FwFM_criteo_x4_002/FwFM_criteo_x4_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_002/FwFM_criteo_x4_tuner_config_01.yaml rename to ranking/ctr/FwFM/FwFM_criteo_x4_002/FwFM_criteo_x4_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_002/FwFM_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/FwFM/FwFM_criteo_x4_002/FwFM_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_002/FwFM_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FwFM/FwFM_criteo_x4_002/FwFM_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_002/FwFM_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/FwFM/FwFM_criteo_x4_002/FwFM_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_002/FwFM_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/FwFM/FwFM_criteo_x4_002/FwFM_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_002/README.md b/ranking/ctr/FwFM/FwFM_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_002/README.md rename to ranking/ctr/FwFM/FwFM_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_002/environments.txt b/ranking/ctr/FwFM/FwFM_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_002/environments.txt rename to ranking/ctr/FwFM/FwFM_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_002/results.csv b/ranking/ctr/FwFM/FwFM_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_002/results.csv rename to ranking/ctr/FwFM/FwFM_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_002/run_expid.py b/ranking/ctr/FwFM/FwFM_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_criteo_x4_002/run_expid.py rename to ranking/ctr/FwFM/FwFM_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_frappe_x1/FwFM_frappe_x1_005_95cf3ccd.log b/ranking/ctr/FwFM/FwFM_frappe_x1/FwFM_frappe_x1_005_95cf3ccd.log similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_frappe_x1/FwFM_frappe_x1_005_95cf3ccd.log rename to ranking/ctr/FwFM/FwFM_frappe_x1/FwFM_frappe_x1_005_95cf3ccd.log diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_frappe_x1/FwFM_frappe_x1_tuner_config_02.csv b/ranking/ctr/FwFM/FwFM_frappe_x1/FwFM_frappe_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_frappe_x1/FwFM_frappe_x1_tuner_config_02.csv rename to ranking/ctr/FwFM/FwFM_frappe_x1/FwFM_frappe_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_frappe_x1/FwFM_frappe_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/FwFM/FwFM_frappe_x1/FwFM_frappe_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_frappe_x1/FwFM_frappe_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FwFM/FwFM_frappe_x1/FwFM_frappe_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_frappe_x1/FwFM_frappe_x1_tuner_config_02/model_config.yaml b/ranking/ctr/FwFM/FwFM_frappe_x1/FwFM_frappe_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_frappe_x1/FwFM_frappe_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/FwFM/FwFM_frappe_x1/FwFM_frappe_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_frappe_x1/README.md b/ranking/ctr/FwFM/FwFM_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_frappe_x1/README.md rename to ranking/ctr/FwFM/FwFM_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_frappe_x1/environments.txt b/ranking/ctr/FwFM/FwFM_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_frappe_x1/environments.txt rename to ranking/ctr/FwFM/FwFM_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_frappe_x1/results.csv b/ranking/ctr/FwFM/FwFM_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_frappe_x1/results.csv rename to ranking/ctr/FwFM/FwFM_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_frappe_x1/run_expid.py b/ranking/ctr/FwFM/FwFM_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_frappe_x1/run_expid.py rename to ranking/ctr/FwFM/FwFM_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_kkbox_x1/FwFM_kkbox_x1_006_d4ec0630.log b/ranking/ctr/FwFM/FwFM_kkbox_x1/FwFM_kkbox_x1_006_d4ec0630.log similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_kkbox_x1/FwFM_kkbox_x1_006_d4ec0630.log rename to ranking/ctr/FwFM/FwFM_kkbox_x1/FwFM_kkbox_x1_006_d4ec0630.log diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_kkbox_x1/FwFM_kkbox_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/FwFM/FwFM_kkbox_x1/FwFM_kkbox_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_kkbox_x1/FwFM_kkbox_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/FwFM/FwFM_kkbox_x1/FwFM_kkbox_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_kkbox_x1/FwFM_kkbox_x1_tuner_config_01/model_config.yaml b/ranking/ctr/FwFM/FwFM_kkbox_x1/FwFM_kkbox_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_kkbox_x1/FwFM_kkbox_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/FwFM/FwFM_kkbox_x1/FwFM_kkbox_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_kkbox_x1/README.md b/ranking/ctr/FwFM/FwFM_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_kkbox_x1/README.md rename to ranking/ctr/FwFM/FwFM_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_kkbox_x1/environments.txt b/ranking/ctr/FwFM/FwFM_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_kkbox_x1/environments.txt rename to ranking/ctr/FwFM/FwFM_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_kkbox_x1/results.csv b/ranking/ctr/FwFM/FwFM_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_kkbox_x1/results.csv rename to ranking/ctr/FwFM/FwFM_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_kkbox_x1/run_expid.py b/ranking/ctr/FwFM/FwFM_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_kkbox_x1/run_expid.py rename to ranking/ctr/FwFM/FwFM_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_movielenslatest_x1/FwFM_movielenslatest_x1_006_e527bbd6.log b/ranking/ctr/FwFM/FwFM_movielenslatest_x1/FwFM_movielenslatest_x1_006_e527bbd6.log similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_movielenslatest_x1/FwFM_movielenslatest_x1_006_e527bbd6.log rename to ranking/ctr/FwFM/FwFM_movielenslatest_x1/FwFM_movielenslatest_x1_006_e527bbd6.log diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_movielenslatest_x1/FwFM_movielenslatest_x1_tuner_config_02.csv b/ranking/ctr/FwFM/FwFM_movielenslatest_x1/FwFM_movielenslatest_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_movielenslatest_x1/FwFM_movielenslatest_x1_tuner_config_02.csv rename to ranking/ctr/FwFM/FwFM_movielenslatest_x1/FwFM_movielenslatest_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_movielenslatest_x1/FwFM_movielenslatest_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/FwFM/FwFM_movielenslatest_x1/FwFM_movielenslatest_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_movielenslatest_x1/FwFM_movielenslatest_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/FwFM/FwFM_movielenslatest_x1/FwFM_movielenslatest_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_movielenslatest_x1/FwFM_movielenslatest_x1_tuner_config_02/model_config.yaml b/ranking/ctr/FwFM/FwFM_movielenslatest_x1/FwFM_movielenslatest_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_movielenslatest_x1/FwFM_movielenslatest_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/FwFM/FwFM_movielenslatest_x1/FwFM_movielenslatest_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_movielenslatest_x1/README.md b/ranking/ctr/FwFM/FwFM_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_movielenslatest_x1/README.md rename to ranking/ctr/FwFM/FwFM_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_movielenslatest_x1/environments.txt b/ranking/ctr/FwFM/FwFM_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_movielenslatest_x1/environments.txt rename to ranking/ctr/FwFM/FwFM_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_movielenslatest_x1/results.csv b/ranking/ctr/FwFM/FwFM_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_movielenslatest_x1/results.csv rename to ranking/ctr/FwFM/FwFM_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/FwFM/FwFM_movielenslatest_x1/run_expid.py b/ranking/ctr/FwFM/FwFM_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/FwFM/FwFM_movielenslatest_x1/run_expid.py rename to ranking/ctr/FwFM/FwFM_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x1/HFM+_avazu_x1_tuner_config_05.csv b/ranking/ctr/HFM/HFM+_avazu_x1/HFM+_avazu_x1_tuner_config_05.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x1/HFM+_avazu_x1_tuner_config_05.csv rename to ranking/ctr/HFM/HFM+_avazu_x1/HFM+_avazu_x1_tuner_config_05.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x1/HFM+_avazu_x1_tuner_config_05/dataset_config.yaml b/ranking/ctr/HFM/HFM+_avazu_x1/HFM+_avazu_x1_tuner_config_05/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x1/HFM+_avazu_x1_tuner_config_05/dataset_config.yaml rename to ranking/ctr/HFM/HFM+_avazu_x1/HFM+_avazu_x1_tuner_config_05/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x1/HFM+_avazu_x1_tuner_config_05/model_config.yaml b/ranking/ctr/HFM/HFM+_avazu_x1/HFM+_avazu_x1_tuner_config_05/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x1/HFM+_avazu_x1_tuner_config_05/model_config.yaml rename to ranking/ctr/HFM/HFM+_avazu_x1/HFM+_avazu_x1_tuner_config_05/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x1/HFM_avazu_x1_002_8a26975b.log b/ranking/ctr/HFM/HFM+_avazu_x1/HFM_avazu_x1_002_8a26975b.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x1/HFM_avazu_x1_002_8a26975b.log rename to ranking/ctr/HFM/HFM+_avazu_x1/HFM_avazu_x1_002_8a26975b.log diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x1/README.md b/ranking/ctr/HFM/HFM+_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x1/README.md rename to ranking/ctr/HFM/HFM+_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x1/environments.txt b/ranking/ctr/HFM/HFM+_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x1/environments.txt rename to ranking/ctr/HFM/HFM+_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x1/results.csv b/ranking/ctr/HFM/HFM+_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x1/results.csv rename to ranking/ctr/HFM/HFM+_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x1/run_expid.py b/ranking/ctr/HFM/HFM+_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x1/run_expid.py rename to ranking/ctr/HFM/HFM+_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_001/HFM+_avazu_x4_tuner_config_12/dataset_config.yaml b/ranking/ctr/HFM/HFM+_avazu_x4_001/HFM+_avazu_x4_tuner_config_12/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_001/HFM+_avazu_x4_tuner_config_12/dataset_config.yaml rename to ranking/ctr/HFM/HFM+_avazu_x4_001/HFM+_avazu_x4_tuner_config_12/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_001/HFM+_avazu_x4_tuner_config_12/model_config.yaml b/ranking/ctr/HFM/HFM+_avazu_x4_001/HFM+_avazu_x4_tuner_config_12/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_001/HFM+_avazu_x4_tuner_config_12/model_config.yaml rename to ranking/ctr/HFM/HFM+_avazu_x4_001/HFM+_avazu_x4_tuner_config_12/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_001/HFM_avazu_x4_005_8f97f15f.log b/ranking/ctr/HFM/HFM+_avazu_x4_001/HFM_avazu_x4_005_8f97f15f.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_001/HFM_avazu_x4_005_8f97f15f.log rename to ranking/ctr/HFM/HFM+_avazu_x4_001/HFM_avazu_x4_005_8f97f15f.log diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_001/README.md b/ranking/ctr/HFM/HFM+_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_001/README.md rename to ranking/ctr/HFM/HFM+_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_001/environments.txt b/ranking/ctr/HFM/HFM+_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_001/environments.txt rename to ranking/ctr/HFM/HFM+_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_001/results.csv b/ranking/ctr/HFM/HFM+_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_001/results.csv rename to ranking/ctr/HFM/HFM+_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_001/run_expid.py b/ranking/ctr/HFM/HFM+_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_001/run_expid.py rename to ranking/ctr/HFM/HFM+_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_002/HFM+_avazu_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/HFM/HFM+_avazu_x4_002/HFM+_avazu_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_002/HFM+_avazu_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/HFM/HFM+_avazu_x4_002/HFM+_avazu_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_002/HFM+_avazu_x4_tuner_config_02/model_config.yaml b/ranking/ctr/HFM/HFM+_avazu_x4_002/HFM+_avazu_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_002/HFM+_avazu_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/HFM/HFM+_avazu_x4_002/HFM+_avazu_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_002/HFM_avazu_x4_001_6f0738f0.log b/ranking/ctr/HFM/HFM+_avazu_x4_002/HFM_avazu_x4_001_6f0738f0.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_002/HFM_avazu_x4_001_6f0738f0.log rename to ranking/ctr/HFM/HFM+_avazu_x4_002/HFM_avazu_x4_001_6f0738f0.log diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_002/README.md b/ranking/ctr/HFM/HFM+_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_002/README.md rename to ranking/ctr/HFM/HFM+_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_002/environments.txt b/ranking/ctr/HFM/HFM+_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_002/environments.txt rename to ranking/ctr/HFM/HFM+_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_002/results.csv b/ranking/ctr/HFM/HFM+_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_002/results.csv rename to ranking/ctr/HFM/HFM+_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_002/run_expid.py b/ranking/ctr/HFM/HFM+_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_avazu_x4_002/run_expid.py rename to ranking/ctr/HFM/HFM+_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x1/HFM+_criteo_x1_tuner_config_04.csv b/ranking/ctr/HFM/HFM+_criteo_x1/HFM+_criteo_x1_tuner_config_04.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x1/HFM+_criteo_x1_tuner_config_04.csv rename to ranking/ctr/HFM/HFM+_criteo_x1/HFM+_criteo_x1_tuner_config_04.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x1/HFM+_criteo_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/HFM/HFM+_criteo_x1/HFM+_criteo_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x1/HFM+_criteo_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/HFM/HFM+_criteo_x1/HFM+_criteo_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x1/HFM+_criteo_x1_tuner_config_04/model_config.yaml b/ranking/ctr/HFM/HFM+_criteo_x1/HFM+_criteo_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x1/HFM+_criteo_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/HFM/HFM+_criteo_x1/HFM+_criteo_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x1/HFM_criteo_x1_009_f964ad0a.log b/ranking/ctr/HFM/HFM+_criteo_x1/HFM_criteo_x1_009_f964ad0a.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x1/HFM_criteo_x1_009_f964ad0a.log rename to ranking/ctr/HFM/HFM+_criteo_x1/HFM_criteo_x1_009_f964ad0a.log diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x1/README.md b/ranking/ctr/HFM/HFM+_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x1/README.md rename to ranking/ctr/HFM/HFM+_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x1/environments.txt b/ranking/ctr/HFM/HFM+_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x1/environments.txt rename to ranking/ctr/HFM/HFM+_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x1/results.csv b/ranking/ctr/HFM/HFM+_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x1/results.csv rename to ranking/ctr/HFM/HFM+_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x1/run_expid.py b/ranking/ctr/HFM/HFM+_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x1/run_expid.py rename to ranking/ctr/HFM/HFM+_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_001/HFM+_criteo_x4_tuner_config_11/dataset_config.yaml b/ranking/ctr/HFM/HFM+_criteo_x4_001/HFM+_criteo_x4_tuner_config_11/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_001/HFM+_criteo_x4_tuner_config_11/dataset_config.yaml rename to ranking/ctr/HFM/HFM+_criteo_x4_001/HFM+_criteo_x4_tuner_config_11/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_001/HFM+_criteo_x4_tuner_config_11/model_config.yaml b/ranking/ctr/HFM/HFM+_criteo_x4_001/HFM+_criteo_x4_tuner_config_11/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_001/HFM+_criteo_x4_tuner_config_11/model_config.yaml rename to ranking/ctr/HFM/HFM+_criteo_x4_001/HFM+_criteo_x4_tuner_config_11/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_001/HFM_criteo_x4_006_2abdb8e2.log b/ranking/ctr/HFM/HFM+_criteo_x4_001/HFM_criteo_x4_006_2abdb8e2.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_001/HFM_criteo_x4_006_2abdb8e2.log rename to ranking/ctr/HFM/HFM+_criteo_x4_001/HFM_criteo_x4_006_2abdb8e2.log diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_001/README.md b/ranking/ctr/HFM/HFM+_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_001/README.md rename to ranking/ctr/HFM/HFM+_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_001/environments.txt b/ranking/ctr/HFM/HFM+_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_001/environments.txt rename to ranking/ctr/HFM/HFM+_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_001/results.csv b/ranking/ctr/HFM/HFM+_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_001/results.csv rename to ranking/ctr/HFM/HFM+_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_001/run_expid.py b/ranking/ctr/HFM/HFM+_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_001/run_expid.py rename to ranking/ctr/HFM/HFM+_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_002/HFM+_criteo_x4_tuner_config_08/dataset_config.yaml b/ranking/ctr/HFM/HFM+_criteo_x4_002/HFM+_criteo_x4_tuner_config_08/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_002/HFM+_criteo_x4_tuner_config_08/dataset_config.yaml rename to ranking/ctr/HFM/HFM+_criteo_x4_002/HFM+_criteo_x4_tuner_config_08/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_002/HFM+_criteo_x4_tuner_config_08/model_config.yaml b/ranking/ctr/HFM/HFM+_criteo_x4_002/HFM+_criteo_x4_tuner_config_08/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_002/HFM+_criteo_x4_tuner_config_08/model_config.yaml rename to ranking/ctr/HFM/HFM+_criteo_x4_002/HFM+_criteo_x4_tuner_config_08/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_002/HFM_criteo_x4_001_0d1cce33.log b/ranking/ctr/HFM/HFM+_criteo_x4_002/HFM_criteo_x4_001_0d1cce33.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_002/HFM_criteo_x4_001_0d1cce33.log rename to ranking/ctr/HFM/HFM+_criteo_x4_002/HFM_criteo_x4_001_0d1cce33.log diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_002/README.md b/ranking/ctr/HFM/HFM+_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_002/README.md rename to ranking/ctr/HFM/HFM+_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_002/environments.txt b/ranking/ctr/HFM/HFM+_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_002/environments.txt rename to ranking/ctr/HFM/HFM+_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_002/results.csv b/ranking/ctr/HFM/HFM+_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_002/results.csv rename to ranking/ctr/HFM/HFM+_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_002/run_expid.py b/ranking/ctr/HFM/HFM+_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_criteo_x4_002/run_expid.py rename to ranking/ctr/HFM/HFM+_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM+_frappe_x1/HFM+_frappe_x1_tuner_config_04.csv b/ranking/ctr/HFM/HFM+_frappe_x1/HFM+_frappe_x1_tuner_config_04.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_frappe_x1/HFM+_frappe_x1_tuner_config_04.csv rename to ranking/ctr/HFM/HFM+_frappe_x1/HFM+_frappe_x1_tuner_config_04.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM+_frappe_x1/HFM+_frappe_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/HFM/HFM+_frappe_x1/HFM+_frappe_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_frappe_x1/HFM+_frappe_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/HFM/HFM+_frappe_x1/HFM+_frappe_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_frappe_x1/HFM+_frappe_x1_tuner_config_04/model_config.yaml b/ranking/ctr/HFM/HFM+_frappe_x1/HFM+_frappe_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_frappe_x1/HFM+_frappe_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/HFM/HFM+_frappe_x1/HFM+_frappe_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_frappe_x1/HFM_frappe_x1_006_0caf1c14.log b/ranking/ctr/HFM/HFM+_frappe_x1/HFM_frappe_x1_006_0caf1c14.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_frappe_x1/HFM_frappe_x1_006_0caf1c14.log rename to ranking/ctr/HFM/HFM+_frappe_x1/HFM_frappe_x1_006_0caf1c14.log diff --git a/ctr_prediction/benchmarks/HFM/HFM+_frappe_x1/README.md b/ranking/ctr/HFM/HFM+_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_frappe_x1/README.md rename to ranking/ctr/HFM/HFM+_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM+_frappe_x1/environments.txt b/ranking/ctr/HFM/HFM+_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_frappe_x1/environments.txt rename to ranking/ctr/HFM/HFM+_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM+_frappe_x1/results.csv b/ranking/ctr/HFM/HFM+_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_frappe_x1/results.csv rename to ranking/ctr/HFM/HFM+_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM+_frappe_x1/run_expid.py b/ranking/ctr/HFM/HFM+_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_frappe_x1/run_expid.py rename to ranking/ctr/HFM/HFM+_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM+_kkbox_x1/HFM+_kkbox_x1_tuner_config_05/dataset_config.yaml b/ranking/ctr/HFM/HFM+_kkbox_x1/HFM+_kkbox_x1_tuner_config_05/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_kkbox_x1/HFM+_kkbox_x1_tuner_config_05/dataset_config.yaml rename to ranking/ctr/HFM/HFM+_kkbox_x1/HFM+_kkbox_x1_tuner_config_05/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_kkbox_x1/HFM+_kkbox_x1_tuner_config_05/model_config.yaml b/ranking/ctr/HFM/HFM+_kkbox_x1/HFM+_kkbox_x1_tuner_config_05/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_kkbox_x1/HFM+_kkbox_x1_tuner_config_05/model_config.yaml rename to ranking/ctr/HFM/HFM+_kkbox_x1/HFM+_kkbox_x1_tuner_config_05/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_kkbox_x1/HFM_kkbox_x1_019_7e9801c4.log b/ranking/ctr/HFM/HFM+_kkbox_x1/HFM_kkbox_x1_019_7e9801c4.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_kkbox_x1/HFM_kkbox_x1_019_7e9801c4.log rename to ranking/ctr/HFM/HFM+_kkbox_x1/HFM_kkbox_x1_019_7e9801c4.log diff --git a/ctr_prediction/benchmarks/HFM/HFM+_kkbox_x1/README.md b/ranking/ctr/HFM/HFM+_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_kkbox_x1/README.md rename to ranking/ctr/HFM/HFM+_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM+_kkbox_x1/environments.txt b/ranking/ctr/HFM/HFM+_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_kkbox_x1/environments.txt rename to ranking/ctr/HFM/HFM+_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM+_kkbox_x1/results.csv b/ranking/ctr/HFM/HFM+_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_kkbox_x1/results.csv rename to ranking/ctr/HFM/HFM+_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM+_kkbox_x1/run_expid.py b/ranking/ctr/HFM/HFM+_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_kkbox_x1/run_expid.py rename to ranking/ctr/HFM/HFM+_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM+_movielenslatest_x1/HFM+_movielenslatest_x1_tuner_config_03.csv b/ranking/ctr/HFM/HFM+_movielenslatest_x1/HFM+_movielenslatest_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_movielenslatest_x1/HFM+_movielenslatest_x1_tuner_config_03.csv rename to ranking/ctr/HFM/HFM+_movielenslatest_x1/HFM+_movielenslatest_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM+_movielenslatest_x1/HFM+_movielenslatest_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/HFM/HFM+_movielenslatest_x1/HFM+_movielenslatest_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_movielenslatest_x1/HFM+_movielenslatest_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/HFM/HFM+_movielenslatest_x1/HFM+_movielenslatest_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_movielenslatest_x1/HFM+_movielenslatest_x1_tuner_config_03/model_config.yaml b/ranking/ctr/HFM/HFM+_movielenslatest_x1/HFM+_movielenslatest_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_movielenslatest_x1/HFM+_movielenslatest_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/HFM/HFM+_movielenslatest_x1/HFM+_movielenslatest_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM+_movielenslatest_x1/HFM_movielenslatest_x1_007_91dbbe82.log b/ranking/ctr/HFM/HFM+_movielenslatest_x1/HFM_movielenslatest_x1_007_91dbbe82.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_movielenslatest_x1/HFM_movielenslatest_x1_007_91dbbe82.log rename to ranking/ctr/HFM/HFM+_movielenslatest_x1/HFM_movielenslatest_x1_007_91dbbe82.log diff --git a/ctr_prediction/benchmarks/HFM/HFM+_movielenslatest_x1/README.md b/ranking/ctr/HFM/HFM+_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_movielenslatest_x1/README.md rename to ranking/ctr/HFM/HFM+_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM+_movielenslatest_x1/environments.txt b/ranking/ctr/HFM/HFM+_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_movielenslatest_x1/environments.txt rename to ranking/ctr/HFM/HFM+_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM+_movielenslatest_x1/results.csv b/ranking/ctr/HFM/HFM+_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_movielenslatest_x1/results.csv rename to ranking/ctr/HFM/HFM+_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM+_movielenslatest_x1/run_expid.py b/ranking/ctr/HFM/HFM+_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM+_movielenslatest_x1/run_expid.py rename to ranking/ctr/HFM/HFM+_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x1/HFM_avazu_x1_007_bf21aada.log b/ranking/ctr/HFM/HFM_avazu_x1/HFM_avazu_x1_007_bf21aada.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x1/HFM_avazu_x1_007_bf21aada.log rename to ranking/ctr/HFM/HFM_avazu_x1/HFM_avazu_x1_007_bf21aada.log diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x1/HFM_avazu_x1_tuner_config_01.csv b/ranking/ctr/HFM/HFM_avazu_x1/HFM_avazu_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x1/HFM_avazu_x1_tuner_config_01.csv rename to ranking/ctr/HFM/HFM_avazu_x1/HFM_avazu_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x1/HFM_avazu_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/HFM/HFM_avazu_x1/HFM_avazu_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x1/HFM_avazu_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/HFM/HFM_avazu_x1/HFM_avazu_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x1/HFM_avazu_x1_tuner_config_01/model_config.yaml b/ranking/ctr/HFM/HFM_avazu_x1/HFM_avazu_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x1/HFM_avazu_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/HFM/HFM_avazu_x1/HFM_avazu_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x1/README.md b/ranking/ctr/HFM/HFM_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x1/README.md rename to ranking/ctr/HFM/HFM_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x1/environments.txt b/ranking/ctr/HFM/HFM_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x1/environments.txt rename to ranking/ctr/HFM/HFM_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x1/results.csv b/ranking/ctr/HFM/HFM_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x1/results.csv rename to ranking/ctr/HFM/HFM_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x1/run_expid.py b/ranking/ctr/HFM/HFM_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x1/run_expid.py rename to ranking/ctr/HFM/HFM_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x4_001/HFM_avazu_x4_009_8053d0ee.log b/ranking/ctr/HFM/HFM_avazu_x4_001/HFM_avazu_x4_009_8053d0ee.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x4_001/HFM_avazu_x4_009_8053d0ee.log rename to ranking/ctr/HFM/HFM_avazu_x4_001/HFM_avazu_x4_009_8053d0ee.log diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x4_001/HFM_avazu_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/HFM/HFM_avazu_x4_001/HFM_avazu_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x4_001/HFM_avazu_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/HFM/HFM_avazu_x4_001/HFM_avazu_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x4_001/HFM_avazu_x4_tuner_config_02/model_config.yaml b/ranking/ctr/HFM/HFM_avazu_x4_001/HFM_avazu_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x4_001/HFM_avazu_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/HFM/HFM_avazu_x4_001/HFM_avazu_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x4_001/README.md b/ranking/ctr/HFM/HFM_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x4_001/README.md rename to ranking/ctr/HFM/HFM_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x4_001/environments.txt b/ranking/ctr/HFM/HFM_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x4_001/environments.txt rename to ranking/ctr/HFM/HFM_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x4_001/results.csv b/ranking/ctr/HFM/HFM_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x4_001/results.csv rename to ranking/ctr/HFM/HFM_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x4_001/run_expid.py b/ranking/ctr/HFM/HFM_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x4_001/run_expid.py rename to ranking/ctr/HFM/HFM_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x4_002/HFM_avazu_x4_004_1109463b.log b/ranking/ctr/HFM/HFM_avazu_x4_002/HFM_avazu_x4_004_1109463b.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x4_002/HFM_avazu_x4_004_1109463b.log rename to ranking/ctr/HFM/HFM_avazu_x4_002/HFM_avazu_x4_004_1109463b.log diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x4_002/HFM_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/HFM/HFM_avazu_x4_002/HFM_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x4_002/HFM_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/HFM/HFM_avazu_x4_002/HFM_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x4_002/HFM_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/HFM/HFM_avazu_x4_002/HFM_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x4_002/HFM_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/HFM/HFM_avazu_x4_002/HFM_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x4_002/README.md b/ranking/ctr/HFM/HFM_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x4_002/README.md rename to ranking/ctr/HFM/HFM_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x4_002/environments.txt b/ranking/ctr/HFM/HFM_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x4_002/environments.txt rename to ranking/ctr/HFM/HFM_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x4_002/results.csv b/ranking/ctr/HFM/HFM_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x4_002/results.csv rename to ranking/ctr/HFM/HFM_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM_avazu_x4_002/run_expid.py b/ranking/ctr/HFM/HFM_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_avazu_x4_002/run_expid.py rename to ranking/ctr/HFM/HFM_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x1/HFM_criteo_x1_006_8906e1e1.log b/ranking/ctr/HFM/HFM_criteo_x1/HFM_criteo_x1_006_8906e1e1.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x1/HFM_criteo_x1_006_8906e1e1.log rename to ranking/ctr/HFM/HFM_criteo_x1/HFM_criteo_x1_006_8906e1e1.log diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x1/HFM_criteo_x1_tuner_config_01.csv b/ranking/ctr/HFM/HFM_criteo_x1/HFM_criteo_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x1/HFM_criteo_x1_tuner_config_01.csv rename to ranking/ctr/HFM/HFM_criteo_x1/HFM_criteo_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x1/HFM_criteo_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/HFM/HFM_criteo_x1/HFM_criteo_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x1/HFM_criteo_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/HFM/HFM_criteo_x1/HFM_criteo_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x1/HFM_criteo_x1_tuner_config_01/model_config.yaml b/ranking/ctr/HFM/HFM_criteo_x1/HFM_criteo_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x1/HFM_criteo_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/HFM/HFM_criteo_x1/HFM_criteo_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x1/README.md b/ranking/ctr/HFM/HFM_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x1/README.md rename to ranking/ctr/HFM/HFM_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x1/environments.txt b/ranking/ctr/HFM/HFM_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x1/environments.txt rename to ranking/ctr/HFM/HFM_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x1/results.csv b/ranking/ctr/HFM/HFM_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x1/results.csv rename to ranking/ctr/HFM/HFM_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x1/run_expid.py b/ranking/ctr/HFM/HFM_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x1/run_expid.py rename to ranking/ctr/HFM/HFM_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x4_001/HFM_criteo_x4_008_c1d1ba8a.log b/ranking/ctr/HFM/HFM_criteo_x4_001/HFM_criteo_x4_008_c1d1ba8a.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x4_001/HFM_criteo_x4_008_c1d1ba8a.log rename to ranking/ctr/HFM/HFM_criteo_x4_001/HFM_criteo_x4_008_c1d1ba8a.log diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x4_001/HFM_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/HFM/HFM_criteo_x4_001/HFM_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x4_001/HFM_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/HFM/HFM_criteo_x4_001/HFM_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x4_001/HFM_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/HFM/HFM_criteo_x4_001/HFM_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x4_001/HFM_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/HFM/HFM_criteo_x4_001/HFM_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x4_001/README.md b/ranking/ctr/HFM/HFM_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x4_001/README.md rename to ranking/ctr/HFM/HFM_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x4_001/environments.txt b/ranking/ctr/HFM/HFM_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x4_001/environments.txt rename to ranking/ctr/HFM/HFM_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x4_001/results.csv b/ranking/ctr/HFM/HFM_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x4_001/results.csv rename to ranking/ctr/HFM/HFM_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x4_001/run_expid.py b/ranking/ctr/HFM/HFM_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x4_001/run_expid.py rename to ranking/ctr/HFM/HFM_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x4_002/HFM_criteo_x4_002_bc5f3953.log b/ranking/ctr/HFM/HFM_criteo_x4_002/HFM_criteo_x4_002_bc5f3953.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x4_002/HFM_criteo_x4_002_bc5f3953.log rename to ranking/ctr/HFM/HFM_criteo_x4_002/HFM_criteo_x4_002_bc5f3953.log diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x4_002/HFM_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/HFM/HFM_criteo_x4_002/HFM_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x4_002/HFM_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/HFM/HFM_criteo_x4_002/HFM_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x4_002/HFM_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/HFM/HFM_criteo_x4_002/HFM_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x4_002/HFM_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/HFM/HFM_criteo_x4_002/HFM_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x4_002/README.md b/ranking/ctr/HFM/HFM_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x4_002/README.md rename to ranking/ctr/HFM/HFM_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x4_002/environments.txt b/ranking/ctr/HFM/HFM_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x4_002/environments.txt rename to ranking/ctr/HFM/HFM_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x4_002/results.csv b/ranking/ctr/HFM/HFM_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x4_002/results.csv rename to ranking/ctr/HFM/HFM_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM_criteo_x4_002/run_expid.py b/ranking/ctr/HFM/HFM_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_criteo_x4_002/run_expid.py rename to ranking/ctr/HFM/HFM_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM_frappe_x1/HFM_frappe_x1_001_3755ddc7.log b/ranking/ctr/HFM/HFM_frappe_x1/HFM_frappe_x1_001_3755ddc7.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_frappe_x1/HFM_frappe_x1_001_3755ddc7.log rename to ranking/ctr/HFM/HFM_frappe_x1/HFM_frappe_x1_001_3755ddc7.log diff --git a/ctr_prediction/benchmarks/HFM/HFM_frappe_x1/HFM_frappe_x1_tuner_config_03.csv b/ranking/ctr/HFM/HFM_frappe_x1/HFM_frappe_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_frappe_x1/HFM_frappe_x1_tuner_config_03.csv rename to ranking/ctr/HFM/HFM_frappe_x1/HFM_frappe_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM_frappe_x1/HFM_frappe_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/HFM/HFM_frappe_x1/HFM_frappe_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_frappe_x1/HFM_frappe_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/HFM/HFM_frappe_x1/HFM_frappe_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_frappe_x1/HFM_frappe_x1_tuner_config_03/model_config.yaml b/ranking/ctr/HFM/HFM_frappe_x1/HFM_frappe_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_frappe_x1/HFM_frappe_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/HFM/HFM_frappe_x1/HFM_frappe_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_frappe_x1/environments.txt b/ranking/ctr/HFM/HFM_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_frappe_x1/environments.txt rename to ranking/ctr/HFM/HFM_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM_frappe_x1/results.csv b/ranking/ctr/HFM/HFM_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_frappe_x1/results.csv rename to ranking/ctr/HFM/HFM_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM_kkbox_x1/HFM_kkbox_x1_006_b549df4b.log b/ranking/ctr/HFM/HFM_kkbox_x1/HFM_kkbox_x1_006_b549df4b.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_kkbox_x1/HFM_kkbox_x1_006_b549df4b.log rename to ranking/ctr/HFM/HFM_kkbox_x1/HFM_kkbox_x1_006_b549df4b.log diff --git a/ctr_prediction/benchmarks/HFM/HFM_kkbox_x1/HFM_kkbox_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/HFM/HFM_kkbox_x1/HFM_kkbox_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_kkbox_x1/HFM_kkbox_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/HFM/HFM_kkbox_x1/HFM_kkbox_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_kkbox_x1/HFM_kkbox_x1_tuner_config_03/model_config.yaml b/ranking/ctr/HFM/HFM_kkbox_x1/HFM_kkbox_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_kkbox_x1/HFM_kkbox_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/HFM/HFM_kkbox_x1/HFM_kkbox_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_kkbox_x1/README.md b/ranking/ctr/HFM/HFM_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_kkbox_x1/README.md rename to ranking/ctr/HFM/HFM_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM_kkbox_x1/environments.txt b/ranking/ctr/HFM/HFM_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_kkbox_x1/environments.txt rename to ranking/ctr/HFM/HFM_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM_kkbox_x1/results.csv b/ranking/ctr/HFM/HFM_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_kkbox_x1/results.csv rename to ranking/ctr/HFM/HFM_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM_kkbox_x1/run_expid.py b/ranking/ctr/HFM/HFM_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_kkbox_x1/run_expid.py rename to ranking/ctr/HFM/HFM_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/HFM/HFM_movielenslatest_x1/HFM_movielenslatest_x1_002_e5873ea8.log b/ranking/ctr/HFM/HFM_movielenslatest_x1/HFM_movielenslatest_x1_002_e5873ea8.log similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_movielenslatest_x1/HFM_movielenslatest_x1_002_e5873ea8.log rename to ranking/ctr/HFM/HFM_movielenslatest_x1/HFM_movielenslatest_x1_002_e5873ea8.log diff --git a/ctr_prediction/benchmarks/HFM/HFM_movielenslatest_x1/HFM_movielenslatest_x1_tuner_config_05.csv b/ranking/ctr/HFM/HFM_movielenslatest_x1/HFM_movielenslatest_x1_tuner_config_05.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_movielenslatest_x1/HFM_movielenslatest_x1_tuner_config_05.csv rename to ranking/ctr/HFM/HFM_movielenslatest_x1/HFM_movielenslatest_x1_tuner_config_05.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM_movielenslatest_x1/HFM_movielenslatest_x1_tuner_config_05/dataset_config.yaml b/ranking/ctr/HFM/HFM_movielenslatest_x1/HFM_movielenslatest_x1_tuner_config_05/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_movielenslatest_x1/HFM_movielenslatest_x1_tuner_config_05/dataset_config.yaml rename to ranking/ctr/HFM/HFM_movielenslatest_x1/HFM_movielenslatest_x1_tuner_config_05/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_movielenslatest_x1/HFM_movielenslatest_x1_tuner_config_05/model_config.yaml b/ranking/ctr/HFM/HFM_movielenslatest_x1/HFM_movielenslatest_x1_tuner_config_05/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_movielenslatest_x1/HFM_movielenslatest_x1_tuner_config_05/model_config.yaml rename to ranking/ctr/HFM/HFM_movielenslatest_x1/HFM_movielenslatest_x1_tuner_config_05/model_config.yaml diff --git a/ctr_prediction/benchmarks/HFM/HFM_movielenslatest_x1/README.md b/ranking/ctr/HFM/HFM_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_movielenslatest_x1/README.md rename to ranking/ctr/HFM/HFM_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/HFM/HFM_movielenslatest_x1/environments.txt b/ranking/ctr/HFM/HFM_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_movielenslatest_x1/environments.txt rename to ranking/ctr/HFM/HFM_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/HFM/HFM_movielenslatest_x1/results.csv b/ranking/ctr/HFM/HFM_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_movielenslatest_x1/results.csv rename to ranking/ctr/HFM/HFM_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/HFM/HFM_movielenslatest_x1/run_expid.py b/ranking/ctr/HFM/HFM_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HFM/HFM_movielenslatest_x1/run_expid.py rename to ranking/ctr/HFM/HFM_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x1/HOFM_avazu_x1_002_11ff3102.log b/ranking/ctr/HOFM/HOFM_avazu_x1/HOFM_avazu_x1_002_11ff3102.log similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x1/HOFM_avazu_x1_002_11ff3102.log rename to ranking/ctr/HOFM/HOFM_avazu_x1/HOFM_avazu_x1_002_11ff3102.log diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x1/HOFM_avazu_x1_tuner_config_02.csv b/ranking/ctr/HOFM/HOFM_avazu_x1/HOFM_avazu_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x1/HOFM_avazu_x1_tuner_config_02.csv rename to ranking/ctr/HOFM/HOFM_avazu_x1/HOFM_avazu_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x1/HOFM_avazu_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/HOFM/HOFM_avazu_x1/HOFM_avazu_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x1/HOFM_avazu_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/HOFM/HOFM_avazu_x1/HOFM_avazu_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x1/HOFM_avazu_x1_tuner_config_02/model_config.yaml b/ranking/ctr/HOFM/HOFM_avazu_x1/HOFM_avazu_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x1/HOFM_avazu_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/HOFM/HOFM_avazu_x1/HOFM_avazu_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x1/README.md b/ranking/ctr/HOFM/HOFM_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x1/README.md rename to ranking/ctr/HOFM/HOFM_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x1/environments.txt b/ranking/ctr/HOFM/HOFM_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x1/environments.txt rename to ranking/ctr/HOFM/HOFM_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x1/results.csv b/ranking/ctr/HOFM/HOFM_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x1/results.csv rename to ranking/ctr/HOFM/HOFM_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x1/run_expid.py b/ranking/ctr/HOFM/HOFM_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x1/run_expid.py rename to ranking/ctr/HOFM/HOFM_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_001/HOFM_avazu_x4_001_a9f41af5.log b/ranking/ctr/HOFM/HOFM_avazu_x4_001/HOFM_avazu_x4_001_a9f41af5.log similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_001/HOFM_avazu_x4_001_a9f41af5.log rename to ranking/ctr/HOFM/HOFM_avazu_x4_001/HOFM_avazu_x4_001_a9f41af5.log diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_001/HOFM_avazu_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/HOFM/HOFM_avazu_x4_001/HOFM_avazu_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_001/HOFM_avazu_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/HOFM/HOFM_avazu_x4_001/HOFM_avazu_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_001/HOFM_avazu_x4_tuner_config_02/model_config.yaml b/ranking/ctr/HOFM/HOFM_avazu_x4_001/HOFM_avazu_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_001/HOFM_avazu_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/HOFM/HOFM_avazu_x4_001/HOFM_avazu_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_001/README.md b/ranking/ctr/HOFM/HOFM_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_001/README.md rename to ranking/ctr/HOFM/HOFM_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_001/environments.txt b/ranking/ctr/HOFM/HOFM_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_001/environments.txt rename to ranking/ctr/HOFM/HOFM_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_001/results.csv b/ranking/ctr/HOFM/HOFM_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_001/results.csv rename to ranking/ctr/HOFM/HOFM_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_001/run_expid.py b/ranking/ctr/HOFM/HOFM_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_001/run_expid.py rename to ranking/ctr/HOFM/HOFM_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_002/HOFM_avazu_x4_001_252b26d7.log b/ranking/ctr/HOFM/HOFM_avazu_x4_002/HOFM_avazu_x4_001_252b26d7.log similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_002/HOFM_avazu_x4_001_252b26d7.log rename to ranking/ctr/HOFM/HOFM_avazu_x4_002/HOFM_avazu_x4_001_252b26d7.log diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_002/HOFM_avazu_x4_tuner_config_07/dataset_config.yaml b/ranking/ctr/HOFM/HOFM_avazu_x4_002/HOFM_avazu_x4_tuner_config_07/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_002/HOFM_avazu_x4_tuner_config_07/dataset_config.yaml rename to ranking/ctr/HOFM/HOFM_avazu_x4_002/HOFM_avazu_x4_tuner_config_07/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_002/HOFM_avazu_x4_tuner_config_07/model_config.yaml b/ranking/ctr/HOFM/HOFM_avazu_x4_002/HOFM_avazu_x4_tuner_config_07/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_002/HOFM_avazu_x4_tuner_config_07/model_config.yaml rename to ranking/ctr/HOFM/HOFM_avazu_x4_002/HOFM_avazu_x4_tuner_config_07/model_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_002/README.md b/ranking/ctr/HOFM/HOFM_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_002/README.md rename to ranking/ctr/HOFM/HOFM_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_002/environments.txt b/ranking/ctr/HOFM/HOFM_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_002/environments.txt rename to ranking/ctr/HOFM/HOFM_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_002/results.csv b/ranking/ctr/HOFM/HOFM_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_002/results.csv rename to ranking/ctr/HOFM/HOFM_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_002/run_expid.py b/ranking/ctr/HOFM/HOFM_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_avazu_x4_002/run_expid.py rename to ranking/ctr/HOFM/HOFM_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x1/HOFM_criteo_x1_001_1b741ae5.log b/ranking/ctr/HOFM/HOFM_criteo_x1/HOFM_criteo_x1_001_1b741ae5.log similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x1/HOFM_criteo_x1_001_1b741ae5.log rename to ranking/ctr/HOFM/HOFM_criteo_x1/HOFM_criteo_x1_001_1b741ae5.log diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x1/HOFM_criteo_x1_tuner_config_02.csv b/ranking/ctr/HOFM/HOFM_criteo_x1/HOFM_criteo_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x1/HOFM_criteo_x1_tuner_config_02.csv rename to ranking/ctr/HOFM/HOFM_criteo_x1/HOFM_criteo_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x1/HOFM_criteo_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/HOFM/HOFM_criteo_x1/HOFM_criteo_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x1/HOFM_criteo_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/HOFM/HOFM_criteo_x1/HOFM_criteo_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x1/HOFM_criteo_x1_tuner_config_02/model_config.yaml b/ranking/ctr/HOFM/HOFM_criteo_x1/HOFM_criteo_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x1/HOFM_criteo_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/HOFM/HOFM_criteo_x1/HOFM_criteo_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x1/README.md b/ranking/ctr/HOFM/HOFM_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x1/README.md rename to ranking/ctr/HOFM/HOFM_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x1/environments.txt b/ranking/ctr/HOFM/HOFM_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x1/environments.txt rename to ranking/ctr/HOFM/HOFM_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x1/results.csv b/ranking/ctr/HOFM/HOFM_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x1/results.csv rename to ranking/ctr/HOFM/HOFM_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x1/run_expid.py b/ranking/ctr/HOFM/HOFM_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x1/run_expid.py rename to ranking/ctr/HOFM/HOFM_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_001/HOFM_criteo_x4_001_af6d1a42.log b/ranking/ctr/HOFM/HOFM_criteo_x4_001/HOFM_criteo_x4_001_af6d1a42.log similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_001/HOFM_criteo_x4_001_af6d1a42.log rename to ranking/ctr/HOFM/HOFM_criteo_x4_001/HOFM_criteo_x4_001_af6d1a42.log diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_001/HOFM_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/HOFM/HOFM_criteo_x4_001/HOFM_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_001/HOFM_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/HOFM/HOFM_criteo_x4_001/HOFM_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_001/HOFM_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/HOFM/HOFM_criteo_x4_001/HOFM_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_001/HOFM_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/HOFM/HOFM_criteo_x4_001/HOFM_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_001/README.md b/ranking/ctr/HOFM/HOFM_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_001/README.md rename to ranking/ctr/HOFM/HOFM_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_001/environments.txt b/ranking/ctr/HOFM/HOFM_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_001/environments.txt rename to ranking/ctr/HOFM/HOFM_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_001/results.csv b/ranking/ctr/HOFM/HOFM_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_001/results.csv rename to ranking/ctr/HOFM/HOFM_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_001/run_expid.py b/ranking/ctr/HOFM/HOFM_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_001/run_expid.py rename to ranking/ctr/HOFM/HOFM_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_002/HOFM_criteo_x4_001_a187c06d.log b/ranking/ctr/HOFM/HOFM_criteo_x4_002/HOFM_criteo_x4_001_a187c06d.log similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_002/HOFM_criteo_x4_001_a187c06d.log rename to ranking/ctr/HOFM/HOFM_criteo_x4_002/HOFM_criteo_x4_001_a187c06d.log diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_002/HOFM_criteo_x4_tuner_config_05/dataset_config.yaml b/ranking/ctr/HOFM/HOFM_criteo_x4_002/HOFM_criteo_x4_tuner_config_05/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_002/HOFM_criteo_x4_tuner_config_05/dataset_config.yaml rename to ranking/ctr/HOFM/HOFM_criteo_x4_002/HOFM_criteo_x4_tuner_config_05/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_002/HOFM_criteo_x4_tuner_config_05/model_config.yaml b/ranking/ctr/HOFM/HOFM_criteo_x4_002/HOFM_criteo_x4_tuner_config_05/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_002/HOFM_criteo_x4_tuner_config_05/model_config.yaml rename to ranking/ctr/HOFM/HOFM_criteo_x4_002/HOFM_criteo_x4_tuner_config_05/model_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_002/README.md b/ranking/ctr/HOFM/HOFM_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_002/README.md rename to ranking/ctr/HOFM/HOFM_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_002/environments.txt b/ranking/ctr/HOFM/HOFM_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_002/environments.txt rename to ranking/ctr/HOFM/HOFM_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_002/results.csv b/ranking/ctr/HOFM/HOFM_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_002/results.csv rename to ranking/ctr/HOFM/HOFM_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_002/run_expid.py b/ranking/ctr/HOFM/HOFM_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_criteo_x4_002/run_expid.py rename to ranking/ctr/HOFM/HOFM_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_frappe_x1/HOFM_frappe_x1_003_64d2d04e.log b/ranking/ctr/HOFM/HOFM_frappe_x1/HOFM_frappe_x1_003_64d2d04e.log similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_frappe_x1/HOFM_frappe_x1_003_64d2d04e.log rename to ranking/ctr/HOFM/HOFM_frappe_x1/HOFM_frappe_x1_003_64d2d04e.log diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_frappe_x1/HOFM_frappe_x1_tuner_config_02.csv b/ranking/ctr/HOFM/HOFM_frappe_x1/HOFM_frappe_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_frappe_x1/HOFM_frappe_x1_tuner_config_02.csv rename to ranking/ctr/HOFM/HOFM_frappe_x1/HOFM_frappe_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_frappe_x1/HOFM_frappe_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/HOFM/HOFM_frappe_x1/HOFM_frappe_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_frappe_x1/HOFM_frappe_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/HOFM/HOFM_frappe_x1/HOFM_frappe_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_frappe_x1/HOFM_frappe_x1_tuner_config_02/model_config.yaml b/ranking/ctr/HOFM/HOFM_frappe_x1/HOFM_frappe_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_frappe_x1/HOFM_frappe_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/HOFM/HOFM_frappe_x1/HOFM_frappe_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_frappe_x1/README.md b/ranking/ctr/HOFM/HOFM_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_frappe_x1/README.md rename to ranking/ctr/HOFM/HOFM_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_frappe_x1/environments.txt b/ranking/ctr/HOFM/HOFM_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_frappe_x1/environments.txt rename to ranking/ctr/HOFM/HOFM_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_frappe_x1/results.csv b/ranking/ctr/HOFM/HOFM_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_frappe_x1/results.csv rename to ranking/ctr/HOFM/HOFM_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_frappe_x1/run_expid.py b/ranking/ctr/HOFM/HOFM_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_frappe_x1/run_expid.py rename to ranking/ctr/HOFM/HOFM_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_kkbox_x1/HOFM_kkbox_x1_017_4b0da798.log b/ranking/ctr/HOFM/HOFM_kkbox_x1/HOFM_kkbox_x1_017_4b0da798.log similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_kkbox_x1/HOFM_kkbox_x1_017_4b0da798.log rename to ranking/ctr/HOFM/HOFM_kkbox_x1/HOFM_kkbox_x1_017_4b0da798.log diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_kkbox_x1/HOFM_kkbox_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/HOFM/HOFM_kkbox_x1/HOFM_kkbox_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_kkbox_x1/HOFM_kkbox_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/HOFM/HOFM_kkbox_x1/HOFM_kkbox_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_kkbox_x1/HOFM_kkbox_x1_tuner_config_01/model_config.yaml b/ranking/ctr/HOFM/HOFM_kkbox_x1/HOFM_kkbox_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_kkbox_x1/HOFM_kkbox_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/HOFM/HOFM_kkbox_x1/HOFM_kkbox_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_kkbox_x1/README.md b/ranking/ctr/HOFM/HOFM_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_kkbox_x1/README.md rename to ranking/ctr/HOFM/HOFM_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_kkbox_x1/environments.txt b/ranking/ctr/HOFM/HOFM_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_kkbox_x1/environments.txt rename to ranking/ctr/HOFM/HOFM_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_kkbox_x1/results.csv b/ranking/ctr/HOFM/HOFM_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_kkbox_x1/results.csv rename to ranking/ctr/HOFM/HOFM_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_kkbox_x1/run_expid.py b/ranking/ctr/HOFM/HOFM_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_kkbox_x1/run_expid.py rename to ranking/ctr/HOFM/HOFM_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_movielenslatest_x1/HOFM_movielenslatest_x1_004_c58c682e.log b/ranking/ctr/HOFM/HOFM_movielenslatest_x1/HOFM_movielenslatest_x1_004_c58c682e.log similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_movielenslatest_x1/HOFM_movielenslatest_x1_004_c58c682e.log rename to ranking/ctr/HOFM/HOFM_movielenslatest_x1/HOFM_movielenslatest_x1_004_c58c682e.log diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_movielenslatest_x1/HOFM_movielenslatest_x1_tuner_config_03.csv b/ranking/ctr/HOFM/HOFM_movielenslatest_x1/HOFM_movielenslatest_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_movielenslatest_x1/HOFM_movielenslatest_x1_tuner_config_03.csv rename to ranking/ctr/HOFM/HOFM_movielenslatest_x1/HOFM_movielenslatest_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_movielenslatest_x1/HOFM_movielenslatest_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/HOFM/HOFM_movielenslatest_x1/HOFM_movielenslatest_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_movielenslatest_x1/HOFM_movielenslatest_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/HOFM/HOFM_movielenslatest_x1/HOFM_movielenslatest_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_movielenslatest_x1/HOFM_movielenslatest_x1_tuner_config_03/model_config.yaml b/ranking/ctr/HOFM/HOFM_movielenslatest_x1/HOFM_movielenslatest_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_movielenslatest_x1/HOFM_movielenslatest_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/HOFM/HOFM_movielenslatest_x1/HOFM_movielenslatest_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_movielenslatest_x1/README.md b/ranking/ctr/HOFM/HOFM_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_movielenslatest_x1/README.md rename to ranking/ctr/HOFM/HOFM_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_movielenslatest_x1/environments.txt b/ranking/ctr/HOFM/HOFM_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_movielenslatest_x1/environments.txt rename to ranking/ctr/HOFM/HOFM_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_movielenslatest_x1/results.csv b/ranking/ctr/HOFM/HOFM_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_movielenslatest_x1/results.csv rename to ranking/ctr/HOFM/HOFM_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/HOFM/HOFM_movielenslatest_x1/run_expid.py b/ranking/ctr/HOFM/HOFM_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/HOFM/HOFM_movielenslatest_x1/run_expid.py rename to ranking/ctr/HOFM/HOFM_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_001/InterHAt_avazu_x4_027_db66045d.log b/ranking/ctr/InterHAt/InterHAt_avazu_x4_001/InterHAt_avazu_x4_027_db66045d.log similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_001/InterHAt_avazu_x4_027_db66045d.log rename to ranking/ctr/InterHAt/InterHAt_avazu_x4_001/InterHAt_avazu_x4_027_db66045d.log diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_001/InterHAt_avazu_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/InterHAt/InterHAt_avazu_x4_001/InterHAt_avazu_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_001/InterHAt_avazu_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/InterHAt/InterHAt_avazu_x4_001/InterHAt_avazu_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_001/InterHAt_avazu_x4_tuner_config_02/model_config.yaml b/ranking/ctr/InterHAt/InterHAt_avazu_x4_001/InterHAt_avazu_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_001/InterHAt_avazu_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/InterHAt/InterHAt_avazu_x4_001/InterHAt_avazu_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_001/README.md b/ranking/ctr/InterHAt/InterHAt_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_001/README.md rename to ranking/ctr/InterHAt/InterHAt_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_001/environments.txt b/ranking/ctr/InterHAt/InterHAt_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_001/environments.txt rename to ranking/ctr/InterHAt/InterHAt_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_001/results.csv b/ranking/ctr/InterHAt/InterHAt_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_001/results.csv rename to ranking/ctr/InterHAt/InterHAt_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_001/run_expid.py b/ranking/ctr/InterHAt/InterHAt_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_001/run_expid.py rename to ranking/ctr/InterHAt/InterHAt_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_002/InterHAt_avazu_x4_009_34e90a84.log b/ranking/ctr/InterHAt/InterHAt_avazu_x4_002/InterHAt_avazu_x4_009_34e90a84.log similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_002/InterHAt_avazu_x4_009_34e90a84.log rename to ranking/ctr/InterHAt/InterHAt_avazu_x4_002/InterHAt_avazu_x4_009_34e90a84.log diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_002/InterHAt_avazu_x4_tuner_config_05/dataset_config.yaml b/ranking/ctr/InterHAt/InterHAt_avazu_x4_002/InterHAt_avazu_x4_tuner_config_05/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_002/InterHAt_avazu_x4_tuner_config_05/dataset_config.yaml rename to ranking/ctr/InterHAt/InterHAt_avazu_x4_002/InterHAt_avazu_x4_tuner_config_05/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_002/InterHAt_avazu_x4_tuner_config_05/model_config.yaml b/ranking/ctr/InterHAt/InterHAt_avazu_x4_002/InterHAt_avazu_x4_tuner_config_05/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_002/InterHAt_avazu_x4_tuner_config_05/model_config.yaml rename to ranking/ctr/InterHAt/InterHAt_avazu_x4_002/InterHAt_avazu_x4_tuner_config_05/model_config.yaml diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_002/README.md b/ranking/ctr/InterHAt/InterHAt_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_002/README.md rename to ranking/ctr/InterHAt/InterHAt_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_002/environments.txt b/ranking/ctr/InterHAt/InterHAt_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_002/environments.txt rename to ranking/ctr/InterHAt/InterHAt_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_002/results.csv b/ranking/ctr/InterHAt/InterHAt_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_002/results.csv rename to ranking/ctr/InterHAt/InterHAt_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_002/run_expid.py b/ranking/ctr/InterHAt/InterHAt_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_avazu_x4_002/run_expid.py rename to ranking/ctr/InterHAt/InterHAt_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_001/InterHAt_criteo_x4_006_372a547e.log b/ranking/ctr/InterHAt/InterHAt_criteo_x4_001/InterHAt_criteo_x4_006_372a547e.log similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_001/InterHAt_criteo_x4_006_372a547e.log rename to ranking/ctr/InterHAt/InterHAt_criteo_x4_001/InterHAt_criteo_x4_006_372a547e.log diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_001/InterHAt_criteo_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/InterHAt/InterHAt_criteo_x4_001/InterHAt_criteo_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_001/InterHAt_criteo_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/InterHAt/InterHAt_criteo_x4_001/InterHAt_criteo_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_001/InterHAt_criteo_x4_tuner_config_02/model_config.yaml b/ranking/ctr/InterHAt/InterHAt_criteo_x4_001/InterHAt_criteo_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_001/InterHAt_criteo_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/InterHAt/InterHAt_criteo_x4_001/InterHAt_criteo_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_001/README.md b/ranking/ctr/InterHAt/InterHAt_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_001/README.md rename to ranking/ctr/InterHAt/InterHAt_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_001/environments.txt b/ranking/ctr/InterHAt/InterHAt_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_001/environments.txt rename to ranking/ctr/InterHAt/InterHAt_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_001/results.csv b/ranking/ctr/InterHAt/InterHAt_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_001/results.csv rename to ranking/ctr/InterHAt/InterHAt_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_001/run_expid.py b/ranking/ctr/InterHAt/InterHAt_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_001/run_expid.py rename to ranking/ctr/InterHAt/InterHAt_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_002/InterHAt_criteo_x4_004_3fdd2b78.log b/ranking/ctr/InterHAt/InterHAt_criteo_x4_002/InterHAt_criteo_x4_004_3fdd2b78.log similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_002/InterHAt_criteo_x4_004_3fdd2b78.log rename to ranking/ctr/InterHAt/InterHAt_criteo_x4_002/InterHAt_criteo_x4_004_3fdd2b78.log diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_002/InterHAt_criteo_x4_tuner_config_11/dataset_config.yaml b/ranking/ctr/InterHAt/InterHAt_criteo_x4_002/InterHAt_criteo_x4_tuner_config_11/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_002/InterHAt_criteo_x4_tuner_config_11/dataset_config.yaml rename to ranking/ctr/InterHAt/InterHAt_criteo_x4_002/InterHAt_criteo_x4_tuner_config_11/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_002/InterHAt_criteo_x4_tuner_config_11/model_config.yaml b/ranking/ctr/InterHAt/InterHAt_criteo_x4_002/InterHAt_criteo_x4_tuner_config_11/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_002/InterHAt_criteo_x4_tuner_config_11/model_config.yaml rename to ranking/ctr/InterHAt/InterHAt_criteo_x4_002/InterHAt_criteo_x4_tuner_config_11/model_config.yaml diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_002/README.md b/ranking/ctr/InterHAt/InterHAt_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_002/README.md rename to ranking/ctr/InterHAt/InterHAt_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_002/environments.txt b/ranking/ctr/InterHAt/InterHAt_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_002/environments.txt rename to ranking/ctr/InterHAt/InterHAt_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_002/results.csv b/ranking/ctr/InterHAt/InterHAt_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_002/results.csv rename to ranking/ctr/InterHAt/InterHAt_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_002/run_expid.py b/ranking/ctr/InterHAt/InterHAt_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_criteo_x4_002/run_expid.py rename to ranking/ctr/InterHAt/InterHAt_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_kkbox_x1/InterHAt_kkbox_x1_007_5a759e50.log b/ranking/ctr/InterHAt/InterHAt_kkbox_x1/InterHAt_kkbox_x1_007_5a759e50.log similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_kkbox_x1/InterHAt_kkbox_x1_007_5a759e50.log rename to ranking/ctr/InterHAt/InterHAt_kkbox_x1/InterHAt_kkbox_x1_007_5a759e50.log diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_kkbox_x1/InterHAt_kkbox_x1_tuner_config_05/dataset_config.yaml b/ranking/ctr/InterHAt/InterHAt_kkbox_x1/InterHAt_kkbox_x1_tuner_config_05/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_kkbox_x1/InterHAt_kkbox_x1_tuner_config_05/dataset_config.yaml rename to ranking/ctr/InterHAt/InterHAt_kkbox_x1/InterHAt_kkbox_x1_tuner_config_05/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_kkbox_x1/InterHAt_kkbox_x1_tuner_config_05/model_config.yaml b/ranking/ctr/InterHAt/InterHAt_kkbox_x1/InterHAt_kkbox_x1_tuner_config_05/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_kkbox_x1/InterHAt_kkbox_x1_tuner_config_05/model_config.yaml rename to ranking/ctr/InterHAt/InterHAt_kkbox_x1/InterHAt_kkbox_x1_tuner_config_05/model_config.yaml diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_kkbox_x1/README.md b/ranking/ctr/InterHAt/InterHAt_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_kkbox_x1/README.md rename to ranking/ctr/InterHAt/InterHAt_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_kkbox_x1/environments.txt b/ranking/ctr/InterHAt/InterHAt_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_kkbox_x1/environments.txt rename to ranking/ctr/InterHAt/InterHAt_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_kkbox_x1/results.csv b/ranking/ctr/InterHAt/InterHAt_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_kkbox_x1/results.csv rename to ranking/ctr/InterHAt/InterHAt_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/InterHAt/InterHAt_kkbox_x1/run_expid.py b/ranking/ctr/InterHAt/InterHAt_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/InterHAt/InterHAt_kkbox_x1/run_expid.py rename to ranking/ctr/InterHAt/InterHAt_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x1/LR_avazu_x1_007_c7701820.log b/ranking/ctr/LR/LR_avazu_x1/LR_avazu_x1_007_c7701820.log similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x1/LR_avazu_x1_007_c7701820.log rename to ranking/ctr/LR/LR_avazu_x1/LR_avazu_x1_007_c7701820.log diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x1/LR_avazu_x1_tuner_config_01.csv b/ranking/ctr/LR/LR_avazu_x1/LR_avazu_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x1/LR_avazu_x1_tuner_config_01.csv rename to ranking/ctr/LR/LR_avazu_x1/LR_avazu_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x1/LR_avazu_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/LR/LR_avazu_x1/LR_avazu_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x1/LR_avazu_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/LR/LR_avazu_x1/LR_avazu_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x1/LR_avazu_x1_tuner_config_01/model_config.yaml b/ranking/ctr/LR/LR_avazu_x1/LR_avazu_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x1/LR_avazu_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/LR/LR_avazu_x1/LR_avazu_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x1/README.md b/ranking/ctr/LR/LR_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x1/README.md rename to ranking/ctr/LR/LR_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x1/environments.txt b/ranking/ctr/LR/LR_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x1/environments.txt rename to ranking/ctr/LR/LR_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x1/results.csv b/ranking/ctr/LR/LR_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x1/results.csv rename to ranking/ctr/LR/LR_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x1/run_expid.py b/ranking/ctr/LR/LR_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x1/run_expid.py rename to ranking/ctr/LR/LR_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x4_001/LR_avazu_x4_005_93043d62.log b/ranking/ctr/LR/LR_avazu_x4_001/LR_avazu_x4_005_93043d62.log similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x4_001/LR_avazu_x4_005_93043d62.log rename to ranking/ctr/LR/LR_avazu_x4_001/LR_avazu_x4_005_93043d62.log diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x4_001/LR_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/LR/LR_avazu_x4_001/LR_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x4_001/LR_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/LR/LR_avazu_x4_001/LR_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x4_001/LR_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/LR/LR_avazu_x4_001/LR_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x4_001/LR_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/LR/LR_avazu_x4_001/LR_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x4_001/README.md b/ranking/ctr/LR/LR_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x4_001/README.md rename to ranking/ctr/LR/LR_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x4_001/environments.txt b/ranking/ctr/LR/LR_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x4_001/environments.txt rename to ranking/ctr/LR/LR_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x4_001/results.csv b/ranking/ctr/LR/LR_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x4_001/results.csv rename to ranking/ctr/LR/LR_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x4_001/run_expid.py b/ranking/ctr/LR/LR_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x4_001/run_expid.py rename to ranking/ctr/LR/LR_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x4_002/LR_avazu_x4_003_509815a1.log b/ranking/ctr/LR/LR_avazu_x4_002/LR_avazu_x4_003_509815a1.log similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x4_002/LR_avazu_x4_003_509815a1.log rename to ranking/ctr/LR/LR_avazu_x4_002/LR_avazu_x4_003_509815a1.log diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x4_002/LR_avazu_x4_tuner_config_04/dataset_config.yaml b/ranking/ctr/LR/LR_avazu_x4_002/LR_avazu_x4_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x4_002/LR_avazu_x4_tuner_config_04/dataset_config.yaml rename to ranking/ctr/LR/LR_avazu_x4_002/LR_avazu_x4_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x4_002/LR_avazu_x4_tuner_config_04/model_config.yaml b/ranking/ctr/LR/LR_avazu_x4_002/LR_avazu_x4_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x4_002/LR_avazu_x4_tuner_config_04/model_config.yaml rename to ranking/ctr/LR/LR_avazu_x4_002/LR_avazu_x4_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x4_002/README.md b/ranking/ctr/LR/LR_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x4_002/README.md rename to ranking/ctr/LR/LR_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x4_002/environments.txt b/ranking/ctr/LR/LR_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x4_002/environments.txt rename to ranking/ctr/LR/LR_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x4_002/results.csv b/ranking/ctr/LR/LR_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x4_002/results.csv rename to ranking/ctr/LR/LR_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/LR/LR_avazu_x4_002/run_expid.py b/ranking/ctr/LR/LR_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_avazu_x4_002/run_expid.py rename to ranking/ctr/LR/LR_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x1/LR_criteo_x1_008_9581f586.log b/ranking/ctr/LR/LR_criteo_x1/LR_criteo_x1_008_9581f586.log similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x1/LR_criteo_x1_008_9581f586.log rename to ranking/ctr/LR/LR_criteo_x1/LR_criteo_x1_008_9581f586.log diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x1/LR_criteo_x1_tuner_config_01.csv b/ranking/ctr/LR/LR_criteo_x1/LR_criteo_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x1/LR_criteo_x1_tuner_config_01.csv rename to ranking/ctr/LR/LR_criteo_x1/LR_criteo_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x1/LR_criteo_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/LR/LR_criteo_x1/LR_criteo_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x1/LR_criteo_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/LR/LR_criteo_x1/LR_criteo_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x1/LR_criteo_x1_tuner_config_01/model_config.yaml b/ranking/ctr/LR/LR_criteo_x1/LR_criteo_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x1/LR_criteo_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/LR/LR_criteo_x1/LR_criteo_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x1/README.md b/ranking/ctr/LR/LR_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x1/README.md rename to ranking/ctr/LR/LR_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x1/environments.txt b/ranking/ctr/LR/LR_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x1/environments.txt rename to ranking/ctr/LR/LR_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x1/results.csv b/ranking/ctr/LR/LR_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x1/results.csv rename to ranking/ctr/LR/LR_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x1/run_expid.py b/ranking/ctr/LR/LR_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x1/run_expid.py rename to ranking/ctr/LR/LR_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x4_001/LR_criteo_x4_003_76cc7982.log b/ranking/ctr/LR/LR_criteo_x4_001/LR_criteo_x4_003_76cc7982.log similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x4_001/LR_criteo_x4_003_76cc7982.log rename to ranking/ctr/LR/LR_criteo_x4_001/LR_criteo_x4_003_76cc7982.log diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x4_001/LR_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/LR/LR_criteo_x4_001/LR_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x4_001/LR_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/LR/LR_criteo_x4_001/LR_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x4_001/LR_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/LR/LR_criteo_x4_001/LR_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x4_001/LR_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/LR/LR_criteo_x4_001/LR_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x4_001/README.md b/ranking/ctr/LR/LR_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x4_001/README.md rename to ranking/ctr/LR/LR_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x4_001/environments.txt b/ranking/ctr/LR/LR_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x4_001/environments.txt rename to ranking/ctr/LR/LR_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x4_001/results.csv b/ranking/ctr/LR/LR_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x4_001/results.csv rename to ranking/ctr/LR/LR_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x4_001/run_expid.py b/ranking/ctr/LR/LR_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x4_001/run_expid.py rename to ranking/ctr/LR/LR_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x4_002/LR_criteo_x4_003_e33124a6.log b/ranking/ctr/LR/LR_criteo_x4_002/LR_criteo_x4_003_e33124a6.log similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x4_002/LR_criteo_x4_003_e33124a6.log rename to ranking/ctr/LR/LR_criteo_x4_002/LR_criteo_x4_003_e33124a6.log diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x4_002/LR_criteo_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/LR/LR_criteo_x4_002/LR_criteo_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x4_002/LR_criteo_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/LR/LR_criteo_x4_002/LR_criteo_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x4_002/LR_criteo_x4_tuner_config_02/model_config.yaml b/ranking/ctr/LR/LR_criteo_x4_002/LR_criteo_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x4_002/LR_criteo_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/LR/LR_criteo_x4_002/LR_criteo_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x4_002/README.md b/ranking/ctr/LR/LR_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x4_002/README.md rename to ranking/ctr/LR/LR_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x4_002/environments.txt b/ranking/ctr/LR/LR_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x4_002/environments.txt rename to ranking/ctr/LR/LR_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x4_002/results.csv b/ranking/ctr/LR/LR_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x4_002/results.csv rename to ranking/ctr/LR/LR_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/LR/LR_criteo_x4_002/run_expid.py b/ranking/ctr/LR/LR_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_criteo_x4_002/run_expid.py rename to ranking/ctr/LR/LR_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/LR/LR_frappe_x1/LR_frappe_x1_001_ff730f43.log b/ranking/ctr/LR/LR_frappe_x1/LR_frappe_x1_001_ff730f43.log similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_frappe_x1/LR_frappe_x1_001_ff730f43.log rename to ranking/ctr/LR/LR_frappe_x1/LR_frappe_x1_001_ff730f43.log diff --git a/ctr_prediction/benchmarks/LR/LR_frappe_x1/LR_frappe_x1_tuner_config_01.csv b/ranking/ctr/LR/LR_frappe_x1/LR_frappe_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_frappe_x1/LR_frappe_x1_tuner_config_01.csv rename to ranking/ctr/LR/LR_frappe_x1/LR_frappe_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/LR/LR_frappe_x1/LR_frappe_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/LR/LR_frappe_x1/LR_frappe_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_frappe_x1/LR_frappe_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/LR/LR_frappe_x1/LR_frappe_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/LR/LR_frappe_x1/LR_frappe_x1_tuner_config_01/model_config.yaml b/ranking/ctr/LR/LR_frappe_x1/LR_frappe_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_frappe_x1/LR_frappe_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/LR/LR_frappe_x1/LR_frappe_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/LR/LR_frappe_x1/README.md b/ranking/ctr/LR/LR_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_frappe_x1/README.md rename to ranking/ctr/LR/LR_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/LR/LR_frappe_x1/environments.txt b/ranking/ctr/LR/LR_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_frappe_x1/environments.txt rename to ranking/ctr/LR/LR_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/LR/LR_frappe_x1/results.csv b/ranking/ctr/LR/LR_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_frappe_x1/results.csv rename to ranking/ctr/LR/LR_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/LR/LR_frappe_x1/run_expid.py b/ranking/ctr/LR/LR_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_frappe_x1/run_expid.py rename to ranking/ctr/LR/LR_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/LR/LR_kkbox_x1/LR_kkbox_x1_008_aab85bd2.log b/ranking/ctr/LR/LR_kkbox_x1/LR_kkbox_x1_008_aab85bd2.log similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_kkbox_x1/LR_kkbox_x1_008_aab85bd2.log rename to ranking/ctr/LR/LR_kkbox_x1/LR_kkbox_x1_008_aab85bd2.log diff --git a/ctr_prediction/benchmarks/LR/LR_kkbox_x1/LR_kkbox_x1_tuner_config_06/dataset_config.yaml b/ranking/ctr/LR/LR_kkbox_x1/LR_kkbox_x1_tuner_config_06/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_kkbox_x1/LR_kkbox_x1_tuner_config_06/dataset_config.yaml rename to ranking/ctr/LR/LR_kkbox_x1/LR_kkbox_x1_tuner_config_06/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/LR/LR_kkbox_x1/LR_kkbox_x1_tuner_config_06/model_config.yaml b/ranking/ctr/LR/LR_kkbox_x1/LR_kkbox_x1_tuner_config_06/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_kkbox_x1/LR_kkbox_x1_tuner_config_06/model_config.yaml rename to ranking/ctr/LR/LR_kkbox_x1/LR_kkbox_x1_tuner_config_06/model_config.yaml diff --git a/ctr_prediction/benchmarks/LR/LR_kkbox_x1/README.md b/ranking/ctr/LR/LR_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_kkbox_x1/README.md rename to ranking/ctr/LR/LR_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/LR/LR_kkbox_x1/environments.txt b/ranking/ctr/LR/LR_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_kkbox_x1/environments.txt rename to ranking/ctr/LR/LR_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/LR/LR_kkbox_x1/results.csv b/ranking/ctr/LR/LR_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_kkbox_x1/results.csv rename to ranking/ctr/LR/LR_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/LR/LR_kkbox_x1/run_expid.py b/ranking/ctr/LR/LR_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_kkbox_x1/run_expid.py rename to ranking/ctr/LR/LR_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/LR/LR_movielenslatest_x1/LR_movielenslatest_x1_001_7530c4ec.log b/ranking/ctr/LR/LR_movielenslatest_x1/LR_movielenslatest_x1_001_7530c4ec.log similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_movielenslatest_x1/LR_movielenslatest_x1_001_7530c4ec.log rename to ranking/ctr/LR/LR_movielenslatest_x1/LR_movielenslatest_x1_001_7530c4ec.log diff --git a/ctr_prediction/benchmarks/LR/LR_movielenslatest_x1/LR_movielenslatest_x1_tuner_config_02.csv b/ranking/ctr/LR/LR_movielenslatest_x1/LR_movielenslatest_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_movielenslatest_x1/LR_movielenslatest_x1_tuner_config_02.csv rename to ranking/ctr/LR/LR_movielenslatest_x1/LR_movielenslatest_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/LR/LR_movielenslatest_x1/LR_movielenslatest_x1_tuner_config_02/dataset_config (4).yaml b/ranking/ctr/LR/LR_movielenslatest_x1/LR_movielenslatest_x1_tuner_config_02/dataset_config (4).yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_movielenslatest_x1/LR_movielenslatest_x1_tuner_config_02/dataset_config (4).yaml rename to ranking/ctr/LR/LR_movielenslatest_x1/LR_movielenslatest_x1_tuner_config_02/dataset_config (4).yaml diff --git a/ctr_prediction/benchmarks/LR/LR_movielenslatest_x1/LR_movielenslatest_x1_tuner_config_02/model_config (9).yaml b/ranking/ctr/LR/LR_movielenslatest_x1/LR_movielenslatest_x1_tuner_config_02/model_config (9).yaml similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_movielenslatest_x1/LR_movielenslatest_x1_tuner_config_02/model_config (9).yaml rename to ranking/ctr/LR/LR_movielenslatest_x1/LR_movielenslatest_x1_tuner_config_02/model_config (9).yaml diff --git a/ctr_prediction/benchmarks/LR/LR_movielenslatest_x1/README.md b/ranking/ctr/LR/LR_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_movielenslatest_x1/README.md rename to ranking/ctr/LR/LR_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/LR/LR_movielenslatest_x1/environments.txt b/ranking/ctr/LR/LR_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_movielenslatest_x1/environments.txt rename to ranking/ctr/LR/LR_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/LR/LR_movielenslatest_x1/results.csv b/ranking/ctr/LR/LR_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_movielenslatest_x1/results.csv rename to ranking/ctr/LR/LR_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/LR/LR_movielenslatest_x1/run_expid.py b/ranking/ctr/LR/LR_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/LR/LR_movielenslatest_x1/run_expid.py rename to ranking/ctr/LR/LR_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_001/LorentzFM_avazu_x4_004_d25a301d.log b/ranking/ctr/LorentzFM/LorentzFM_avazu_x4_001/LorentzFM_avazu_x4_004_d25a301d.log similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_001/LorentzFM_avazu_x4_004_d25a301d.log rename to ranking/ctr/LorentzFM/LorentzFM_avazu_x4_001/LorentzFM_avazu_x4_004_d25a301d.log diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_001/LorentzFM_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/LorentzFM/LorentzFM_avazu_x4_001/LorentzFM_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_001/LorentzFM_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/LorentzFM/LorentzFM_avazu_x4_001/LorentzFM_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_001/LorentzFM_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/LorentzFM/LorentzFM_avazu_x4_001/LorentzFM_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_001/LorentzFM_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/LorentzFM/LorentzFM_avazu_x4_001/LorentzFM_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_001/README.md b/ranking/ctr/LorentzFM/LorentzFM_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_001/README.md rename to ranking/ctr/LorentzFM/LorentzFM_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_001/environments.txt b/ranking/ctr/LorentzFM/LorentzFM_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_001/environments.txt rename to ranking/ctr/LorentzFM/LorentzFM_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_001/results.csv b/ranking/ctr/LorentzFM/LorentzFM_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_001/results.csv rename to ranking/ctr/LorentzFM/LorentzFM_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_001/run_expid.py b/ranking/ctr/LorentzFM/LorentzFM_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_001/run_expid.py rename to ranking/ctr/LorentzFM/LorentzFM_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_002/LorentzFM_avazu_x4_001_c77e5dd3.log b/ranking/ctr/LorentzFM/LorentzFM_avazu_x4_002/LorentzFM_avazu_x4_001_c77e5dd3.log similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_002/LorentzFM_avazu_x4_001_c77e5dd3.log rename to ranking/ctr/LorentzFM/LorentzFM_avazu_x4_002/LorentzFM_avazu_x4_001_c77e5dd3.log diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_002/LorentzFM_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/LorentzFM/LorentzFM_avazu_x4_002/LorentzFM_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_002/LorentzFM_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/LorentzFM/LorentzFM_avazu_x4_002/LorentzFM_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_002/LorentzFM_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/LorentzFM/LorentzFM_avazu_x4_002/LorentzFM_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_002/LorentzFM_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/LorentzFM/LorentzFM_avazu_x4_002/LorentzFM_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_002/README.md b/ranking/ctr/LorentzFM/LorentzFM_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_002/README.md rename to ranking/ctr/LorentzFM/LorentzFM_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_002/environments.txt b/ranking/ctr/LorentzFM/LorentzFM_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_002/environments.txt rename to ranking/ctr/LorentzFM/LorentzFM_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_002/results.csv b/ranking/ctr/LorentzFM/LorentzFM_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_002/results.csv rename to ranking/ctr/LorentzFM/LorentzFM_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_002/run_expid.py b/ranking/ctr/LorentzFM/LorentzFM_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_avazu_x4_002/run_expid.py rename to ranking/ctr/LorentzFM/LorentzFM_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_001/LorentzFM_criteo_x4_001_342b3588.log b/ranking/ctr/LorentzFM/LorentzFM_criteo_x4_001/LorentzFM_criteo_x4_001_342b3588.log similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_001/LorentzFM_criteo_x4_001_342b3588.log rename to ranking/ctr/LorentzFM/LorentzFM_criteo_x4_001/LorentzFM_criteo_x4_001_342b3588.log diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_001/LorentzFM_criteo_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/LorentzFM/LorentzFM_criteo_x4_001/LorentzFM_criteo_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_001/LorentzFM_criteo_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/LorentzFM/LorentzFM_criteo_x4_001/LorentzFM_criteo_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_001/LorentzFM_criteo_x4_tuner_config_02/model_config.yaml b/ranking/ctr/LorentzFM/LorentzFM_criteo_x4_001/LorentzFM_criteo_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_001/LorentzFM_criteo_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/LorentzFM/LorentzFM_criteo_x4_001/LorentzFM_criteo_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_001/README.md b/ranking/ctr/LorentzFM/LorentzFM_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_001/README.md rename to ranking/ctr/LorentzFM/LorentzFM_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_001/environments.txt b/ranking/ctr/LorentzFM/LorentzFM_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_001/environments.txt rename to ranking/ctr/LorentzFM/LorentzFM_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_001/results.csv b/ranking/ctr/LorentzFM/LorentzFM_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_001/results.csv rename to ranking/ctr/LorentzFM/LorentzFM_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_001/run_expid.py b/ranking/ctr/LorentzFM/LorentzFM_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_001/run_expid.py rename to ranking/ctr/LorentzFM/LorentzFM_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_002/LorentzFM_criteo_x4_002_6d2e078f.log b/ranking/ctr/LorentzFM/LorentzFM_criteo_x4_002/LorentzFM_criteo_x4_002_6d2e078f.log similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_002/LorentzFM_criteo_x4_002_6d2e078f.log rename to ranking/ctr/LorentzFM/LorentzFM_criteo_x4_002/LorentzFM_criteo_x4_002_6d2e078f.log diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_002/LorentzFM_criteo_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/LorentzFM/LorentzFM_criteo_x4_002/LorentzFM_criteo_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_002/LorentzFM_criteo_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/LorentzFM/LorentzFM_criteo_x4_002/LorentzFM_criteo_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_002/LorentzFM_criteo_x4_tuner_config_02/model_config.yaml b/ranking/ctr/LorentzFM/LorentzFM_criteo_x4_002/LorentzFM_criteo_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_002/LorentzFM_criteo_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/LorentzFM/LorentzFM_criteo_x4_002/LorentzFM_criteo_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_002/README.md b/ranking/ctr/LorentzFM/LorentzFM_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_002/README.md rename to ranking/ctr/LorentzFM/LorentzFM_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_002/environments.txt b/ranking/ctr/LorentzFM/LorentzFM_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_002/environments.txt rename to ranking/ctr/LorentzFM/LorentzFM_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_002/results.csv b/ranking/ctr/LorentzFM/LorentzFM_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_002/results.csv rename to ranking/ctr/LorentzFM/LorentzFM_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_002/run_expid.py b/ranking/ctr/LorentzFM/LorentzFM_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_criteo_x4_002/run_expid.py rename to ranking/ctr/LorentzFM/LorentzFM_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_kkbox_x1/LorentzFM_kkbox_x1_003_0455bdef.log b/ranking/ctr/LorentzFM/LorentzFM_kkbox_x1/LorentzFM_kkbox_x1_003_0455bdef.log similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_kkbox_x1/LorentzFM_kkbox_x1_003_0455bdef.log rename to ranking/ctr/LorentzFM/LorentzFM_kkbox_x1/LorentzFM_kkbox_x1_003_0455bdef.log diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_kkbox_x1/LorentzFM_kkbox_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/LorentzFM/LorentzFM_kkbox_x1/LorentzFM_kkbox_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_kkbox_x1/LorentzFM_kkbox_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/LorentzFM/LorentzFM_kkbox_x1/LorentzFM_kkbox_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_kkbox_x1/LorentzFM_kkbox_x1_tuner_config_02/model_config.yaml b/ranking/ctr/LorentzFM/LorentzFM_kkbox_x1/LorentzFM_kkbox_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_kkbox_x1/LorentzFM_kkbox_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/LorentzFM/LorentzFM_kkbox_x1/LorentzFM_kkbox_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_kkbox_x1/README.md b/ranking/ctr/LorentzFM/LorentzFM_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_kkbox_x1/README.md rename to ranking/ctr/LorentzFM/LorentzFM_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_kkbox_x1/environments.txt b/ranking/ctr/LorentzFM/LorentzFM_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_kkbox_x1/environments.txt rename to ranking/ctr/LorentzFM/LorentzFM_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_kkbox_x1/results.csv b/ranking/ctr/LorentzFM/LorentzFM_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_kkbox_x1/results.csv rename to ranking/ctr/LorentzFM/LorentzFM_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/LorentzFM/LorentzFM_kkbox_x1/run_expid.py b/ranking/ctr/LorentzFM/LorentzFM_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/LorentzFM/LorentzFM_kkbox_x1/run_expid.py rename to ranking/ctr/LorentzFM/LorentzFM_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_avazu_x1/MaskNet_avazu_x1_034_99b442f6.log b/ranking/ctr/MaskNet/MaskNet_avazu_x1/MaskNet_avazu_x1_034_99b442f6.log similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_avazu_x1/MaskNet_avazu_x1_034_99b442f6.log rename to ranking/ctr/MaskNet/MaskNet_avazu_x1/MaskNet_avazu_x1_034_99b442f6.log diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_avazu_x1/MaskNet_avazu_x1_tuner_config_02.csv b/ranking/ctr/MaskNet/MaskNet_avazu_x1/MaskNet_avazu_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_avazu_x1/MaskNet_avazu_x1_tuner_config_02.csv rename to ranking/ctr/MaskNet/MaskNet_avazu_x1/MaskNet_avazu_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_avazu_x1/MaskNet_avazu_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/MaskNet/MaskNet_avazu_x1/MaskNet_avazu_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_avazu_x1/MaskNet_avazu_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/MaskNet/MaskNet_avazu_x1/MaskNet_avazu_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_avazu_x1/MaskNet_avazu_x1_tuner_config_02/model_config.yaml b/ranking/ctr/MaskNet/MaskNet_avazu_x1/MaskNet_avazu_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_avazu_x1/MaskNet_avazu_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/MaskNet/MaskNet_avazu_x1/MaskNet_avazu_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_avazu_x1/README.md b/ranking/ctr/MaskNet/MaskNet_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_avazu_x1/README.md rename to ranking/ctr/MaskNet/MaskNet_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_avazu_x1/environments.txt b/ranking/ctr/MaskNet/MaskNet_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_avazu_x1/environments.txt rename to ranking/ctr/MaskNet/MaskNet_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_avazu_x1/results.csv b/ranking/ctr/MaskNet/MaskNet_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_avazu_x1/results.csv rename to ranking/ctr/MaskNet/MaskNet_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_avazu_x1/run_expid.py b/ranking/ctr/MaskNet/MaskNet_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_avazu_x1/run_expid.py rename to ranking/ctr/MaskNet/MaskNet_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_criteo_x1/MaskNet_criteo_x1_008_7071fa3f.log b/ranking/ctr/MaskNet/MaskNet_criteo_x1/MaskNet_criteo_x1_008_7071fa3f.log similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_criteo_x1/MaskNet_criteo_x1_008_7071fa3f.log rename to ranking/ctr/MaskNet/MaskNet_criteo_x1/MaskNet_criteo_x1_008_7071fa3f.log diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_criteo_x1/MaskNet_criteo_x1_tuner_config_03.csv b/ranking/ctr/MaskNet/MaskNet_criteo_x1/MaskNet_criteo_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_criteo_x1/MaskNet_criteo_x1_tuner_config_03.csv rename to ranking/ctr/MaskNet/MaskNet_criteo_x1/MaskNet_criteo_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_criteo_x1/MaskNet_criteo_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/MaskNet/MaskNet_criteo_x1/MaskNet_criteo_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_criteo_x1/MaskNet_criteo_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/MaskNet/MaskNet_criteo_x1/MaskNet_criteo_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_criteo_x1/MaskNet_criteo_x1_tuner_config_03/model_config.yaml b/ranking/ctr/MaskNet/MaskNet_criteo_x1/MaskNet_criteo_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_criteo_x1/MaskNet_criteo_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/MaskNet/MaskNet_criteo_x1/MaskNet_criteo_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_criteo_x1/README.md b/ranking/ctr/MaskNet/MaskNet_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_criteo_x1/README.md rename to ranking/ctr/MaskNet/MaskNet_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_criteo_x1/environments.txt b/ranking/ctr/MaskNet/MaskNet_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_criteo_x1/environments.txt rename to ranking/ctr/MaskNet/MaskNet_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_criteo_x1/results.csv b/ranking/ctr/MaskNet/MaskNet_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_criteo_x1/results.csv rename to ranking/ctr/MaskNet/MaskNet_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_criteo_x1/run_expid.py b/ranking/ctr/MaskNet/MaskNet_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_criteo_x1/run_expid.py rename to ranking/ctr/MaskNet/MaskNet_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_frappe_x1/MaskNet_frappe_x1_028_015da53e.log b/ranking/ctr/MaskNet/MaskNet_frappe_x1/MaskNet_frappe_x1_028_015da53e.log similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_frappe_x1/MaskNet_frappe_x1_028_015da53e.log rename to ranking/ctr/MaskNet/MaskNet_frappe_x1/MaskNet_frappe_x1_028_015da53e.log diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_frappe_x1/MaskNet_frappe_x1_tuner_config_05.csv b/ranking/ctr/MaskNet/MaskNet_frappe_x1/MaskNet_frappe_x1_tuner_config_05.csv similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_frappe_x1/MaskNet_frappe_x1_tuner_config_05.csv rename to ranking/ctr/MaskNet/MaskNet_frappe_x1/MaskNet_frappe_x1_tuner_config_05.csv diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_frappe_x1/MaskNet_frappe_x1_tuner_config_05/dataset_config.yaml b/ranking/ctr/MaskNet/MaskNet_frappe_x1/MaskNet_frappe_x1_tuner_config_05/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_frappe_x1/MaskNet_frappe_x1_tuner_config_05/dataset_config.yaml rename to ranking/ctr/MaskNet/MaskNet_frappe_x1/MaskNet_frappe_x1_tuner_config_05/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_frappe_x1/MaskNet_frappe_x1_tuner_config_05/model_config.yaml b/ranking/ctr/MaskNet/MaskNet_frappe_x1/MaskNet_frappe_x1_tuner_config_05/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_frappe_x1/MaskNet_frappe_x1_tuner_config_05/model_config.yaml rename to ranking/ctr/MaskNet/MaskNet_frappe_x1/MaskNet_frappe_x1_tuner_config_05/model_config.yaml diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_frappe_x1/README.md b/ranking/ctr/MaskNet/MaskNet_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_frappe_x1/README.md rename to ranking/ctr/MaskNet/MaskNet_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_frappe_x1/environments.txt b/ranking/ctr/MaskNet/MaskNet_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_frappe_x1/environments.txt rename to ranking/ctr/MaskNet/MaskNet_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_frappe_x1/results.csv b/ranking/ctr/MaskNet/MaskNet_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_frappe_x1/results.csv rename to ranking/ctr/MaskNet/MaskNet_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_frappe_x1/run_expid.py b/ranking/ctr/MaskNet/MaskNet_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_frappe_x1/run_expid.py rename to ranking/ctr/MaskNet/MaskNet_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_movielenslatest_x1/MaskNet_movielenslatest_x1_010_13a8d29c.log b/ranking/ctr/MaskNet/MaskNet_movielenslatest_x1/MaskNet_movielenslatest_x1_010_13a8d29c.log similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_movielenslatest_x1/MaskNet_movielenslatest_x1_010_13a8d29c.log rename to ranking/ctr/MaskNet/MaskNet_movielenslatest_x1/MaskNet_movielenslatest_x1_010_13a8d29c.log diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_movielenslatest_x1/MaskNet_movielenslatest_x1_tuner_config_03.csv b/ranking/ctr/MaskNet/MaskNet_movielenslatest_x1/MaskNet_movielenslatest_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_movielenslatest_x1/MaskNet_movielenslatest_x1_tuner_config_03.csv rename to ranking/ctr/MaskNet/MaskNet_movielenslatest_x1/MaskNet_movielenslatest_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_movielenslatest_x1/MaskNet_movielenslatest_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/MaskNet/MaskNet_movielenslatest_x1/MaskNet_movielenslatest_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_movielenslatest_x1/MaskNet_movielenslatest_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/MaskNet/MaskNet_movielenslatest_x1/MaskNet_movielenslatest_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_movielenslatest_x1/MaskNet_movielenslatest_x1_tuner_config_03/model_config.yaml b/ranking/ctr/MaskNet/MaskNet_movielenslatest_x1/MaskNet_movielenslatest_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_movielenslatest_x1/MaskNet_movielenslatest_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/MaskNet/MaskNet_movielenslatest_x1/MaskNet_movielenslatest_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_movielenslatest_x1/README.md b/ranking/ctr/MaskNet/MaskNet_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_movielenslatest_x1/README.md rename to ranking/ctr/MaskNet/MaskNet_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_movielenslatest_x1/environments.txt b/ranking/ctr/MaskNet/MaskNet_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_movielenslatest_x1/environments.txt rename to ranking/ctr/MaskNet/MaskNet_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_movielenslatest_x1/results.csv b/ranking/ctr/MaskNet/MaskNet_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_movielenslatest_x1/results.csv rename to ranking/ctr/MaskNet/MaskNet_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/MaskNet/MaskNet_movielenslatest_x1/run_expid.py b/ranking/ctr/MaskNet/MaskNet_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/MaskNet/MaskNet_movielenslatest_x1/run_expid.py rename to ranking/ctr/MaskNet/MaskNet_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x1/NFM_avazu_x1_005_1fe17989.log b/ranking/ctr/NFM/NFM_avazu_x1/NFM_avazu_x1_005_1fe17989.log similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x1/NFM_avazu_x1_005_1fe17989.log rename to ranking/ctr/NFM/NFM_avazu_x1/NFM_avazu_x1_005_1fe17989.log diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x1/NFM_avazu_x1_tuner_config_02.csv b/ranking/ctr/NFM/NFM_avazu_x1/NFM_avazu_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x1/NFM_avazu_x1_tuner_config_02.csv rename to ranking/ctr/NFM/NFM_avazu_x1/NFM_avazu_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x1/NFM_avazu_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/NFM/NFM_avazu_x1/NFM_avazu_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x1/NFM_avazu_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/NFM/NFM_avazu_x1/NFM_avazu_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x1/NFM_avazu_x1_tuner_config_02/model_config.yaml b/ranking/ctr/NFM/NFM_avazu_x1/NFM_avazu_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x1/NFM_avazu_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/NFM/NFM_avazu_x1/NFM_avazu_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x1/README.md b/ranking/ctr/NFM/NFM_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x1/README.md rename to ranking/ctr/NFM/NFM_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x1/environments.txt b/ranking/ctr/NFM/NFM_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x1/environments.txt rename to ranking/ctr/NFM/NFM_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x1/results.csv b/ranking/ctr/NFM/NFM_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x1/results.csv rename to ranking/ctr/NFM/NFM_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x1/run_expid.py b/ranking/ctr/NFM/NFM_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x1/run_expid.py rename to ranking/ctr/NFM/NFM_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x4_001/NFM_avazu_x4_009_d6601a18.log b/ranking/ctr/NFM/NFM_avazu_x4_001/NFM_avazu_x4_009_d6601a18.log similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x4_001/NFM_avazu_x4_009_d6601a18.log rename to ranking/ctr/NFM/NFM_avazu_x4_001/NFM_avazu_x4_009_d6601a18.log diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x4_001/NFM_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/NFM/NFM_avazu_x4_001/NFM_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x4_001/NFM_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/NFM/NFM_avazu_x4_001/NFM_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x4_001/NFM_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/NFM/NFM_avazu_x4_001/NFM_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x4_001/NFM_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/NFM/NFM_avazu_x4_001/NFM_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x4_001/README.md b/ranking/ctr/NFM/NFM_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x4_001/README.md rename to ranking/ctr/NFM/NFM_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x4_001/environments.txt b/ranking/ctr/NFM/NFM_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x4_001/environments.txt rename to ranking/ctr/NFM/NFM_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x4_001/results.csv b/ranking/ctr/NFM/NFM_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x4_001/results.csv rename to ranking/ctr/NFM/NFM_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x4_001/run_expid.py b/ranking/ctr/NFM/NFM_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x4_001/run_expid.py rename to ranking/ctr/NFM/NFM_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x4_002/NFM_avazu_x4_015_2dd318c6.log b/ranking/ctr/NFM/NFM_avazu_x4_002/NFM_avazu_x4_015_2dd318c6.log similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x4_002/NFM_avazu_x4_015_2dd318c6.log rename to ranking/ctr/NFM/NFM_avazu_x4_002/NFM_avazu_x4_015_2dd318c6.log diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x4_002/NFM_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/NFM/NFM_avazu_x4_002/NFM_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x4_002/NFM_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/NFM/NFM_avazu_x4_002/NFM_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x4_002/NFM_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/NFM/NFM_avazu_x4_002/NFM_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x4_002/NFM_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/NFM/NFM_avazu_x4_002/NFM_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x4_002/README.md b/ranking/ctr/NFM/NFM_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x4_002/README.md rename to ranking/ctr/NFM/NFM_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x4_002/environments.txt b/ranking/ctr/NFM/NFM_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x4_002/environments.txt rename to ranking/ctr/NFM/NFM_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x4_002/results.csv b/ranking/ctr/NFM/NFM_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x4_002/results.csv rename to ranking/ctr/NFM/NFM_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/NFM/NFM_avazu_x4_002/run_expid.py b/ranking/ctr/NFM/NFM_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_avazu_x4_002/run_expid.py rename to ranking/ctr/NFM/NFM_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x1/NFM_criteo_x1_002_e07a761d.log b/ranking/ctr/NFM/NFM_criteo_x1/NFM_criteo_x1_002_e07a761d.log similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x1/NFM_criteo_x1_002_e07a761d.log rename to ranking/ctr/NFM/NFM_criteo_x1/NFM_criteo_x1_002_e07a761d.log diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x1/NFM_criteo_x1_tuner_config_02.csv b/ranking/ctr/NFM/NFM_criteo_x1/NFM_criteo_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x1/NFM_criteo_x1_tuner_config_02.csv rename to ranking/ctr/NFM/NFM_criteo_x1/NFM_criteo_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x1/NFM_criteo_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/NFM/NFM_criteo_x1/NFM_criteo_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x1/NFM_criteo_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/NFM/NFM_criteo_x1/NFM_criteo_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x1/NFM_criteo_x1_tuner_config_02/model_config.yaml b/ranking/ctr/NFM/NFM_criteo_x1/NFM_criteo_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x1/NFM_criteo_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/NFM/NFM_criteo_x1/NFM_criteo_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x1/README.md b/ranking/ctr/NFM/NFM_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x1/README.md rename to ranking/ctr/NFM/NFM_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x1/environments.txt b/ranking/ctr/NFM/NFM_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x1/environments.txt rename to ranking/ctr/NFM/NFM_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x1/results.csv b/ranking/ctr/NFM/NFM_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x1/results.csv rename to ranking/ctr/NFM/NFM_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x1/run_expid.py b/ranking/ctr/NFM/NFM_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x1/run_expid.py rename to ranking/ctr/NFM/NFM_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x4_001/NFM_criteo_x4_017_2326425e.log b/ranking/ctr/NFM/NFM_criteo_x4_001/NFM_criteo_x4_017_2326425e.log similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x4_001/NFM_criteo_x4_017_2326425e.log rename to ranking/ctr/NFM/NFM_criteo_x4_001/NFM_criteo_x4_017_2326425e.log diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x4_001/NFM_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/NFM/NFM_criteo_x4_001/NFM_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x4_001/NFM_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/NFM/NFM_criteo_x4_001/NFM_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x4_001/NFM_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/NFM/NFM_criteo_x4_001/NFM_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x4_001/NFM_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/NFM/NFM_criteo_x4_001/NFM_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x4_001/README.md b/ranking/ctr/NFM/NFM_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x4_001/README.md rename to ranking/ctr/NFM/NFM_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x4_001/environments.txt b/ranking/ctr/NFM/NFM_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x4_001/environments.txt rename to ranking/ctr/NFM/NFM_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x4_001/results.csv b/ranking/ctr/NFM/NFM_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x4_001/results.csv rename to ranking/ctr/NFM/NFM_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x4_001/run_expid.py b/ranking/ctr/NFM/NFM_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x4_001/run_expid.py rename to ranking/ctr/NFM/NFM_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x4_002/NFM_criteo_x4_023_54434d6e.log b/ranking/ctr/NFM/NFM_criteo_x4_002/NFM_criteo_x4_023_54434d6e.log similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x4_002/NFM_criteo_x4_023_54434d6e.log rename to ranking/ctr/NFM/NFM_criteo_x4_002/NFM_criteo_x4_023_54434d6e.log diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x4_002/NFM_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/NFM/NFM_criteo_x4_002/NFM_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x4_002/NFM_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/NFM/NFM_criteo_x4_002/NFM_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x4_002/NFM_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/NFM/NFM_criteo_x4_002/NFM_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x4_002/NFM_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/NFM/NFM_criteo_x4_002/NFM_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x4_002/README.md b/ranking/ctr/NFM/NFM_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x4_002/README.md rename to ranking/ctr/NFM/NFM_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x4_002/environments.txt b/ranking/ctr/NFM/NFM_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x4_002/environments.txt rename to ranking/ctr/NFM/NFM_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x4_002/results.csv b/ranking/ctr/NFM/NFM_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x4_002/results.csv rename to ranking/ctr/NFM/NFM_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/NFM/NFM_criteo_x4_002/run_expid.py b/ranking/ctr/NFM/NFM_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_criteo_x4_002/run_expid.py rename to ranking/ctr/NFM/NFM_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/NFM/NFM_frappe_x1/NFM_frappe_x1_012_3ebff53c.log b/ranking/ctr/NFM/NFM_frappe_x1/NFM_frappe_x1_012_3ebff53c.log similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_frappe_x1/NFM_frappe_x1_012_3ebff53c.log rename to ranking/ctr/NFM/NFM_frappe_x1/NFM_frappe_x1_012_3ebff53c.log diff --git a/ctr_prediction/benchmarks/NFM/NFM_frappe_x1/NFM_frappe_x1_tuner_config_01.csv b/ranking/ctr/NFM/NFM_frappe_x1/NFM_frappe_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_frappe_x1/NFM_frappe_x1_tuner_config_01.csv rename to ranking/ctr/NFM/NFM_frappe_x1/NFM_frappe_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/NFM/NFM_frappe_x1/NFM_frappe_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/NFM/NFM_frappe_x1/NFM_frappe_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_frappe_x1/NFM_frappe_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/NFM/NFM_frappe_x1/NFM_frappe_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_frappe_x1/NFM_frappe_x1_tuner_config_01/model_config.yaml b/ranking/ctr/NFM/NFM_frappe_x1/NFM_frappe_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_frappe_x1/NFM_frappe_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/NFM/NFM_frappe_x1/NFM_frappe_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_frappe_x1/README.md b/ranking/ctr/NFM/NFM_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_frappe_x1/README.md rename to ranking/ctr/NFM/NFM_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/NFM/NFM_frappe_x1/environments.txt b/ranking/ctr/NFM/NFM_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_frappe_x1/environments.txt rename to ranking/ctr/NFM/NFM_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/NFM/NFM_frappe_x1/results.csv b/ranking/ctr/NFM/NFM_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_frappe_x1/results.csv rename to ranking/ctr/NFM/NFM_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/NFM/NFM_frappe_x1/run_expid.py b/ranking/ctr/NFM/NFM_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_frappe_x1/run_expid.py rename to ranking/ctr/NFM/NFM_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/NFM/NFM_kkbox_x1/NFM_kkbox_x1_015_cbae7b1e.log b/ranking/ctr/NFM/NFM_kkbox_x1/NFM_kkbox_x1_015_cbae7b1e.log similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_kkbox_x1/NFM_kkbox_x1_015_cbae7b1e.log rename to ranking/ctr/NFM/NFM_kkbox_x1/NFM_kkbox_x1_015_cbae7b1e.log diff --git a/ctr_prediction/benchmarks/NFM/NFM_kkbox_x1/NFM_kkbox_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/NFM/NFM_kkbox_x1/NFM_kkbox_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_kkbox_x1/NFM_kkbox_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/NFM/NFM_kkbox_x1/NFM_kkbox_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_kkbox_x1/NFM_kkbox_x1_tuner_config_03/model_config.yaml b/ranking/ctr/NFM/NFM_kkbox_x1/NFM_kkbox_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_kkbox_x1/NFM_kkbox_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/NFM/NFM_kkbox_x1/NFM_kkbox_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_kkbox_x1/README.md b/ranking/ctr/NFM/NFM_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_kkbox_x1/README.md rename to ranking/ctr/NFM/NFM_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/NFM/NFM_kkbox_x1/environments.txt b/ranking/ctr/NFM/NFM_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_kkbox_x1/environments.txt rename to ranking/ctr/NFM/NFM_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/NFM/NFM_kkbox_x1/results.csv b/ranking/ctr/NFM/NFM_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_kkbox_x1/results.csv rename to ranking/ctr/NFM/NFM_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/NFM/NFM_kkbox_x1/run_expid.py b/ranking/ctr/NFM/NFM_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_kkbox_x1/run_expid.py rename to ranking/ctr/NFM/NFM_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/NFM/NFM_movielenslatest_x1/NFM_movielenslatest_x1_010_f3d546bc.log b/ranking/ctr/NFM/NFM_movielenslatest_x1/NFM_movielenslatest_x1_010_f3d546bc.log similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_movielenslatest_x1/NFM_movielenslatest_x1_010_f3d546bc.log rename to ranking/ctr/NFM/NFM_movielenslatest_x1/NFM_movielenslatest_x1_010_f3d546bc.log diff --git a/ctr_prediction/benchmarks/NFM/NFM_movielenslatest_x1/NFM_movielenslatest_x1_tuner_config_01.csv b/ranking/ctr/NFM/NFM_movielenslatest_x1/NFM_movielenslatest_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_movielenslatest_x1/NFM_movielenslatest_x1_tuner_config_01.csv rename to ranking/ctr/NFM/NFM_movielenslatest_x1/NFM_movielenslatest_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/NFM/NFM_movielenslatest_x1/NFM_movielenslatest_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/NFM/NFM_movielenslatest_x1/NFM_movielenslatest_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_movielenslatest_x1/NFM_movielenslatest_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/NFM/NFM_movielenslatest_x1/NFM_movielenslatest_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_movielenslatest_x1/NFM_movielenslatest_x1_tuner_config_01/model_config.yaml b/ranking/ctr/NFM/NFM_movielenslatest_x1/NFM_movielenslatest_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_movielenslatest_x1/NFM_movielenslatest_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/NFM/NFM_movielenslatest_x1/NFM_movielenslatest_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/NFM/NFM_movielenslatest_x1/README.md b/ranking/ctr/NFM/NFM_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_movielenslatest_x1/README.md rename to ranking/ctr/NFM/NFM_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/NFM/NFM_movielenslatest_x1/environments.txt b/ranking/ctr/NFM/NFM_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_movielenslatest_x1/environments.txt rename to ranking/ctr/NFM/NFM_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/NFM/NFM_movielenslatest_x1/results.csv b/ranking/ctr/NFM/NFM_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_movielenslatest_x1/results.csv rename to ranking/ctr/NFM/NFM_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/NFM/NFM_movielenslatest_x1/run_expid.py b/ranking/ctr/NFM/NFM_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/NFM/NFM_movielenslatest_x1/run_expid.py rename to ranking/ctr/NFM/NFM_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/ONN/ONN_avazu_x4_001/ONN_avazu_x4_006_614049da.log b/ranking/ctr/ONN/ONN_avazu_x4_001/ONN_avazu_x4_006_614049da.log similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_avazu_x4_001/ONN_avazu_x4_006_614049da.log rename to ranking/ctr/ONN/ONN_avazu_x4_001/ONN_avazu_x4_006_614049da.log diff --git a/ctr_prediction/benchmarks/ONN/ONN_avazu_x4_001/ONN_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/ONN/ONN_avazu_x4_001/ONN_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_avazu_x4_001/ONN_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/ONN/ONN_avazu_x4_001/ONN_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/ONN/ONN_avazu_x4_001/ONN_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/ONN/ONN_avazu_x4_001/ONN_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_avazu_x4_001/ONN_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/ONN/ONN_avazu_x4_001/ONN_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/ONN/ONN_avazu_x4_001/README.md b/ranking/ctr/ONN/ONN_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_avazu_x4_001/README.md rename to ranking/ctr/ONN/ONN_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/ONN/ONN_avazu_x4_001/environments.txt b/ranking/ctr/ONN/ONN_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_avazu_x4_001/environments.txt rename to ranking/ctr/ONN/ONN_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/ONN/ONN_avazu_x4_001/results.csv b/ranking/ctr/ONN/ONN_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_avazu_x4_001/results.csv rename to ranking/ctr/ONN/ONN_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/ONN/ONN_avazu_x4_001/run_expid.py b/ranking/ctr/ONN/ONN_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_avazu_x4_001/run_expid.py rename to ranking/ctr/ONN/ONN_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/ONN/ONN_avazu_x4_002/ONN_avazu_x4_036_23b6ec31.log b/ranking/ctr/ONN/ONN_avazu_x4_002/ONN_avazu_x4_036_23b6ec31.log similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_avazu_x4_002/ONN_avazu_x4_036_23b6ec31.log rename to ranking/ctr/ONN/ONN_avazu_x4_002/ONN_avazu_x4_036_23b6ec31.log diff --git a/ctr_prediction/benchmarks/ONN/ONN_avazu_x4_002/ONN_avazu_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/ONN/ONN_avazu_x4_002/ONN_avazu_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_avazu_x4_002/ONN_avazu_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/ONN/ONN_avazu_x4_002/ONN_avazu_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/ONN/ONN_avazu_x4_002/ONN_avazu_x4_tuner_config_02/model_config.yaml b/ranking/ctr/ONN/ONN_avazu_x4_002/ONN_avazu_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_avazu_x4_002/ONN_avazu_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/ONN/ONN_avazu_x4_002/ONN_avazu_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/ONN/ONN_avazu_x4_002/README.md b/ranking/ctr/ONN/ONN_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_avazu_x4_002/README.md rename to ranking/ctr/ONN/ONN_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/ONN/ONN_avazu_x4_002/environments.txt b/ranking/ctr/ONN/ONN_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_avazu_x4_002/environments.txt rename to ranking/ctr/ONN/ONN_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/ONN/ONN_avazu_x4_002/results.csv b/ranking/ctr/ONN/ONN_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_avazu_x4_002/results.csv rename to ranking/ctr/ONN/ONN_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/ONN/ONN_avazu_x4_002/run_expid.py b/ranking/ctr/ONN/ONN_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_avazu_x4_002/run_expid.py rename to ranking/ctr/ONN/ONN_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/ONN/ONN_criteo_x4_001/ONN_criteo_x4_002_e3541fe7.log b/ranking/ctr/ONN/ONN_criteo_x4_001/ONN_criteo_x4_002_e3541fe7.log similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_criteo_x4_001/ONN_criteo_x4_002_e3541fe7.log rename to ranking/ctr/ONN/ONN_criteo_x4_001/ONN_criteo_x4_002_e3541fe7.log diff --git a/ctr_prediction/benchmarks/ONN/ONN_criteo_x4_001/ONN_criteo_x4_tuner_config_03/dataset_config.yaml b/ranking/ctr/ONN/ONN_criteo_x4_001/ONN_criteo_x4_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_criteo_x4_001/ONN_criteo_x4_tuner_config_03/dataset_config.yaml rename to ranking/ctr/ONN/ONN_criteo_x4_001/ONN_criteo_x4_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/ONN/ONN_criteo_x4_001/ONN_criteo_x4_tuner_config_03/model_config.yaml b/ranking/ctr/ONN/ONN_criteo_x4_001/ONN_criteo_x4_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_criteo_x4_001/ONN_criteo_x4_tuner_config_03/model_config.yaml rename to ranking/ctr/ONN/ONN_criteo_x4_001/ONN_criteo_x4_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/ONN/ONN_criteo_x4_001/README.md b/ranking/ctr/ONN/ONN_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_criteo_x4_001/README.md rename to ranking/ctr/ONN/ONN_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/ONN/ONN_criteo_x4_001/environments.txt b/ranking/ctr/ONN/ONN_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_criteo_x4_001/environments.txt rename to ranking/ctr/ONN/ONN_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/ONN/ONN_criteo_x4_001/results.csv b/ranking/ctr/ONN/ONN_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_criteo_x4_001/results.csv rename to ranking/ctr/ONN/ONN_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/ONN/ONN_criteo_x4_001/run_expid.py b/ranking/ctr/ONN/ONN_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_criteo_x4_001/run_expid.py rename to ranking/ctr/ONN/ONN_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/ONN/ONN_criteo_x4_002/ONN_criteo_x4_106_7a261a01.log b/ranking/ctr/ONN/ONN_criteo_x4_002/ONN_criteo_x4_106_7a261a01.log similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_criteo_x4_002/ONN_criteo_x4_106_7a261a01.log rename to ranking/ctr/ONN/ONN_criteo_x4_002/ONN_criteo_x4_106_7a261a01.log diff --git a/ctr_prediction/benchmarks/ONN/ONN_criteo_x4_002/ONN_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/ONN/ONN_criteo_x4_002/ONN_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_criteo_x4_002/ONN_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/ONN/ONN_criteo_x4_002/ONN_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/ONN/ONN_criteo_x4_002/ONN_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/ONN/ONN_criteo_x4_002/ONN_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_criteo_x4_002/ONN_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/ONN/ONN_criteo_x4_002/ONN_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/ONN/ONN_criteo_x4_002/README.md b/ranking/ctr/ONN/ONN_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_criteo_x4_002/README.md rename to ranking/ctr/ONN/ONN_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/ONN/ONN_criteo_x4_002/environments.txt b/ranking/ctr/ONN/ONN_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_criteo_x4_002/environments.txt rename to ranking/ctr/ONN/ONN_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/ONN/ONN_criteo_x4_002/results.csv b/ranking/ctr/ONN/ONN_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_criteo_x4_002/results.csv rename to ranking/ctr/ONN/ONN_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/ONN/ONN_criteo_x4_002/run_expid.py b/ranking/ctr/ONN/ONN_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_criteo_x4_002/run_expid.py rename to ranking/ctr/ONN/ONN_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/ONN/ONN_kkbox_x1/ONN_kkbox_x1_024_650b448f.log b/ranking/ctr/ONN/ONN_kkbox_x1/ONN_kkbox_x1_024_650b448f.log similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_kkbox_x1/ONN_kkbox_x1_024_650b448f.log rename to ranking/ctr/ONN/ONN_kkbox_x1/ONN_kkbox_x1_024_650b448f.log diff --git a/ctr_prediction/benchmarks/ONN/ONN_kkbox_x1/ONN_kkbox_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/ONN/ONN_kkbox_x1/ONN_kkbox_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_kkbox_x1/ONN_kkbox_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/ONN/ONN_kkbox_x1/ONN_kkbox_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/ONN/ONN_kkbox_x1/ONN_kkbox_x1_tuner_config_02/model_config.yaml b/ranking/ctr/ONN/ONN_kkbox_x1/ONN_kkbox_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_kkbox_x1/ONN_kkbox_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/ONN/ONN_kkbox_x1/ONN_kkbox_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/ONN/ONN_kkbox_x1/README.md b/ranking/ctr/ONN/ONN_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_kkbox_x1/README.md rename to ranking/ctr/ONN/ONN_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/ONN/ONN_kkbox_x1/environments.txt b/ranking/ctr/ONN/ONN_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_kkbox_x1/environments.txt rename to ranking/ctr/ONN/ONN_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/ONN/ONN_kkbox_x1/results.csv b/ranking/ctr/ONN/ONN_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_kkbox_x1/results.csv rename to ranking/ctr/ONN/ONN_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/ONN/ONN_kkbox_x1/run_expid.py b/ranking/ctr/ONN/ONN_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/ONN/ONN_kkbox_x1/run_expid.py rename to ranking/ctr/ONN/ONN_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x1/PNN_avazu_x1_002_8a91559c.log b/ranking/ctr/PNN/IPNN_avazu_x1/PNN_avazu_x1_002_8a91559c.log similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x1/PNN_avazu_x1_002_8a91559c.log rename to ranking/ctr/PNN/IPNN_avazu_x1/PNN_avazu_x1_002_8a91559c.log diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x1/PNN_avazu_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/PNN/IPNN_avazu_x1/PNN_avazu_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x1/PNN_avazu_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/PNN/IPNN_avazu_x1/PNN_avazu_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x1/PNN_avazu_x1_tuner_config_02/model_config.yaml b/ranking/ctr/PNN/IPNN_avazu_x1/PNN_avazu_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x1/PNN_avazu_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/PNN/IPNN_avazu_x1/PNN_avazu_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x1/README.md b/ranking/ctr/PNN/IPNN_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x1/README.md rename to ranking/ctr/PNN/IPNN_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x1/environments.txt b/ranking/ctr/PNN/IPNN_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x1/environments.txt rename to ranking/ctr/PNN/IPNN_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x1/results.csv b/ranking/ctr/PNN/IPNN_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x1/results.csv rename to ranking/ctr/PNN/IPNN_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x1/run_expid.py b/ranking/ctr/PNN/IPNN_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x1/run_expid.py rename to ranking/ctr/PNN/IPNN_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_001/PNN_avazu_x4_009_3c99a8b5.log b/ranking/ctr/PNN/IPNN_avazu_x4_001/PNN_avazu_x4_009_3c99a8b5.log similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_001/PNN_avazu_x4_009_3c99a8b5.log rename to ranking/ctr/PNN/IPNN_avazu_x4_001/PNN_avazu_x4_009_3c99a8b5.log diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_001/PNN_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/PNN/IPNN_avazu_x4_001/PNN_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_001/PNN_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/PNN/IPNN_avazu_x4_001/PNN_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_001/PNN_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/PNN/IPNN_avazu_x4_001/PNN_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_001/PNN_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/PNN/IPNN_avazu_x4_001/PNN_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_001/README.md b/ranking/ctr/PNN/IPNN_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_001/README.md rename to ranking/ctr/PNN/IPNN_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_001/environments.txt b/ranking/ctr/PNN/IPNN_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_001/environments.txt rename to ranking/ctr/PNN/IPNN_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_001/results.csv b/ranking/ctr/PNN/IPNN_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_001/results.csv rename to ranking/ctr/PNN/IPNN_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_001/run_expid.py b/ranking/ctr/PNN/IPNN_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_001/run_expid.py rename to ranking/ctr/PNN/IPNN_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_002/PNN_avazu_x4_013_1893e8ce.log b/ranking/ctr/PNN/IPNN_avazu_x4_002/PNN_avazu_x4_013_1893e8ce.log similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_002/PNN_avazu_x4_013_1893e8ce.log rename to ranking/ctr/PNN/IPNN_avazu_x4_002/PNN_avazu_x4_013_1893e8ce.log diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_002/PNN_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/PNN/IPNN_avazu_x4_002/PNN_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_002/PNN_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/PNN/IPNN_avazu_x4_002/PNN_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_002/PNN_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/PNN/IPNN_avazu_x4_002/PNN_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_002/PNN_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/PNN/IPNN_avazu_x4_002/PNN_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_002/README.md b/ranking/ctr/PNN/IPNN_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_002/README.md rename to ranking/ctr/PNN/IPNN_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_002/environments.txt b/ranking/ctr/PNN/IPNN_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_002/environments.txt rename to ranking/ctr/PNN/IPNN_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_002/results.csv b/ranking/ctr/PNN/IPNN_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_002/results.csv rename to ranking/ctr/PNN/IPNN_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_002/run_expid.py b/ranking/ctr/PNN/IPNN_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_avazu_x4_002/run_expid.py rename to ranking/ctr/PNN/IPNN_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x1/PNN_criteo_x1_007_491aff75.log b/ranking/ctr/PNN/IPNN_criteo_x1/PNN_criteo_x1_007_491aff75.log similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x1/PNN_criteo_x1_007_491aff75.log rename to ranking/ctr/PNN/IPNN_criteo_x1/PNN_criteo_x1_007_491aff75.log diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x1/PNN_criteo_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/PNN/IPNN_criteo_x1/PNN_criteo_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x1/PNN_criteo_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/PNN/IPNN_criteo_x1/PNN_criteo_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x1/PNN_criteo_x1_tuner_config_01/model_config.yaml b/ranking/ctr/PNN/IPNN_criteo_x1/PNN_criteo_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x1/PNN_criteo_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/PNN/IPNN_criteo_x1/PNN_criteo_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x1/README.md b/ranking/ctr/PNN/IPNN_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x1/README.md rename to ranking/ctr/PNN/IPNN_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x1/environments.txt b/ranking/ctr/PNN/IPNN_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x1/environments.txt rename to ranking/ctr/PNN/IPNN_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x1/results.csv b/ranking/ctr/PNN/IPNN_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x1/results.csv rename to ranking/ctr/PNN/IPNN_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x1/run_expid.py b/ranking/ctr/PNN/IPNN_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x1/run_expid.py rename to ranking/ctr/PNN/IPNN_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_001/PNN_criteo_x4_002_69a1df28.log b/ranking/ctr/PNN/IPNN_criteo_x4_001/PNN_criteo_x4_002_69a1df28.log similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_001/PNN_criteo_x4_002_69a1df28.log rename to ranking/ctr/PNN/IPNN_criteo_x4_001/PNN_criteo_x4_002_69a1df28.log diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_001/PNN_criteo_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/PNN/IPNN_criteo_x4_001/PNN_criteo_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_001/PNN_criteo_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/PNN/IPNN_criteo_x4_001/PNN_criteo_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_001/PNN_criteo_x4_tuner_config_02/model_config.yaml b/ranking/ctr/PNN/IPNN_criteo_x4_001/PNN_criteo_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_001/PNN_criteo_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/PNN/IPNN_criteo_x4_001/PNN_criteo_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_001/README.md b/ranking/ctr/PNN/IPNN_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_001/README.md rename to ranking/ctr/PNN/IPNN_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_001/environments.txt b/ranking/ctr/PNN/IPNN_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_001/environments.txt rename to ranking/ctr/PNN/IPNN_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_001/results.csv b/ranking/ctr/PNN/IPNN_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_001/results.csv rename to ranking/ctr/PNN/IPNN_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_001/run_expid.py b/ranking/ctr/PNN/IPNN_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_001/run_expid.py rename to ranking/ctr/PNN/IPNN_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_002/PNN_criteo_x4_005_2c82efe5.log b/ranking/ctr/PNN/IPNN_criteo_x4_002/PNN_criteo_x4_005_2c82efe5.log similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_002/PNN_criteo_x4_005_2c82efe5.log rename to ranking/ctr/PNN/IPNN_criteo_x4_002/PNN_criteo_x4_005_2c82efe5.log diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_002/PNN_criteo_x4_tuner_config_04/dataset_config.yaml b/ranking/ctr/PNN/IPNN_criteo_x4_002/PNN_criteo_x4_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_002/PNN_criteo_x4_tuner_config_04/dataset_config.yaml rename to ranking/ctr/PNN/IPNN_criteo_x4_002/PNN_criteo_x4_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_002/PNN_criteo_x4_tuner_config_04/model_config.yaml b/ranking/ctr/PNN/IPNN_criteo_x4_002/PNN_criteo_x4_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_002/PNN_criteo_x4_tuner_config_04/model_config.yaml rename to ranking/ctr/PNN/IPNN_criteo_x4_002/PNN_criteo_x4_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_002/README.md b/ranking/ctr/PNN/IPNN_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_002/README.md rename to ranking/ctr/PNN/IPNN_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_002/environments.txt b/ranking/ctr/PNN/IPNN_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_002/environments.txt rename to ranking/ctr/PNN/IPNN_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_002/results.csv b/ranking/ctr/PNN/IPNN_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_002/results.csv rename to ranking/ctr/PNN/IPNN_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_002/run_expid.py b/ranking/ctr/PNN/IPNN_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_criteo_x4_002/run_expid.py rename to ranking/ctr/PNN/IPNN_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/PNN/IPNN_frappe_x1/PNN_frappe_x1_020_f371a3ab.log b/ranking/ctr/PNN/IPNN_frappe_x1/PNN_frappe_x1_020_f371a3ab.log similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_frappe_x1/PNN_frappe_x1_020_f371a3ab.log rename to ranking/ctr/PNN/IPNN_frappe_x1/PNN_frappe_x1_020_f371a3ab.log diff --git a/ctr_prediction/benchmarks/PNN/IPNN_frappe_x1/PNN_frappe_x1_tuner_config_06/dataset_config.yaml b/ranking/ctr/PNN/IPNN_frappe_x1/PNN_frappe_x1_tuner_config_06/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_frappe_x1/PNN_frappe_x1_tuner_config_06/dataset_config.yaml rename to ranking/ctr/PNN/IPNN_frappe_x1/PNN_frappe_x1_tuner_config_06/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_frappe_x1/PNN_frappe_x1_tuner_config_06/model_config.yaml b/ranking/ctr/PNN/IPNN_frappe_x1/PNN_frappe_x1_tuner_config_06/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_frappe_x1/PNN_frappe_x1_tuner_config_06/model_config.yaml rename to ranking/ctr/PNN/IPNN_frappe_x1/PNN_frappe_x1_tuner_config_06/model_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_frappe_x1/README.md b/ranking/ctr/PNN/IPNN_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_frappe_x1/README.md rename to ranking/ctr/PNN/IPNN_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/PNN/IPNN_frappe_x1/environments.txt b/ranking/ctr/PNN/IPNN_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_frappe_x1/environments.txt rename to ranking/ctr/PNN/IPNN_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/PNN/IPNN_frappe_x1/results.csv b/ranking/ctr/PNN/IPNN_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_frappe_x1/results.csv rename to ranking/ctr/PNN/IPNN_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/PNN/IPNN_frappe_x1/run_expid.py b/ranking/ctr/PNN/IPNN_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_frappe_x1/run_expid.py rename to ranking/ctr/PNN/IPNN_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/PNN/IPNN_kkbox_x1/PNN_kkbox_x1_015_1b4d837a.log b/ranking/ctr/PNN/IPNN_kkbox_x1/PNN_kkbox_x1_015_1b4d837a.log similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_kkbox_x1/PNN_kkbox_x1_015_1b4d837a.log rename to ranking/ctr/PNN/IPNN_kkbox_x1/PNN_kkbox_x1_015_1b4d837a.log diff --git a/ctr_prediction/benchmarks/PNN/IPNN_kkbox_x1/PNN_kkbox_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/PNN/IPNN_kkbox_x1/PNN_kkbox_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_kkbox_x1/PNN_kkbox_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/PNN/IPNN_kkbox_x1/PNN_kkbox_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_kkbox_x1/PNN_kkbox_x1_tuner_config_02/model_config.yaml b/ranking/ctr/PNN/IPNN_kkbox_x1/PNN_kkbox_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_kkbox_x1/PNN_kkbox_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/PNN/IPNN_kkbox_x1/PNN_kkbox_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_kkbox_x1/README.md b/ranking/ctr/PNN/IPNN_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_kkbox_x1/README.md rename to ranking/ctr/PNN/IPNN_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/PNN/IPNN_kkbox_x1/environments.txt b/ranking/ctr/PNN/IPNN_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_kkbox_x1/environments.txt rename to ranking/ctr/PNN/IPNN_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/PNN/IPNN_kkbox_x1/results.csv b/ranking/ctr/PNN/IPNN_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_kkbox_x1/results.csv rename to ranking/ctr/PNN/IPNN_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/PNN/IPNN_kkbox_x1/run_expid.py b/ranking/ctr/PNN/IPNN_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_kkbox_x1/run_expid.py rename to ranking/ctr/PNN/IPNN_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/PNN/IPNN_movielenslatest_x1/PNN_movielenslatest_x1_026_3f8280ec.log b/ranking/ctr/PNN/IPNN_movielenslatest_x1/PNN_movielenslatest_x1_026_3f8280ec.log similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_movielenslatest_x1/PNN_movielenslatest_x1_026_3f8280ec.log rename to ranking/ctr/PNN/IPNN_movielenslatest_x1/PNN_movielenslatest_x1_026_3f8280ec.log diff --git a/ctr_prediction/benchmarks/PNN/IPNN_movielenslatest_x1/PNN_movielenslatest_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/PNN/IPNN_movielenslatest_x1/PNN_movielenslatest_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_movielenslatest_x1/PNN_movielenslatest_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/PNN/IPNN_movielenslatest_x1/PNN_movielenslatest_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_movielenslatest_x1/PNN_movielenslatest_x1_tuner_config_02/model_config.yaml b/ranking/ctr/PNN/IPNN_movielenslatest_x1/PNN_movielenslatest_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_movielenslatest_x1/PNN_movielenslatest_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/PNN/IPNN_movielenslatest_x1/PNN_movielenslatest_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/PNN/IPNN_movielenslatest_x1/README.md b/ranking/ctr/PNN/IPNN_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_movielenslatest_x1/README.md rename to ranking/ctr/PNN/IPNN_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/PNN/IPNN_movielenslatest_x1/environments.txt b/ranking/ctr/PNN/IPNN_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_movielenslatest_x1/environments.txt rename to ranking/ctr/PNN/IPNN_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/PNN/IPNN_movielenslatest_x1/results.csv b/ranking/ctr/PNN/IPNN_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_movielenslatest_x1/results.csv rename to ranking/ctr/PNN/IPNN_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/PNN/IPNN_movielenslatest_x1/run_expid.py b/ranking/ctr/PNN/IPNN_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/PNN/IPNN_movielenslatest_x1/run_expid.py rename to ranking/ctr/PNN/IPNN_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/README.md b/ranking/ctr/README.md similarity index 96% rename from ctr_prediction/benchmarks/README.md rename to ranking/ctr/README.md index 5904e92c..e9a18cbb 100644 --- a/ctr_prediction/benchmarks/README.md +++ b/ranking/ctr/README.md @@ -1,10 +1,6 @@ -# Benchmarks - -To allow extensibility of the BARS benchmark, the code implementation is not binded to any single framework or library. But we strongly recommend the following libraries due to their abundant model implementations. - -+ FuxiCTR (pytorch): https://github.com/xue-pai/FuxiCTR -+ DeepCTR (tensorflow): https://github.com/shenweichen/DeepCTR +# BARS-CTR Benchmark +BARS-CTR: An Open Benchmark for CTR Prediction https://openbenchmark.github.io/BARS/CTR ## Model List @@ -47,4 +43,5 @@ The following models have been benchmarked with open-source code and detailed re | 33 | CIKM'21 | [EDCN](./fuxictr/pytorch/models/EDCN.py) | [Enhancing Explicit and Implicit Feature Interactions via Information Sharing for Parallel Deep CTR Models](https://dlp-kdd.github.io/assets/pdf/DLP-KDD_2021_paper_12.pdf) :triangular_flag_on_post:**Huawei** | [:arrow_upper_right:](https://github.com/openbenchmark/BARS/tree/master/ctr_prediction/benchmarks/EDCN) | | 34 | DLP-KDD'21 | [MaskNet](./fuxictr/pytorch/models/MaskNet.py) | [MaskNet: Introducing Feature-Wise Multiplication to CTR Ranking Models by Instance-Guided Mask](https://arxiv.org/abs/2102.07619) :triangular_flag_on_post:**Sina Weibo** | [:arrow_upper_right:](https://github.com/openbenchmark/BARS/tree/master/ctr_prediction/benchmarks/MaskNet) | | 35 | SIGIR'21 | [SAM](./fuxictr/pytorch/models/SAM.py) | [Looking at CTR Prediction Again: Is Attention All You Need?](https://arxiv.org/abs/2105.05563) :triangular_flag_on_post:**BOSS Zhipin** | [:arrow_upper_right:](https://github.com/openbenchmark/BARS/tree/master/ctr_prediction/benchmarks/SAM) | -| 36 | KDD'21 | [AOANet](./fuxictr/pytorch/models/AOANet.py) | [Architecture and Operation Adaptive Network for Online Recommendations](https://dl.acm.org/doi/10.1145/3447548.3467133) :triangular_flag_on_post:**Didi Chuxing** | [:arrow_upper_right:](https://github.com/openbenchmark/BARS/tree/master/ctr_prediction/benchmarks/AOANet) | \ No newline at end of file +| 36 | KDD'21 | [AOANet](./fuxictr/pytorch/models/AOANet.py) | [Architecture and Operation Adaptive Network for Online Recommendations](https://dl.acm.org/doi/10.1145/3447548.3467133) :triangular_flag_on_post:**Didi Chuxing** | [:arrow_upper_right:](https://github.com/openbenchmark/BARS/tree/master/ctr_prediction/benchmarks/AOANet) | + diff --git a/ctr_prediction/benchmarks/SAM/SAM_avazu_x1/README.md b/ranking/ctr/SAM/SAM_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_avazu_x1/README.md rename to ranking/ctr/SAM/SAM_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/SAM/SAM_avazu_x1/SAM_avazu_x1_012_1cecba8c.log b/ranking/ctr/SAM/SAM_avazu_x1/SAM_avazu_x1_012_1cecba8c.log similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_avazu_x1/SAM_avazu_x1_012_1cecba8c.log rename to ranking/ctr/SAM/SAM_avazu_x1/SAM_avazu_x1_012_1cecba8c.log diff --git a/ctr_prediction/benchmarks/SAM/SAM_avazu_x1/SAM_avazu_x1_tuner_config_01.csv b/ranking/ctr/SAM/SAM_avazu_x1/SAM_avazu_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_avazu_x1/SAM_avazu_x1_tuner_config_01.csv rename to ranking/ctr/SAM/SAM_avazu_x1/SAM_avazu_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/SAM/SAM_avazu_x1/SAM_avazu_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/SAM/SAM_avazu_x1/SAM_avazu_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_avazu_x1/SAM_avazu_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/SAM/SAM_avazu_x1/SAM_avazu_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/SAM/SAM_avazu_x1/SAM_avazu_x1_tuner_config_01/model_config.yaml b/ranking/ctr/SAM/SAM_avazu_x1/SAM_avazu_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_avazu_x1/SAM_avazu_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/SAM/SAM_avazu_x1/SAM_avazu_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/SAM/SAM_avazu_x1/environments.txt b/ranking/ctr/SAM/SAM_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_avazu_x1/environments.txt rename to ranking/ctr/SAM/SAM_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/SAM/SAM_avazu_x1/results.csv b/ranking/ctr/SAM/SAM_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_avazu_x1/results.csv rename to ranking/ctr/SAM/SAM_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/SAM/SAM_avazu_x1/run_expid.py b/ranking/ctr/SAM/SAM_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_avazu_x1/run_expid.py rename to ranking/ctr/SAM/SAM_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/SAM/SAM_criteo_x1/README.md b/ranking/ctr/SAM/SAM_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_criteo_x1/README.md rename to ranking/ctr/SAM/SAM_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/SAM/SAM_criteo_x1/SAM_criteo_x1_012_55e25f89.log b/ranking/ctr/SAM/SAM_criteo_x1/SAM_criteo_x1_012_55e25f89.log similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_criteo_x1/SAM_criteo_x1_012_55e25f89.log rename to ranking/ctr/SAM/SAM_criteo_x1/SAM_criteo_x1_012_55e25f89.log diff --git a/ctr_prediction/benchmarks/SAM/SAM_criteo_x1/SAM_criteo_x1_tuner_config_03.csv b/ranking/ctr/SAM/SAM_criteo_x1/SAM_criteo_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_criteo_x1/SAM_criteo_x1_tuner_config_03.csv rename to ranking/ctr/SAM/SAM_criteo_x1/SAM_criteo_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/SAM/SAM_criteo_x1/SAM_criteo_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/SAM/SAM_criteo_x1/SAM_criteo_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_criteo_x1/SAM_criteo_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/SAM/SAM_criteo_x1/SAM_criteo_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/SAM/SAM_criteo_x1/SAM_criteo_x1_tuner_config_03/model_config.yaml b/ranking/ctr/SAM/SAM_criteo_x1/SAM_criteo_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_criteo_x1/SAM_criteo_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/SAM/SAM_criteo_x1/SAM_criteo_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/SAM/SAM_criteo_x1/environments.txt b/ranking/ctr/SAM/SAM_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_criteo_x1/environments.txt rename to ranking/ctr/SAM/SAM_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/SAM/SAM_criteo_x1/results.csv b/ranking/ctr/SAM/SAM_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_criteo_x1/results.csv rename to ranking/ctr/SAM/SAM_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/SAM/SAM_criteo_x1/run_expid.py b/ranking/ctr/SAM/SAM_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_criteo_x1/run_expid.py rename to ranking/ctr/SAM/SAM_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/SAM/SAM_frappe_x1/README.md b/ranking/ctr/SAM/SAM_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_frappe_x1/README.md rename to ranking/ctr/SAM/SAM_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/SAM/SAM_frappe_x1/SAM_frappe_x1_005_1368acce.log b/ranking/ctr/SAM/SAM_frappe_x1/SAM_frappe_x1_005_1368acce.log similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_frappe_x1/SAM_frappe_x1_005_1368acce.log rename to ranking/ctr/SAM/SAM_frappe_x1/SAM_frappe_x1_005_1368acce.log diff --git a/ctr_prediction/benchmarks/SAM/SAM_frappe_x1/SAM_frappe_x1_tuner_config_02.csv b/ranking/ctr/SAM/SAM_frappe_x1/SAM_frappe_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_frappe_x1/SAM_frappe_x1_tuner_config_02.csv rename to ranking/ctr/SAM/SAM_frappe_x1/SAM_frappe_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/SAM/SAM_frappe_x1/SAM_frappe_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/SAM/SAM_frappe_x1/SAM_frappe_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_frappe_x1/SAM_frappe_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/SAM/SAM_frappe_x1/SAM_frappe_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/SAM/SAM_frappe_x1/SAM_frappe_x1_tuner_config_02/model_config.yaml b/ranking/ctr/SAM/SAM_frappe_x1/SAM_frappe_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_frappe_x1/SAM_frappe_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/SAM/SAM_frappe_x1/SAM_frappe_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/SAM/SAM_frappe_x1/environments.txt b/ranking/ctr/SAM/SAM_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_frappe_x1/environments.txt rename to ranking/ctr/SAM/SAM_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/SAM/SAM_frappe_x1/results.csv b/ranking/ctr/SAM/SAM_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_frappe_x1/results.csv rename to ranking/ctr/SAM/SAM_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/SAM/SAM_frappe_x1/run_expid.py b/ranking/ctr/SAM/SAM_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_frappe_x1/run_expid.py rename to ranking/ctr/SAM/SAM_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/SAM/SAM_movielenslatest_x1/README.md b/ranking/ctr/SAM/SAM_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_movielenslatest_x1/README.md rename to ranking/ctr/SAM/SAM_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/SAM/SAM_movielenslatest_x1/SAM_movielenslatest_x1_013_68a6bc8b.log b/ranking/ctr/SAM/SAM_movielenslatest_x1/SAM_movielenslatest_x1_013_68a6bc8b.log similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_movielenslatest_x1/SAM_movielenslatest_x1_013_68a6bc8b.log rename to ranking/ctr/SAM/SAM_movielenslatest_x1/SAM_movielenslatest_x1_013_68a6bc8b.log diff --git a/ctr_prediction/benchmarks/SAM/SAM_movielenslatest_x1/SAM_movielenslatest_x1_tuner_config_06.csv b/ranking/ctr/SAM/SAM_movielenslatest_x1/SAM_movielenslatest_x1_tuner_config_06.csv similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_movielenslatest_x1/SAM_movielenslatest_x1_tuner_config_06.csv rename to ranking/ctr/SAM/SAM_movielenslatest_x1/SAM_movielenslatest_x1_tuner_config_06.csv diff --git a/ctr_prediction/benchmarks/SAM/SAM_movielenslatest_x1/SAM_movielenslatest_x1_tuner_config_06/dataset_config.yaml b/ranking/ctr/SAM/SAM_movielenslatest_x1/SAM_movielenslatest_x1_tuner_config_06/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_movielenslatest_x1/SAM_movielenslatest_x1_tuner_config_06/dataset_config.yaml rename to ranking/ctr/SAM/SAM_movielenslatest_x1/SAM_movielenslatest_x1_tuner_config_06/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/SAM/SAM_movielenslatest_x1/SAM_movielenslatest_x1_tuner_config_06/model_config.yaml b/ranking/ctr/SAM/SAM_movielenslatest_x1/SAM_movielenslatest_x1_tuner_config_06/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_movielenslatest_x1/SAM_movielenslatest_x1_tuner_config_06/model_config.yaml rename to ranking/ctr/SAM/SAM_movielenslatest_x1/SAM_movielenslatest_x1_tuner_config_06/model_config.yaml diff --git a/ctr_prediction/benchmarks/SAM/SAM_movielenslatest_x1/environments.txt b/ranking/ctr/SAM/SAM_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_movielenslatest_x1/environments.txt rename to ranking/ctr/SAM/SAM_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/SAM/SAM_movielenslatest_x1/results.csv b/ranking/ctr/SAM/SAM_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_movielenslatest_x1/results.csv rename to ranking/ctr/SAM/SAM_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/SAM/SAM_movielenslatest_x1/run_expid.py b/ranking/ctr/SAM/SAM_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/SAM/SAM_movielenslatest_x1/run_expid.py rename to ranking/ctr/SAM/SAM_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x1/README.md b/ranking/ctr/WideDeep/WideDeep_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x1/README.md rename to ranking/ctr/WideDeep/WideDeep_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x1/WideDeep_avazu_x1_002_d8c146ac.log b/ranking/ctr/WideDeep/WideDeep_avazu_x1/WideDeep_avazu_x1_002_d8c146ac.log similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x1/WideDeep_avazu_x1_002_d8c146ac.log rename to ranking/ctr/WideDeep/WideDeep_avazu_x1/WideDeep_avazu_x1_002_d8c146ac.log diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x1/WideDeep_avazu_x1_tuner_config_03.csv b/ranking/ctr/WideDeep/WideDeep_avazu_x1/WideDeep_avazu_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x1/WideDeep_avazu_x1_tuner_config_03.csv rename to ranking/ctr/WideDeep/WideDeep_avazu_x1/WideDeep_avazu_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x1/WideDeep_avazu_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/WideDeep/WideDeep_avazu_x1/WideDeep_avazu_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x1/WideDeep_avazu_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/WideDeep/WideDeep_avazu_x1/WideDeep_avazu_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x1/WideDeep_avazu_x1_tuner_config_03/model_config.yaml b/ranking/ctr/WideDeep/WideDeep_avazu_x1/WideDeep_avazu_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x1/WideDeep_avazu_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/WideDeep/WideDeep_avazu_x1/WideDeep_avazu_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x1/environments.txt b/ranking/ctr/WideDeep/WideDeep_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x1/environments.txt rename to ranking/ctr/WideDeep/WideDeep_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x1/results.csv b/ranking/ctr/WideDeep/WideDeep_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x1/results.csv rename to ranking/ctr/WideDeep/WideDeep_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x1/run_expid.py b/ranking/ctr/WideDeep/WideDeep_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x1/run_expid.py rename to ranking/ctr/WideDeep/WideDeep_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_001/README.md b/ranking/ctr/WideDeep/WideDeep_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_001/README.md rename to ranking/ctr/WideDeep/WideDeep_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_001/WideDeep_avazu_x4_016_233585a0.log b/ranking/ctr/WideDeep/WideDeep_avazu_x4_001/WideDeep_avazu_x4_016_233585a0.log similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_001/WideDeep_avazu_x4_016_233585a0.log rename to ranking/ctr/WideDeep/WideDeep_avazu_x4_001/WideDeep_avazu_x4_016_233585a0.log diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_001/WideDeep_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/WideDeep/WideDeep_avazu_x4_001/WideDeep_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_001/WideDeep_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/WideDeep/WideDeep_avazu_x4_001/WideDeep_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_001/WideDeep_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/WideDeep/WideDeep_avazu_x4_001/WideDeep_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_001/WideDeep_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/WideDeep/WideDeep_avazu_x4_001/WideDeep_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_001/environments.txt b/ranking/ctr/WideDeep/WideDeep_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_001/environments.txt rename to ranking/ctr/WideDeep/WideDeep_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_001/results.csv b/ranking/ctr/WideDeep/WideDeep_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_001/results.csv rename to ranking/ctr/WideDeep/WideDeep_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_001/run_expid.py b/ranking/ctr/WideDeep/WideDeep_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_001/run_expid.py rename to ranking/ctr/WideDeep/WideDeep_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_002/README.md b/ranking/ctr/WideDeep/WideDeep_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_002/README.md rename to ranking/ctr/WideDeep/WideDeep_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_002/WideDeep_avazu_x4_043_23ffe850.log b/ranking/ctr/WideDeep/WideDeep_avazu_x4_002/WideDeep_avazu_x4_043_23ffe850.log similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_002/WideDeep_avazu_x4_043_23ffe850.log rename to ranking/ctr/WideDeep/WideDeep_avazu_x4_002/WideDeep_avazu_x4_043_23ffe850.log diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_002/WideDeep_avazu_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/WideDeep/WideDeep_avazu_x4_002/WideDeep_avazu_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_002/WideDeep_avazu_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/WideDeep/WideDeep_avazu_x4_002/WideDeep_avazu_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_002/WideDeep_avazu_x4_tuner_config_01/model_config.yaml b/ranking/ctr/WideDeep/WideDeep_avazu_x4_002/WideDeep_avazu_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_002/WideDeep_avazu_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/WideDeep/WideDeep_avazu_x4_002/WideDeep_avazu_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_002/environments.txt b/ranking/ctr/WideDeep/WideDeep_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_002/environments.txt rename to ranking/ctr/WideDeep/WideDeep_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_002/results.csv b/ranking/ctr/WideDeep/WideDeep_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_002/results.csv rename to ranking/ctr/WideDeep/WideDeep_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_002/run_expid.py b/ranking/ctr/WideDeep/WideDeep_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_avazu_x4_002/run_expid.py rename to ranking/ctr/WideDeep/WideDeep_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x1/README.md b/ranking/ctr/WideDeep/WideDeep_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x1/README.md rename to ranking/ctr/WideDeep/WideDeep_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x1/WideDeep_criteo_x1_010_3a06c5bc.log b/ranking/ctr/WideDeep/WideDeep_criteo_x1/WideDeep_criteo_x1_010_3a06c5bc.log similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x1/WideDeep_criteo_x1_010_3a06c5bc.log rename to ranking/ctr/WideDeep/WideDeep_criteo_x1/WideDeep_criteo_x1_010_3a06c5bc.log diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x1/WideDeep_criteo_x1_tuner_config_01.csv b/ranking/ctr/WideDeep/WideDeep_criteo_x1/WideDeep_criteo_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x1/WideDeep_criteo_x1_tuner_config_01.csv rename to ranking/ctr/WideDeep/WideDeep_criteo_x1/WideDeep_criteo_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x1/WideDeep_criteo_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/WideDeep/WideDeep_criteo_x1/WideDeep_criteo_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x1/WideDeep_criteo_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/WideDeep/WideDeep_criteo_x1/WideDeep_criteo_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x1/WideDeep_criteo_x1_tuner_config_01/model_config.yaml b/ranking/ctr/WideDeep/WideDeep_criteo_x1/WideDeep_criteo_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x1/WideDeep_criteo_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/WideDeep/WideDeep_criteo_x1/WideDeep_criteo_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x1/environments.txt b/ranking/ctr/WideDeep/WideDeep_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x1/environments.txt rename to ranking/ctr/WideDeep/WideDeep_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x1/results.csv b/ranking/ctr/WideDeep/WideDeep_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x1/results.csv rename to ranking/ctr/WideDeep/WideDeep_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x1/run_expid.py b/ranking/ctr/WideDeep/WideDeep_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x1/run_expid.py rename to ranking/ctr/WideDeep/WideDeep_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_001/README.md b/ranking/ctr/WideDeep/WideDeep_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_001/README.md rename to ranking/ctr/WideDeep/WideDeep_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_001/WideDeep_criteo_x4_024_ba0036f7.log b/ranking/ctr/WideDeep/WideDeep_criteo_x4_001/WideDeep_criteo_x4_024_ba0036f7.log similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_001/WideDeep_criteo_x4_024_ba0036f7.log rename to ranking/ctr/WideDeep/WideDeep_criteo_x4_001/WideDeep_criteo_x4_024_ba0036f7.log diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_001/WideDeep_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/WideDeep/WideDeep_criteo_x4_001/WideDeep_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_001/WideDeep_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/WideDeep/WideDeep_criteo_x4_001/WideDeep_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_001/WideDeep_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/WideDeep/WideDeep_criteo_x4_001/WideDeep_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_001/WideDeep_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/WideDeep/WideDeep_criteo_x4_001/WideDeep_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_001/environments.txt b/ranking/ctr/WideDeep/WideDeep_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_001/environments.txt rename to ranking/ctr/WideDeep/WideDeep_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_001/results.csv b/ranking/ctr/WideDeep/WideDeep_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_001/results.csv rename to ranking/ctr/WideDeep/WideDeep_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_001/run_expid.py b/ranking/ctr/WideDeep/WideDeep_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_001/run_expid.py rename to ranking/ctr/WideDeep/WideDeep_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_002/README.md b/ranking/ctr/WideDeep/WideDeep_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_002/README.md rename to ranking/ctr/WideDeep/WideDeep_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_002/WideDeep_criteo_x4_007_dbcfd11c.log b/ranking/ctr/WideDeep/WideDeep_criteo_x4_002/WideDeep_criteo_x4_007_dbcfd11c.log similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_002/WideDeep_criteo_x4_007_dbcfd11c.log rename to ranking/ctr/WideDeep/WideDeep_criteo_x4_002/WideDeep_criteo_x4_007_dbcfd11c.log diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_002/WideDeep_criteo_x4_tuner_config_02/dataset_config.yaml b/ranking/ctr/WideDeep/WideDeep_criteo_x4_002/WideDeep_criteo_x4_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_002/WideDeep_criteo_x4_tuner_config_02/dataset_config.yaml rename to ranking/ctr/WideDeep/WideDeep_criteo_x4_002/WideDeep_criteo_x4_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_002/WideDeep_criteo_x4_tuner_config_02/model_config.yaml b/ranking/ctr/WideDeep/WideDeep_criteo_x4_002/WideDeep_criteo_x4_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_002/WideDeep_criteo_x4_tuner_config_02/model_config.yaml rename to ranking/ctr/WideDeep/WideDeep_criteo_x4_002/WideDeep_criteo_x4_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_002/environments.txt b/ranking/ctr/WideDeep/WideDeep_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_002/environments.txt rename to ranking/ctr/WideDeep/WideDeep_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_002/results.csv b/ranking/ctr/WideDeep/WideDeep_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_002/results.csv rename to ranking/ctr/WideDeep/WideDeep_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_002/run_expid.py b/ranking/ctr/WideDeep/WideDeep_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_criteo_x4_002/run_expid.py rename to ranking/ctr/WideDeep/WideDeep_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_frappe_x1/README.md b/ranking/ctr/WideDeep/WideDeep_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_frappe_x1/README.md rename to ranking/ctr/WideDeep/WideDeep_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_frappe_x1/WideDeep_frappe_x1_030_af559975.log b/ranking/ctr/WideDeep/WideDeep_frappe_x1/WideDeep_frappe_x1_030_af559975.log similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_frappe_x1/WideDeep_frappe_x1_030_af559975.log rename to ranking/ctr/WideDeep/WideDeep_frappe_x1/WideDeep_frappe_x1_030_af559975.log diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_frappe_x1/WideDeep_frappe_x1_tuner_config_01.csv b/ranking/ctr/WideDeep/WideDeep_frappe_x1/WideDeep_frappe_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_frappe_x1/WideDeep_frappe_x1_tuner_config_01.csv rename to ranking/ctr/WideDeep/WideDeep_frappe_x1/WideDeep_frappe_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_frappe_x1/WideDeep_frappe_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/WideDeep/WideDeep_frappe_x1/WideDeep_frappe_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_frappe_x1/WideDeep_frappe_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/WideDeep/WideDeep_frappe_x1/WideDeep_frappe_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_frappe_x1/WideDeep_frappe_x1_tuner_config_01/model_config.yaml b/ranking/ctr/WideDeep/WideDeep_frappe_x1/WideDeep_frappe_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_frappe_x1/WideDeep_frappe_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/WideDeep/WideDeep_frappe_x1/WideDeep_frappe_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_frappe_x1/environments.txt b/ranking/ctr/WideDeep/WideDeep_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_frappe_x1/environments.txt rename to ranking/ctr/WideDeep/WideDeep_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_frappe_x1/results.csv b/ranking/ctr/WideDeep/WideDeep_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_frappe_x1/results.csv rename to ranking/ctr/WideDeep/WideDeep_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_frappe_x1/run_expid.py b/ranking/ctr/WideDeep/WideDeep_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_frappe_x1/run_expid.py rename to ranking/ctr/WideDeep/WideDeep_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_kkbox_x1/README.md b/ranking/ctr/WideDeep/WideDeep_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_kkbox_x1/README.md rename to ranking/ctr/WideDeep/WideDeep_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_kkbox_x1/WideDeep_kkbox_x1_012_62151ded.log b/ranking/ctr/WideDeep/WideDeep_kkbox_x1/WideDeep_kkbox_x1_012_62151ded.log similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_kkbox_x1/WideDeep_kkbox_x1_012_62151ded.log rename to ranking/ctr/WideDeep/WideDeep_kkbox_x1/WideDeep_kkbox_x1_012_62151ded.log diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_kkbox_x1/WideDeep_kkbox_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/WideDeep/WideDeep_kkbox_x1/WideDeep_kkbox_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_kkbox_x1/WideDeep_kkbox_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/WideDeep/WideDeep_kkbox_x1/WideDeep_kkbox_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_kkbox_x1/WideDeep_kkbox_x1_tuner_config_03/model_config.yaml b/ranking/ctr/WideDeep/WideDeep_kkbox_x1/WideDeep_kkbox_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_kkbox_x1/WideDeep_kkbox_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/WideDeep/WideDeep_kkbox_x1/WideDeep_kkbox_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_kkbox_x1/environments.txt b/ranking/ctr/WideDeep/WideDeep_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_kkbox_x1/environments.txt rename to ranking/ctr/WideDeep/WideDeep_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_kkbox_x1/results.csv b/ranking/ctr/WideDeep/WideDeep_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_kkbox_x1/results.csv rename to ranking/ctr/WideDeep/WideDeep_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_kkbox_x1/run_expid.py b/ranking/ctr/WideDeep/WideDeep_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_kkbox_x1/run_expid.py rename to ranking/ctr/WideDeep/WideDeep_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_movielenslatest_x1/README.md b/ranking/ctr/WideDeep/WideDeep_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_movielenslatest_x1/README.md rename to ranking/ctr/WideDeep/WideDeep_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_movielenslatest_x1/WideDeep_movielenslatest_x1_007_1b65cd45.log b/ranking/ctr/WideDeep/WideDeep_movielenslatest_x1/WideDeep_movielenslatest_x1_007_1b65cd45.log similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_movielenslatest_x1/WideDeep_movielenslatest_x1_007_1b65cd45.log rename to ranking/ctr/WideDeep/WideDeep_movielenslatest_x1/WideDeep_movielenslatest_x1_007_1b65cd45.log diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_movielenslatest_x1/WideDeep_movielenslatest_x1_tuner_config_01.csv b/ranking/ctr/WideDeep/WideDeep_movielenslatest_x1/WideDeep_movielenslatest_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_movielenslatest_x1/WideDeep_movielenslatest_x1_tuner_config_01.csv rename to ranking/ctr/WideDeep/WideDeep_movielenslatest_x1/WideDeep_movielenslatest_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_movielenslatest_x1/WideDeep_movielenslatest_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/WideDeep/WideDeep_movielenslatest_x1/WideDeep_movielenslatest_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_movielenslatest_x1/WideDeep_movielenslatest_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/WideDeep/WideDeep_movielenslatest_x1/WideDeep_movielenslatest_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_movielenslatest_x1/WideDeep_movielenslatest_x1_tuner_config_01/model_config.yaml b/ranking/ctr/WideDeep/WideDeep_movielenslatest_x1/WideDeep_movielenslatest_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_movielenslatest_x1/WideDeep_movielenslatest_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/WideDeep/WideDeep_movielenslatest_x1/WideDeep_movielenslatest_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_movielenslatest_x1/environments.txt b/ranking/ctr/WideDeep/WideDeep_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_movielenslatest_x1/environments.txt rename to ranking/ctr/WideDeep/WideDeep_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_movielenslatest_x1/results.csv b/ranking/ctr/WideDeep/WideDeep_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_movielenslatest_x1/results.csv rename to ranking/ctr/WideDeep/WideDeep_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/WideDeep/WideDeep_movielenslatest_x1/run_expid.py b/ranking/ctr/WideDeep/WideDeep_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/WideDeep/WideDeep_movielenslatest_x1/run_expid.py rename to ranking/ctr/WideDeep/WideDeep_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x1/CIN_avazu_x1_tuner_config_01.csv b/ranking/ctr/xDeepFM/CIN_avazu_x1/CIN_avazu_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x1/CIN_avazu_x1_tuner_config_01.csv rename to ranking/ctr/xDeepFM/CIN_avazu_x1/CIN_avazu_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x1/CIN_avazu_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/xDeepFM/CIN_avazu_x1/CIN_avazu_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x1/CIN_avazu_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/xDeepFM/CIN_avazu_x1/CIN_avazu_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x1/CIN_avazu_x1_tuner_config_01/model_config.yaml b/ranking/ctr/xDeepFM/CIN_avazu_x1/CIN_avazu_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x1/CIN_avazu_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/xDeepFM/CIN_avazu_x1/CIN_avazu_x1_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x1/README.md b/ranking/ctr/xDeepFM/CIN_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x1/README.md rename to ranking/ctr/xDeepFM/CIN_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x1/environments.txt b/ranking/ctr/xDeepFM/CIN_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x1/environments.txt rename to ranking/ctr/xDeepFM/CIN_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x1/results.csv b/ranking/ctr/xDeepFM/CIN_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x1/results.csv rename to ranking/ctr/xDeepFM/CIN_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x1/run_expid.py b/ranking/ctr/xDeepFM/CIN_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x1/run_expid.py rename to ranking/ctr/xDeepFM/CIN_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x1/xDeepFM_avazu_x1_003_7f3fed32.log b/ranking/ctr/xDeepFM/CIN_avazu_x1/xDeepFM_avazu_x1_003_7f3fed32.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x1/xDeepFM_avazu_x1_003_7f3fed32.log rename to ranking/ctr/xDeepFM/CIN_avazu_x1/xDeepFM_avazu_x1_003_7f3fed32.log diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_001/CIN_avazu_x4_tuner_config_03/dataset_config.yaml b/ranking/ctr/xDeepFM/CIN_avazu_x4_001/CIN_avazu_x4_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_001/CIN_avazu_x4_tuner_config_03/dataset_config.yaml rename to ranking/ctr/xDeepFM/CIN_avazu_x4_001/CIN_avazu_x4_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_001/CIN_avazu_x4_tuner_config_03/model_config.yaml b/ranking/ctr/xDeepFM/CIN_avazu_x4_001/CIN_avazu_x4_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_001/CIN_avazu_x4_tuner_config_03/model_config.yaml rename to ranking/ctr/xDeepFM/CIN_avazu_x4_001/CIN_avazu_x4_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_001/README.md b/ranking/ctr/xDeepFM/CIN_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_001/README.md rename to ranking/ctr/xDeepFM/CIN_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_001/environments.txt b/ranking/ctr/xDeepFM/CIN_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_001/environments.txt rename to ranking/ctr/xDeepFM/CIN_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_001/results.csv b/ranking/ctr/xDeepFM/CIN_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_001/results.csv rename to ranking/ctr/xDeepFM/CIN_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_001/run_expid.py b/ranking/ctr/xDeepFM/CIN_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_001/run_expid.py rename to ranking/ctr/xDeepFM/CIN_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_001/xDeepFM_avazu_x4_002_9d81123c.log b/ranking/ctr/xDeepFM/CIN_avazu_x4_001/xDeepFM_avazu_x4_002_9d81123c.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_001/xDeepFM_avazu_x4_002_9d81123c.log rename to ranking/ctr/xDeepFM/CIN_avazu_x4_001/xDeepFM_avazu_x4_002_9d81123c.log diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_002/CIN_avazu_x4_tuner_config_04/dataset_config.yaml b/ranking/ctr/xDeepFM/CIN_avazu_x4_002/CIN_avazu_x4_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_002/CIN_avazu_x4_tuner_config_04/dataset_config.yaml rename to ranking/ctr/xDeepFM/CIN_avazu_x4_002/CIN_avazu_x4_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_002/CIN_avazu_x4_tuner_config_04/model_config.yaml b/ranking/ctr/xDeepFM/CIN_avazu_x4_002/CIN_avazu_x4_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_002/CIN_avazu_x4_tuner_config_04/model_config.yaml rename to ranking/ctr/xDeepFM/CIN_avazu_x4_002/CIN_avazu_x4_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_002/README.md b/ranking/ctr/xDeepFM/CIN_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_002/README.md rename to ranking/ctr/xDeepFM/CIN_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_002/environments.txt b/ranking/ctr/xDeepFM/CIN_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_002/environments.txt rename to ranking/ctr/xDeepFM/CIN_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_002/results.csv b/ranking/ctr/xDeepFM/CIN_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_002/results.csv rename to ranking/ctr/xDeepFM/CIN_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_002/run_expid.py b/ranking/ctr/xDeepFM/CIN_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_002/run_expid.py rename to ranking/ctr/xDeepFM/CIN_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_002/xDeepFM_avazu_x4_009_ce557b77.log b/ranking/ctr/xDeepFM/CIN_avazu_x4_002/xDeepFM_avazu_x4_009_ce557b77.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_avazu_x4_002/xDeepFM_avazu_x4_009_ce557b77.log rename to ranking/ctr/xDeepFM/CIN_avazu_x4_002/xDeepFM_avazu_x4_009_ce557b77.log diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x1/CIN_criteo_x1_tuner_config_02.csv b/ranking/ctr/xDeepFM/CIN_criteo_x1/CIN_criteo_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x1/CIN_criteo_x1_tuner_config_02.csv rename to ranking/ctr/xDeepFM/CIN_criteo_x1/CIN_criteo_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x1/CIN_criteo_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/xDeepFM/CIN_criteo_x1/CIN_criteo_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x1/CIN_criteo_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/xDeepFM/CIN_criteo_x1/CIN_criteo_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x1/CIN_criteo_x1_tuner_config_02/model_config.yaml b/ranking/ctr/xDeepFM/CIN_criteo_x1/CIN_criteo_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x1/CIN_criteo_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/xDeepFM/CIN_criteo_x1/CIN_criteo_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x1/README.md b/ranking/ctr/xDeepFM/CIN_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x1/README.md rename to ranking/ctr/xDeepFM/CIN_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x1/environments.txt b/ranking/ctr/xDeepFM/CIN_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x1/environments.txt rename to ranking/ctr/xDeepFM/CIN_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x1/results.csv b/ranking/ctr/xDeepFM/CIN_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x1/results.csv rename to ranking/ctr/xDeepFM/CIN_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x1/run_expid.py b/ranking/ctr/xDeepFM/CIN_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x1/run_expid.py rename to ranking/ctr/xDeepFM/CIN_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x1/xDeepFM_criteo_x1_004_6b5adc06.log b/ranking/ctr/xDeepFM/CIN_criteo_x1/xDeepFM_criteo_x1_004_6b5adc06.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x1/xDeepFM_criteo_x1_004_6b5adc06.log rename to ranking/ctr/xDeepFM/CIN_criteo_x1/xDeepFM_criteo_x1_004_6b5adc06.log diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_001/CIN_criteo_x4_tuner_config_01/dataset_config.yaml b/ranking/ctr/xDeepFM/CIN_criteo_x4_001/CIN_criteo_x4_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_001/CIN_criteo_x4_tuner_config_01/dataset_config.yaml rename to ranking/ctr/xDeepFM/CIN_criteo_x4_001/CIN_criteo_x4_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_001/CIN_criteo_x4_tuner_config_01/model_config.yaml b/ranking/ctr/xDeepFM/CIN_criteo_x4_001/CIN_criteo_x4_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_001/CIN_criteo_x4_tuner_config_01/model_config.yaml rename to ranking/ctr/xDeepFM/CIN_criteo_x4_001/CIN_criteo_x4_tuner_config_01/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_001/README.md b/ranking/ctr/xDeepFM/CIN_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_001/README.md rename to ranking/ctr/xDeepFM/CIN_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_001/environments.txt b/ranking/ctr/xDeepFM/CIN_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_001/environments.txt rename to ranking/ctr/xDeepFM/CIN_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_001/results.csv b/ranking/ctr/xDeepFM/CIN_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_001/results.csv rename to ranking/ctr/xDeepFM/CIN_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_001/run_expid.py b/ranking/ctr/xDeepFM/CIN_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_001/run_expid.py rename to ranking/ctr/xDeepFM/CIN_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_001/xDeepFM_criteo_x4_007_f93ff06e.log b/ranking/ctr/xDeepFM/CIN_criteo_x4_001/xDeepFM_criteo_x4_007_f93ff06e.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_001/xDeepFM_criteo_x4_007_f93ff06e.log rename to ranking/ctr/xDeepFM/CIN_criteo_x4_001/xDeepFM_criteo_x4_007_f93ff06e.log diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_002/CIN_criteo_x4_tuner_config_11/dataset_config.yaml b/ranking/ctr/xDeepFM/CIN_criteo_x4_002/CIN_criteo_x4_tuner_config_11/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_002/CIN_criteo_x4_tuner_config_11/dataset_config.yaml rename to ranking/ctr/xDeepFM/CIN_criteo_x4_002/CIN_criteo_x4_tuner_config_11/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_002/CIN_criteo_x4_tuner_config_11/model_config.yaml b/ranking/ctr/xDeepFM/CIN_criteo_x4_002/CIN_criteo_x4_tuner_config_11/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_002/CIN_criteo_x4_tuner_config_11/model_config.yaml rename to ranking/ctr/xDeepFM/CIN_criteo_x4_002/CIN_criteo_x4_tuner_config_11/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_002/README.md b/ranking/ctr/xDeepFM/CIN_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_002/README.md rename to ranking/ctr/xDeepFM/CIN_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_002/environments.txt b/ranking/ctr/xDeepFM/CIN_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_002/environments.txt rename to ranking/ctr/xDeepFM/CIN_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_002/results.csv b/ranking/ctr/xDeepFM/CIN_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_002/results.csv rename to ranking/ctr/xDeepFM/CIN_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_002/run_expid.py b/ranking/ctr/xDeepFM/CIN_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_002/run_expid.py rename to ranking/ctr/xDeepFM/CIN_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_002/xDeepFM_criteo_x4_003_444046bd.log b/ranking/ctr/xDeepFM/CIN_criteo_x4_002/xDeepFM_criteo_x4_003_444046bd.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_criteo_x4_002/xDeepFM_criteo_x4_003_444046bd.log rename to ranking/ctr/xDeepFM/CIN_criteo_x4_002/xDeepFM_criteo_x4_003_444046bd.log diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_frappe_x1/CIN_frappe_x1_tuner_config_03.csv b/ranking/ctr/xDeepFM/CIN_frappe_x1/CIN_frappe_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_frappe_x1/CIN_frappe_x1_tuner_config_03.csv rename to ranking/ctr/xDeepFM/CIN_frappe_x1/CIN_frappe_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_frappe_x1/CIN_frappe_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/xDeepFM/CIN_frappe_x1/CIN_frappe_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_frappe_x1/CIN_frappe_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/xDeepFM/CIN_frappe_x1/CIN_frappe_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_frappe_x1/CIN_frappe_x1_tuner_config_03/model_config.yaml b/ranking/ctr/xDeepFM/CIN_frappe_x1/CIN_frappe_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_frappe_x1/CIN_frappe_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/xDeepFM/CIN_frappe_x1/CIN_frappe_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_frappe_x1/README.md b/ranking/ctr/xDeepFM/CIN_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_frappe_x1/README.md rename to ranking/ctr/xDeepFM/CIN_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_frappe_x1/environments.txt b/ranking/ctr/xDeepFM/CIN_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_frappe_x1/environments.txt rename to ranking/ctr/xDeepFM/CIN_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_frappe_x1/results.csv b/ranking/ctr/xDeepFM/CIN_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_frappe_x1/results.csv rename to ranking/ctr/xDeepFM/CIN_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_frappe_x1/run_expid.py b/ranking/ctr/xDeepFM/CIN_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_frappe_x1/run_expid.py rename to ranking/ctr/xDeepFM/CIN_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_frappe_x1/xDeepFM_frappe_x1_008_9e8759c4.log b/ranking/ctr/xDeepFM/CIN_frappe_x1/xDeepFM_frappe_x1_008_9e8759c4.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_frappe_x1/xDeepFM_frappe_x1_008_9e8759c4.log rename to ranking/ctr/xDeepFM/CIN_frappe_x1/xDeepFM_frappe_x1_008_9e8759c4.log diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_kkbox_x1/CIN_kkbox_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/xDeepFM/CIN_kkbox_x1/CIN_kkbox_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_kkbox_x1/CIN_kkbox_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/xDeepFM/CIN_kkbox_x1/CIN_kkbox_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_kkbox_x1/CIN_kkbox_x1_tuner_config_02/model_config.yaml b/ranking/ctr/xDeepFM/CIN_kkbox_x1/CIN_kkbox_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_kkbox_x1/CIN_kkbox_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/xDeepFM/CIN_kkbox_x1/CIN_kkbox_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_kkbox_x1/README.md b/ranking/ctr/xDeepFM/CIN_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_kkbox_x1/README.md rename to ranking/ctr/xDeepFM/CIN_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_kkbox_x1/environments.txt b/ranking/ctr/xDeepFM/CIN_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_kkbox_x1/environments.txt rename to ranking/ctr/xDeepFM/CIN_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_kkbox_x1/results.csv b/ranking/ctr/xDeepFM/CIN_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_kkbox_x1/results.csv rename to ranking/ctr/xDeepFM/CIN_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_kkbox_x1/run_expid.py b/ranking/ctr/xDeepFM/CIN_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_kkbox_x1/run_expid.py rename to ranking/ctr/xDeepFM/CIN_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_kkbox_x1/xDeepFM_kkbox_x1_009_117199f0.log b/ranking/ctr/xDeepFM/CIN_kkbox_x1/xDeepFM_kkbox_x1_009_117199f0.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_kkbox_x1/xDeepFM_kkbox_x1_009_117199f0.log rename to ranking/ctr/xDeepFM/CIN_kkbox_x1/xDeepFM_kkbox_x1_009_117199f0.log diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_movielenslatest_x1/CIN_movielenslatest_x1_tuner_config_02.csv b/ranking/ctr/xDeepFM/CIN_movielenslatest_x1/CIN_movielenslatest_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_movielenslatest_x1/CIN_movielenslatest_x1_tuner_config_02.csv rename to ranking/ctr/xDeepFM/CIN_movielenslatest_x1/CIN_movielenslatest_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_movielenslatest_x1/CIN_movielenslatest_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/xDeepFM/CIN_movielenslatest_x1/CIN_movielenslatest_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_movielenslatest_x1/CIN_movielenslatest_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/xDeepFM/CIN_movielenslatest_x1/CIN_movielenslatest_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_movielenslatest_x1/CIN_movielenslatest_x1_tuner_config_02/model_config.yaml b/ranking/ctr/xDeepFM/CIN_movielenslatest_x1/CIN_movielenslatest_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_movielenslatest_x1/CIN_movielenslatest_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/xDeepFM/CIN_movielenslatest_x1/CIN_movielenslatest_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_movielenslatest_x1/README.md b/ranking/ctr/xDeepFM/CIN_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_movielenslatest_x1/README.md rename to ranking/ctr/xDeepFM/CIN_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_movielenslatest_x1/environments.txt b/ranking/ctr/xDeepFM/CIN_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_movielenslatest_x1/environments.txt rename to ranking/ctr/xDeepFM/CIN_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_movielenslatest_x1/results.csv b/ranking/ctr/xDeepFM/CIN_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_movielenslatest_x1/results.csv rename to ranking/ctr/xDeepFM/CIN_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_movielenslatest_x1/run_expid.py b/ranking/ctr/xDeepFM/CIN_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_movielenslatest_x1/run_expid.py rename to ranking/ctr/xDeepFM/CIN_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/CIN_movielenslatest_x1/xDeepFM_movielenslatest_x1_001_0de9d14a.log b/ranking/ctr/xDeepFM/CIN_movielenslatest_x1/xDeepFM_movielenslatest_x1_001_0de9d14a.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/CIN_movielenslatest_x1/xDeepFM_movielenslatest_x1_001_0de9d14a.log rename to ranking/ctr/xDeepFM/CIN_movielenslatest_x1/xDeepFM_movielenslatest_x1_001_0de9d14a.log diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_amazonelectronics_x1/README.md b/ranking/ctr/xDeepFM/xDeepFM_amazonelectronics_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_amazonelectronics_x1/README.md rename to ranking/ctr/xDeepFM/xDeepFM_amazonelectronics_x1/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_amazonelectronics_x1/environments.txt b/ranking/ctr/xDeepFM/xDeepFM_amazonelectronics_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_amazonelectronics_x1/environments.txt rename to ranking/ctr/xDeepFM/xDeepFM_amazonelectronics_x1/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_amazonelectronics_x1/results.csv b/ranking/ctr/xDeepFM/xDeepFM_amazonelectronics_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_amazonelectronics_x1/results.csv rename to ranking/ctr/xDeepFM/xDeepFM_amazonelectronics_x1/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_024_abaa0691.log b/ranking/ctr/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_024_abaa0691.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_024_abaa0691.log rename to ranking/ctr/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_024_abaa0691.log diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_tuner_config_04.csv b/ranking/ctr/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_tuner_config_04.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_tuner_config_04.csv rename to ranking/ctr/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_tuner_config_04.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_tuner_config_04.yaml b/ranking/ctr/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_tuner_config_04.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_tuner_config_04.yaml rename to ranking/ctr/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_tuner_config_04.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_tuner_config_04/model_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_amazonelectronics_x1/xDeepFM_amazonelectronics_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x1/README.md b/ranking/ctr/xDeepFM/xDeepFM_avazu_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x1/README.md rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x1/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x1/environments.txt b/ranking/ctr/xDeepFM/xDeepFM_avazu_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x1/environments.txt rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x1/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x1/results.csv b/ranking/ctr/xDeepFM/xDeepFM_avazu_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x1/results.csv rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x1/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x1/run_expid.py b/ranking/ctr/xDeepFM/xDeepFM_avazu_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x1/run_expid.py rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x1/xDeepFM_avazu_x1_002_937cff08.log b/ranking/ctr/xDeepFM/xDeepFM_avazu_x1/xDeepFM_avazu_x1_002_937cff08.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x1/xDeepFM_avazu_x1_002_937cff08.log rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x1/xDeepFM_avazu_x1_002_937cff08.log diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x1/xDeepFM_avazu_x1_tuner_config_07.csv b/ranking/ctr/xDeepFM/xDeepFM_avazu_x1/xDeepFM_avazu_x1_tuner_config_07.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x1/xDeepFM_avazu_x1_tuner_config_07.csv rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x1/xDeepFM_avazu_x1_tuner_config_07.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x1/xDeepFM_avazu_x1_tuner_config_07/dataset_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_avazu_x1/xDeepFM_avazu_x1_tuner_config_07/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x1/xDeepFM_avazu_x1_tuner_config_07/dataset_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x1/xDeepFM_avazu_x1_tuner_config_07/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x1/xDeepFM_avazu_x1_tuner_config_07/model_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_avazu_x1/xDeepFM_avazu_x1_tuner_config_07/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x1/xDeepFM_avazu_x1_tuner_config_07/model_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x1/xDeepFM_avazu_x1_tuner_config_07/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_001/README.md b/ranking/ctr/xDeepFM/xDeepFM_avazu_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_001/README.md rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x4_001/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_001/environments.txt b/ranking/ctr/xDeepFM/xDeepFM_avazu_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_001/environments.txt rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_001/results.csv b/ranking/ctr/xDeepFM/xDeepFM_avazu_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_001/results.csv rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_001/run_expid.py b/ranking/ctr/xDeepFM/xDeepFM_avazu_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_001/run_expid.py rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_001/xDeepFM_avazu_x4_001_579eef11.log b/ranking/ctr/xDeepFM/xDeepFM_avazu_x4_001/xDeepFM_avazu_x4_001_579eef11.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_001/xDeepFM_avazu_x4_001_579eef11.log rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x4_001/xDeepFM_avazu_x4_001_579eef11.log diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_001/xDeepFM_avazu_x4_tuner_config_03/dataset_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_avazu_x4_001/xDeepFM_avazu_x4_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_001/xDeepFM_avazu_x4_tuner_config_03/dataset_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x4_001/xDeepFM_avazu_x4_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_001/xDeepFM_avazu_x4_tuner_config_03/model_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_avazu_x4_001/xDeepFM_avazu_x4_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_001/xDeepFM_avazu_x4_tuner_config_03/model_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x4_001/xDeepFM_avazu_x4_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_002/README.md b/ranking/ctr/xDeepFM/xDeepFM_avazu_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_002/README.md rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x4_002/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_002/environments.txt b/ranking/ctr/xDeepFM/xDeepFM_avazu_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_002/environments.txt rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_002/results.csv b/ranking/ctr/xDeepFM/xDeepFM_avazu_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_002/results.csv rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_002/run_expid.py b/ranking/ctr/xDeepFM/xDeepFM_avazu_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_002/run_expid.py rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_002/xDeepFM_avazu_x4_008_e76be77b.log b/ranking/ctr/xDeepFM/xDeepFM_avazu_x4_002/xDeepFM_avazu_x4_008_e76be77b.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_002/xDeepFM_avazu_x4_008_e76be77b.log rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x4_002/xDeepFM_avazu_x4_008_e76be77b.log diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_002/xDeepFM_avazu_x4_tuner_config_08/dataset_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_avazu_x4_002/xDeepFM_avazu_x4_tuner_config_08/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_002/xDeepFM_avazu_x4_tuner_config_08/dataset_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x4_002/xDeepFM_avazu_x4_tuner_config_08/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_002/xDeepFM_avazu_x4_tuner_config_08/model_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_avazu_x4_002/xDeepFM_avazu_x4_tuner_config_08/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_avazu_x4_002/xDeepFM_avazu_x4_tuner_config_08/model_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_avazu_x4_002/xDeepFM_avazu_x4_tuner_config_08/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x1/README.md b/ranking/ctr/xDeepFM/xDeepFM_criteo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x1/README.md rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x1/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x1/environments.txt b/ranking/ctr/xDeepFM/xDeepFM_criteo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x1/environments.txt rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x1/results.csv b/ranking/ctr/xDeepFM/xDeepFM_criteo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x1/results.csv rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x1/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x1/run_expid.py b/ranking/ctr/xDeepFM/xDeepFM_criteo_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x1/run_expid.py rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x1/xDeepFM_criteo_x1_001_e08ec7de.log b/ranking/ctr/xDeepFM/xDeepFM_criteo_x1/xDeepFM_criteo_x1_001_e08ec7de.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x1/xDeepFM_criteo_x1_001_e08ec7de.log rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x1/xDeepFM_criteo_x1_001_e08ec7de.log diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x1/xDeepFM_criteo_x1_tuner_config_03.csv b/ranking/ctr/xDeepFM/xDeepFM_criteo_x1/xDeepFM_criteo_x1_tuner_config_03.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x1/xDeepFM_criteo_x1_tuner_config_03.csv rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x1/xDeepFM_criteo_x1_tuner_config_03.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x1/xDeepFM_criteo_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_criteo_x1/xDeepFM_criteo_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x1/xDeepFM_criteo_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x1/xDeepFM_criteo_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x1/xDeepFM_criteo_x1_tuner_config_03/model_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_criteo_x1/xDeepFM_criteo_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x1/xDeepFM_criteo_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x1/xDeepFM_criteo_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_001/README.md b/ranking/ctr/xDeepFM/xDeepFM_criteo_x4_001/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_001/README.md rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x4_001/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_001/environments.txt b/ranking/ctr/xDeepFM/xDeepFM_criteo_x4_001/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_001/environments.txt rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x4_001/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_001/results.csv b/ranking/ctr/xDeepFM/xDeepFM_criteo_x4_001/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_001/results.csv rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x4_001/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_001/run_expid.py b/ranking/ctr/xDeepFM/xDeepFM_criteo_x4_001/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_001/run_expid.py rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x4_001/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_001/xDeepFM_criteo_x4_001_d509bfbe.log b/ranking/ctr/xDeepFM/xDeepFM_criteo_x4_001/xDeepFM_criteo_x4_001_d509bfbe.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_001/xDeepFM_criteo_x4_001_d509bfbe.log rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x4_001/xDeepFM_criteo_x4_001_d509bfbe.log diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_001/xDeepFM_criteo_x4_tuner_config_03/dataset_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_criteo_x4_001/xDeepFM_criteo_x4_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_001/xDeepFM_criteo_x4_tuner_config_03/dataset_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x4_001/xDeepFM_criteo_x4_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_001/xDeepFM_criteo_x4_tuner_config_03/model_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_criteo_x4_001/xDeepFM_criteo_x4_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_001/xDeepFM_criteo_x4_tuner_config_03/model_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x4_001/xDeepFM_criteo_x4_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_002/README.md b/ranking/ctr/xDeepFM/xDeepFM_criteo_x4_002/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_002/README.md rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x4_002/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_002/environments.txt b/ranking/ctr/xDeepFM/xDeepFM_criteo_x4_002/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_002/environments.txt rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x4_002/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_002/results.csv b/ranking/ctr/xDeepFM/xDeepFM_criteo_x4_002/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_002/results.csv rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x4_002/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_002/run_expid.py b/ranking/ctr/xDeepFM/xDeepFM_criteo_x4_002/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_002/run_expid.py rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x4_002/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_002/xDeepFM_criteo_x4_003_c601dd6b.log b/ranking/ctr/xDeepFM/xDeepFM_criteo_x4_002/xDeepFM_criteo_x4_003_c601dd6b.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_002/xDeepFM_criteo_x4_003_c601dd6b.log rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x4_002/xDeepFM_criteo_x4_003_c601dd6b.log diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_002/xDeepFM_criteo_x4_tuner_config_03/dataset_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_criteo_x4_002/xDeepFM_criteo_x4_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_002/xDeepFM_criteo_x4_tuner_config_03/dataset_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x4_002/xDeepFM_criteo_x4_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_002/xDeepFM_criteo_x4_tuner_config_03/model_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_criteo_x4_002/xDeepFM_criteo_x4_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_criteo_x4_002/xDeepFM_criteo_x4_tuner_config_03/model_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_criteo_x4_002/xDeepFM_criteo_x4_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_frappe_x1/README.md b/ranking/ctr/xDeepFM/xDeepFM_frappe_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_frappe_x1/README.md rename to ranking/ctr/xDeepFM/xDeepFM_frappe_x1/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_frappe_x1/environments.txt b/ranking/ctr/xDeepFM/xDeepFM_frappe_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_frappe_x1/environments.txt rename to ranking/ctr/xDeepFM/xDeepFM_frappe_x1/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_frappe_x1/results.csv b/ranking/ctr/xDeepFM/xDeepFM_frappe_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_frappe_x1/results.csv rename to ranking/ctr/xDeepFM/xDeepFM_frappe_x1/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_frappe_x1/run_expid.py b/ranking/ctr/xDeepFM/xDeepFM_frappe_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_frappe_x1/run_expid.py rename to ranking/ctr/xDeepFM/xDeepFM_frappe_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_frappe_x1/xDeepFM_frappe_x1_005_447fa536.log b/ranking/ctr/xDeepFM/xDeepFM_frappe_x1/xDeepFM_frappe_x1_005_447fa536.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_frappe_x1/xDeepFM_frappe_x1_005_447fa536.log rename to ranking/ctr/xDeepFM/xDeepFM_frappe_x1/xDeepFM_frappe_x1_005_447fa536.log diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_frappe_x1/xDeepFM_frappe_x1_tuner_config_02.csv b/ranking/ctr/xDeepFM/xDeepFM_frappe_x1/xDeepFM_frappe_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_frappe_x1/xDeepFM_frappe_x1_tuner_config_02.csv rename to ranking/ctr/xDeepFM/xDeepFM_frappe_x1/xDeepFM_frappe_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_frappe_x1/xDeepFM_frappe_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_frappe_x1/xDeepFM_frappe_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_frappe_x1/xDeepFM_frappe_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_frappe_x1/xDeepFM_frappe_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_frappe_x1/xDeepFM_frappe_x1_tuner_config_02/model_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_frappe_x1/xDeepFM_frappe_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_frappe_x1/xDeepFM_frappe_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_frappe_x1/xDeepFM_frappe_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_kkbox_x1/README.md b/ranking/ctr/xDeepFM/xDeepFM_kkbox_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_kkbox_x1/README.md rename to ranking/ctr/xDeepFM/xDeepFM_kkbox_x1/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_kkbox_x1/environments.txt b/ranking/ctr/xDeepFM/xDeepFM_kkbox_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_kkbox_x1/environments.txt rename to ranking/ctr/xDeepFM/xDeepFM_kkbox_x1/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_kkbox_x1/results.csv b/ranking/ctr/xDeepFM/xDeepFM_kkbox_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_kkbox_x1/results.csv rename to ranking/ctr/xDeepFM/xDeepFM_kkbox_x1/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_kkbox_x1/run_expid.py b/ranking/ctr/xDeepFM/xDeepFM_kkbox_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_kkbox_x1/run_expid.py rename to ranking/ctr/xDeepFM/xDeepFM_kkbox_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_kkbox_x1/xDeepFM_kkbox_x1_019_49197bdf.log b/ranking/ctr/xDeepFM/xDeepFM_kkbox_x1/xDeepFM_kkbox_x1_019_49197bdf.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_kkbox_x1/xDeepFM_kkbox_x1_019_49197bdf.log rename to ranking/ctr/xDeepFM/xDeepFM_kkbox_x1/xDeepFM_kkbox_x1_019_49197bdf.log diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_kkbox_x1/xDeepFM_kkbox_x1_tuner_config_03/dataset_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_kkbox_x1/xDeepFM_kkbox_x1_tuner_config_03/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_kkbox_x1/xDeepFM_kkbox_x1_tuner_config_03/dataset_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_kkbox_x1/xDeepFM_kkbox_x1_tuner_config_03/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_kkbox_x1/xDeepFM_kkbox_x1_tuner_config_03/model_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_kkbox_x1/xDeepFM_kkbox_x1_tuner_config_03/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_kkbox_x1/xDeepFM_kkbox_x1_tuner_config_03/model_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_kkbox_x1/xDeepFM_kkbox_x1_tuner_config_03/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_kuaivideo_x1/README.md b/ranking/ctr/xDeepFM/xDeepFM_kuaivideo_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_kuaivideo_x1/README.md rename to ranking/ctr/xDeepFM/xDeepFM_kuaivideo_x1/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_kuaivideo_x1/environments.txt b/ranking/ctr/xDeepFM/xDeepFM_kuaivideo_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_kuaivideo_x1/environments.txt rename to ranking/ctr/xDeepFM/xDeepFM_kuaivideo_x1/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_kuaivideo_x1/results.csv b/ranking/ctr/xDeepFM/xDeepFM_kuaivideo_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_kuaivideo_x1/results.csv rename to ranking/ctr/xDeepFM/xDeepFM_kuaivideo_x1/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_016_0372eae9.log b/ranking/ctr/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_016_0372eae9.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_016_0372eae9.log rename to ranking/ctr/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_016_0372eae9.log diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_tuner_config_02.csv b/ranking/ctr/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_tuner_config_02.csv rename to ranking/ctr/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_tuner_config_02.yaml b/ranking/ctr/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_tuner_config_02.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_tuner_config_02.yaml rename to ranking/ctr/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_tuner_config_02.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_tuner_config_02/model_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_kuaivideo_x1/xDeepFM_kuaivideo_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_microvideo1.7m_x1/README.md b/ranking/ctr/xDeepFM/xDeepFM_microvideo1.7m_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_microvideo1.7m_x1/README.md rename to ranking/ctr/xDeepFM/xDeepFM_microvideo1.7m_x1/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_microvideo1.7m_x1/environments.txt b/ranking/ctr/xDeepFM/xDeepFM_microvideo1.7m_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_microvideo1.7m_x1/environments.txt rename to ranking/ctr/xDeepFM/xDeepFM_microvideo1.7m_x1/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_microvideo1.7m_x1/results.csv b/ranking/ctr/xDeepFM/xDeepFM_microvideo1.7m_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_microvideo1.7m_x1/results.csv rename to ranking/ctr/xDeepFM/xDeepFM_microvideo1.7m_x1/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_011_7df31553.log b/ranking/ctr/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_011_7df31553.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_011_7df31553.log rename to ranking/ctr/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_011_7df31553.log diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_tuner_config_04.csv b/ranking/ctr/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_tuner_config_04.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_tuner_config_04.csv rename to ranking/ctr/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_tuner_config_04.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_tuner_config_04.yaml b/ranking/ctr/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_tuner_config_04.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_tuner_config_04.yaml rename to ranking/ctr/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_tuner_config_04.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_tuner_config_04/dataset_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_tuner_config_04/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_tuner_config_04/dataset_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_tuner_config_04/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_tuner_config_04/model_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_tuner_config_04/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_tuner_config_04/model_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_microvideo1.7m_x1/xDeepFM_microvideo1.7m_x1_tuner_config_04/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_movielenslatest_x1/README.md b/ranking/ctr/xDeepFM/xDeepFM_movielenslatest_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_movielenslatest_x1/README.md rename to ranking/ctr/xDeepFM/xDeepFM_movielenslatest_x1/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_movielenslatest_x1/environments.txt b/ranking/ctr/xDeepFM/xDeepFM_movielenslatest_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_movielenslatest_x1/environments.txt rename to ranking/ctr/xDeepFM/xDeepFM_movielenslatest_x1/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_movielenslatest_x1/results.csv b/ranking/ctr/xDeepFM/xDeepFM_movielenslatest_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_movielenslatest_x1/results.csv rename to ranking/ctr/xDeepFM/xDeepFM_movielenslatest_x1/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_movielenslatest_x1/run_expid.py b/ranking/ctr/xDeepFM/xDeepFM_movielenslatest_x1/run_expid.py similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_movielenslatest_x1/run_expid.py rename to ranking/ctr/xDeepFM/xDeepFM_movielenslatest_x1/run_expid.py diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_movielenslatest_x1/xDeepFM_movielenslatest_x1_018_57ed221b.log b/ranking/ctr/xDeepFM/xDeepFM_movielenslatest_x1/xDeepFM_movielenslatest_x1_018_57ed221b.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_movielenslatest_x1/xDeepFM_movielenslatest_x1_018_57ed221b.log rename to ranking/ctr/xDeepFM/xDeepFM_movielenslatest_x1/xDeepFM_movielenslatest_x1_018_57ed221b.log diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_movielenslatest_x1/xDeepFM_movielenslatest_x1_tuner_config_02.csv b/ranking/ctr/xDeepFM/xDeepFM_movielenslatest_x1/xDeepFM_movielenslatest_x1_tuner_config_02.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_movielenslatest_x1/xDeepFM_movielenslatest_x1_tuner_config_02.csv rename to ranking/ctr/xDeepFM/xDeepFM_movielenslatest_x1/xDeepFM_movielenslatest_x1_tuner_config_02.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_movielenslatest_x1/xDeepFM_movielenslatest_x1_tuner_config_02/dataset_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_movielenslatest_x1/xDeepFM_movielenslatest_x1_tuner_config_02/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_movielenslatest_x1/xDeepFM_movielenslatest_x1_tuner_config_02/dataset_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_movielenslatest_x1/xDeepFM_movielenslatest_x1_tuner_config_02/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_movielenslatest_x1/xDeepFM_movielenslatest_x1_tuner_config_02/model_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_movielenslatest_x1/xDeepFM_movielenslatest_x1_tuner_config_02/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_movielenslatest_x1/xDeepFM_movielenslatest_x1_tuner_config_02/model_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_movielenslatest_x1/xDeepFM_movielenslatest_x1_tuner_config_02/model_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_taobaoad_x1/README.md b/ranking/ctr/xDeepFM/xDeepFM_taobaoad_x1/README.md similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_taobaoad_x1/README.md rename to ranking/ctr/xDeepFM/xDeepFM_taobaoad_x1/README.md diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_taobaoad_x1/environments.txt b/ranking/ctr/xDeepFM/xDeepFM_taobaoad_x1/environments.txt similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_taobaoad_x1/environments.txt rename to ranking/ctr/xDeepFM/xDeepFM_taobaoad_x1/environments.txt diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_taobaoad_x1/results.csv b/ranking/ctr/xDeepFM/xDeepFM_taobaoad_x1/results.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_taobaoad_x1/results.csv rename to ranking/ctr/xDeepFM/xDeepFM_taobaoad_x1/results.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_002_3f7806c5.log b/ranking/ctr/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_002_3f7806c5.log similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_002_3f7806c5.log rename to ranking/ctr/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_002_3f7806c5.log diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_tuner_config_01.csv b/ranking/ctr/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_tuner_config_01.csv similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_tuner_config_01.csv rename to ranking/ctr/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_tuner_config_01.csv diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_tuner_config_01.yaml b/ranking/ctr/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_tuner_config_01.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_tuner_config_01.yaml rename to ranking/ctr/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_tuner_config_01.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_tuner_config_01/dataset_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_tuner_config_01/dataset_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_tuner_config_01/dataset_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_tuner_config_01/dataset_config.yaml diff --git a/ctr_prediction/benchmarks/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_tuner_config_01/model_config.yaml b/ranking/ctr/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_tuner_config_01/model_config.yaml similarity index 100% rename from ctr_prediction/benchmarks/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_tuner_config_01/model_config.yaml rename to ranking/ctr/xDeepFM/xDeepFM_taobaoad_x1/xDeepFM_taobaoad_x1_tuner_config_01/model_config.yaml diff --git a/requirements.txt b/requirements.txt index 45f73dcd..ed8f708a 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1 +1,2 @@ -fuxictr \ No newline at end of file +fuxictr +recbox \ No newline at end of file