Learning to Expire - Pytorch (wip)

An implementation of Transformer with Expire-Span, a proposed technique for learning which memories to retain for long-range learning in attention-based networks.

Citations

@inproceedings{
    anonymous2021not,
    title={Not All Memories are Created Equal: Learning to Expire},
    author={Anonymous},
    booktitle={Submitted to International Conference on Learning Representations},
    year={2021},
    url={https://openreview.net/forum?id=ZVBtN6B_6i7},
    note={under review}
}