Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add COMET metric to EvaluationStep #30

Open
varisd opened this issue Jul 26, 2024 · 0 comments
Open

Add COMET metric to EvaluationStep #30

varisd opened this issue Jul 26, 2024 · 0 comments
Labels
feature New feature or request help wanted Extra attention is needed

Comments

@varisd
Copy link
Contributor

varisd commented Jul 26, 2024

Current version of the EvaluationStep only supports automatic Sacrebleu metrics. We want to add additional support for COMET (which was used during the D5.1 evaluation).

It should likely be based on the code from the deliverable repo.
@varisd varisd added the feature New feature or request label Jul 26, 2024
@varisd varisd added this to the MVP - August 31, 2024 milestone Jul 26, 2024
@varisd varisd added the help wanted Extra attention is needed label Aug 23, 2024
@rggdmonk rggdmonk removed this from the MVP - August 31, 2024 milestone Aug 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature New feature or request help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@varisd @rggdmonk