Skip to content

Update evaluators.py#77

Open
Miking98 wants to merge 1 commit intoweb-arena-x:mainfrom
Miking98:fix-url-eval
Open

Update evaluators.py#77
Miking98 wants to merge 1 commit intoweb-arena-x:mainfrom
Miking98:fix-url-eval

Conversation

@Miking98
Copy link

Addresses an eval bug that causes false positives.

Bug: Before, the URL ec2-3-131-244-37.us-east-2.compute.amazonaws.com:7780/admin/reports/report_product/viewedasdf would get a score of 1.0 for the reference URL ec2-3-131-244-37.us-east-2.compute.amazonaws.com:7780/admin/reports/report_product/viewed.

Edit. This commit requires the URLs to match exactly to get credit.

Fixes bug that causes false positives.


e.g. before, the URL `ec2-3-131-244-37.us-east-2.compute.amazonaws.com:7780/admin/reports/report_product/viewedasdf` would get a score of 1.0 for the reference URL `ec2-3-131-244-37.us-east-2.compute.amazonaws.com:7780/admin/reports/report_product/viewed`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments