Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can an online assessment yield a score, or can the process of an offline assessment be visualized? #22

Open
Tangent-90C opened this issue Apr 8, 2024 · 0 comments

Comments

@Tangent-90C
Copy link

1712585243173

I wanted to visualize how the model action on the Mind2Web dataset, but SeeAct didn't seem to do that.
When computing online, the output "success_or_not" is always empty, which means that success is not known.
Offline evaluation does not allow you to visualize model actions.

Despite the Action_history, it is not readable by humans. So can we make SeeAct support the actions of the visual model? Or tell me which user implemented that feature in their code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant