[Bug]: Browsing bug result from accessibility tree

[yufansong]

Is there an existing issue for the same bug?

• I have checked the existing issues.

Describe the bug and reproduction steps

The "x" button on the top right corner doesn't show up in the accessibility tree, so agents has a hard time closing this layover.
The following image come from the owncloud service.

OpenHands Installation

Docker command in README

OpenHands Version

No response

Operating System

None

Logs, Errors, Screenshots, and Additional Context

No response

[ryanhoangt]

I can have a 👀 at the browsergym to see if we can raise a PR to fix it.

[ryanhoangt]

I looked into this issue, and it seems to me like a bug from the owncloud service. Below is the HTML from the modal, where the <a> tag misses the href attribute (which should be set to "#"). This causes the ax tree to use a generic node role for that close button (id 821) instead of a clickable link.

We may need to raise an issue for the owncloud repo to fix it.

[yufansong]

Get it. Thanks for the investigation!

[li-boxuan]

@ryanhoangt Great finding! I still have a doubt: per this standard an a tag without a href attribute is still valid, and it doesn't seem to be an uncommon usage: https://stackoverflow.com/questions/10510191/valid-to-use-a-anchor-tag-without-href-attribute

I'd say browsergym should treat all a tags as clickable.

[ryanhoangt]

Actually I see if the <a> tag misses the href attribute, the accessibility tree returned by the browser will use the generic node type. Browsergym just does the transformation job on that axtree and makes it available to the agent I think.

[li-boxuan]

Actually I see if the <a> tag misses the href attribute, the accessibility tree returned by the browser will use the generic node type. Browsergym just does the transformation job on that axtree and makes it available to the agent I think.

Thanks, I guess we could say it's a limitation of accessibility tree approach then