Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added readme for rfb project #67

Merged
merged 3 commits into from
Jun 18, 2024
Merged

Added readme for rfb project #67

merged 3 commits into from
Jun 18, 2024

Conversation

jarumihooi
Copy link
Collaborator

@jarumihooi jarumihooi commented Oct 18, 2023

fixes #65

@jarumihooi
Copy link
Collaborator Author

Requesting review of this readme for developing maturity of the project's documentation. Readme should be in a readable state containing up to present information.

@jarumihooi jarumihooi requested review from keighrim and removed request for keighrim November 30, 2023 02:19
@keighrim
Copy link
Member

Let's wait for

  • "raw" annotation files are ready to go public
  • resolution on handling dually annotated data #71
  • decision on where to put IAA code
  • (possibly) revision of the readme for additional SWT types

If you want to switch the target branch to a somewhat intermediate "develop" branch for RFB in general, feel free to do so. I just don't think the readme file only isn't enough to go into the main branch.

@keighrim keighrim added this to the rfb-v1 milestone Jan 29, 2024
@keighrim
Copy link
Member

keighrim commented Feb 6, 2024

From yesterday's meeting, here's some (interim) decisions:

  • "raw" annotation files are ready to go public

R2 annotation files are ready.

(interim) For R2 annotation, after eyeballing on the initial IAA results, we decided to use Gabe's as base and do "union" of two annotation sets to create the first public version gold.

  • decision on where to put IAA code

The code is being pushed to https://github.com/clamsproject/aapb-annenv-role-filler-binder/tree/iaa_calculation branch.

  • (possibly) revision of the readme for additional SWT types

We decided a while ago to completely disregard SWT-like subtask from the RFB annotation, so this point shouldn't be relevant any more.

Copy link
Member

@keighrim keighrim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please

  1. add "gold" (by simple merge) (Manual or by writing code? Pick whichever you think the quicker)
  2. update readme with documenting (process.py section) of the adjudication steps
  3. update the guideline link to the newly formatted document

@jarumihooi
Copy link
Collaborator Author

During another conversation, I believe for these two reasons, we decided to hold off on a publicized golds:

  1. We decided to use Gabe's raws as uploaded raws for now.
  2. Other tools may have requests for certain formatting of the golds, therefore more might be required during processing.

Let's scope this issue as starting a readme, documenting the batches, and uploading one of the raws. A new issue can be created for formalizing the golds format and process.py

@keighrim keighrim modified the milestones: rfb-v1, eval-v1 May 30, 2024
@keighrim keighrim merged commit 9cbe41a into main Jun 18, 2024
@keighrim keighrim deleted the 65-rfb-readme branch June 18, 2024 18:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Add readme and guidelines to RFB project
2 participants