-
Notifications
You must be signed in to change notification settings - Fork 266
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
probe: add Arabic donotanswer #1017
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: Emad Alghamdi <37039039+Eaalghamdi@users.noreply.github.com>
DCO Assistant Lite bot All contributors have signed the DCO ✍️ ✅ |
I have read the DCO Document and I hereby sign the DCO |
recheck |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for this! Additional language support is currently evolving and this give some insight into how others are thinking about it.
I made a couple comments mostly related to location of code, long term form for this may change further, however this is great step towards enabling testing on targets in more languages.
garak/detectors/mitigation_ar.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think language specific detectors are likely better contained in the same module. In this case adding this class to garak/detectors/mitigation.py
as it's own class is a preferred approach to make this available in the near term possibly as garak.detectors.mitigation.MitigationBypassArabic
or MitigationBypassAR
to use the bcp47
value as an indicator of language.
In a future iteration I would like to see garak.detectors.mitigation.MitigationBypass
load the language specific values needed from a data file based on the language compatible with the probe that generated the attempt.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Language specific implementations are likely better contained in the same module. In this case adding these classes to garak/probes/donotanswer.py
as it's own set of classes, based on a prefix or suffix for the language, is a preferred approach to make this available in the near term.
In a future iteration I would like to see garak.probes.donotanswer.*
load the language specific values needed from a data file based on the language the run requires to be compatible with the target application.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I modified the original code to support Arabic (and potentially other languages). Now language specific implementation is included within the targeted probe and detector module. Finally, Arabic data path is changed to /data/ar/*
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A minor file structure item, consider using bcp47
values as a standard already adopted by the project to determine the storage path for language specific values, in this case garak/data/ar/*
might provide a more programatic access friendly pattern locating files for a specific language.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
Signed-off-by: Emad Alghamdi 37039039+Eaalghamdi@users.noreply.github.com
I added Arabic translation of all donotanswer probes which were quality checked by human to ensure suitability to the Arabic language, added Arabic detector for the probe. The new probes and detectors passed the test during development.