Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proper filter for aarch64, win32 #1201

Open
JPrazak opened this issue Sep 5, 2022 · 5 comments
Open

Proper filter for aarch64, win32 #1201

JPrazak opened this issue Sep 5, 2022 · 5 comments
Labels
help wanted Extra attention is needed needs_external_pr Will rely on non maintainer PR in order to close

Comments

@JPrazak
Copy link

JPrazak commented Sep 5, 2022

Hi,
Iam little bit confused with packages filtering. Plugin exclude_platfrom work well for py versions and 4 main platforms. Can I also blocklist/filter aarch64 and win32 packages or which filter I should use?
Thank you.

@89ao
Copy link
Contributor

89ao commented Sep 6, 2022

maybe this can help you:

Platform/Python-specific binaries filtering
This filter allows advanced users not interesting in Windows/macOS/Linux specific binaries to not mirror the corresponding files.

You can also exclude Python versions by their minor version (ex. Python 2.6, 2.7) if you’re sure your mirror does not need to serve these binaries.

[plugins]
enabled =
    exclude_platform
[blocklist]
platforms =
    windows
    py2.6
    py2.7
Available platforms are:

windows

macos

freebsd

linux

Available python versions are:

py2.4 ~ py2.7

py3.1 ~ py3.10


or just use Regex filtering

@JPrazak
Copy link
Author

JPrazak commented Sep 6, 2022

Yes, I use this plugin and blocklist filter. It works very well as excepted for python versions, macos and freebsd.
But I can't filter aarch64 or win32 packages.
Thank You very much.

@cooperlees
Copy link
Contributor

I wouldn't be surprised of bug(s) existing here. A lot of plugins have been community contributed and had little involvement from any core bandersnatch developer (me). Debug logging and running a local build of bandersnatch with more logging to workout why there are not working is probably needed here. Another way would be to improve our plugin unittests as at the end of the day it's just checking metadata in a way to workout if we download a package or not.

As always, a PR fixing, with unittests, would be accepted.

@cooperlees cooperlees added help wanted Extra attention is needed needs_external_pr Will rely on non maintainer PR in order to close labels Sep 6, 2022
@imnoterel
Copy link

You can already do that to some extent.

Instead of just "linux" or "windows", you can put any platform type listed in the ExcludePlatformFilter._windowsPlatformTypes or ExcludePlatformFilter._linuxPlatformTypes found in src/bandersnatch_filter_plugins/filename_name.py in your blocklist and this particular platform will be excluded from your mirror.

In other words, this should work in your bandersnatch config :

[plugins]
enabled = 
    exclude_platform

[blocklist]
platforms = 
    linux-i686
    manylinux2014_aarch64
    .win32
    -win32

It's not properly documented and there are some platform types missing (musl for example) though. I can work on a small PR to add the missing platforms and update the documentation to give more details on how it works / which platforms are available if you want.

@cooperlees
Copy link
Contributor

Thanks. Will totally take that PR(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed needs_external_pr Will rely on non maintainer PR in order to close
Projects
None yet
Development

No branches or pull requests

4 participants