-
Notifications
You must be signed in to change notification settings - Fork 137
Google Summer of Code 2024 Ideas
This idea page is for multiple Borg-related projects listed below. In short what each project does:
- Borg is a Python-based file backup tool. It can also compress, encrypt/authenticate and deduplicate the data. Repository for Borg
- Borgmatic is a CLI wrapper for it that keeps settings and runs pre- and post-backup tasks. It basically helps manage your Borg repos, settings and tasks that go with it. It also deals with monitoring. Repository for Borgmatic
- Vorta is a desktop GUI for Borg. It sits in your task bar and runs backups in the background. You can also use it to view and restore different versions of your files. Vorta officially supports Linux and MacOS while the community is currently testing running vorta on Windows. Our latest release was downloaded over 9000 times via flathub or the MacOS installer. However many users install Vorta from the package repository of their Linux distribution. Repository for Vorta
In addition we have an Ansible role and a Docker image to make installing Borg and Borgmatic easier on servers.
Technologies used: All projects use Python as main language. Vorta also relies on Qt for the GUI parts. The devops tasks use Ansible and Docker.
GSoC is a program that allows students and open source beginners to learn contributing to an open-source project while receiving a stipend from Google, and mentorship from open-source software developers. For details about this year's GSoC, please refer to this page.
The Borg Collective is part of the Python Software Foundation. Please read about our expectations.
- Get in touch on Github or IRC and tell us that you are interested in one of our projects. This doesn't represent any commitment to apply as a contributor. You can choose to apply for a different project at any time.
- Select an easy issue which are tagged/labelled with
good first issueand try solving it. (view such issues in each project: Vorta, Borg, Borgmatic) This allows you to get to know the project and its source code. This helps you determining whether it suits you and whether you want to apply as a contributor. Solving an issue and opening a pull request (PR) for it will also be required for applying. - Before working on an issue, please comment and make sure it's assigned to you, so no duplicate work is done. Also outline how you plan on solving the issue, so you are on the right track from the start. We can give you some hints on how to go about, but we also want to see how you manage on your own.
- Open a PR fixing the issue. We will then review it as usual. Don't feel discouraged if the reviewer requests changes to your PR. This doesn't question your skills. We just want the code to meet our (formal) standards.
- When you are sure that you want to apply as a contributor to one of our projects, it is time to discuss the project ideas that you are interested in with your prospected mentors on IRC/Matrix. We will set up a private room with the relevant members. Make sure to communicate your current knowledge and expertise so that your prospected mentors can guide you adequately. After this step you should have a clear understanding of task you will apply for including expected results, tasks and steps towards that goal, time effort.
- Your prospective mentors will then help you with writing your application. It is very important to check in with them on your application before submitting. Else your likelihood of being accepted shrinks drastically. To get quick feedback it's best to share your application using something like Google Docs with comments enabled.
During all steps communication is key. Talk to the mentors as well as other community members, take their feedback and advise into account. You can find more information about the process alongside tips on the website of the PSF.
To start coding, see the existing contributor guides for each project and setup your development environment:
Feel free to ask questions. On your journey you might encounter unclear, insufficient or missing documentation. Don't hesitate to point out the problems, so we can help you out as well as improve the documentation of our projects.
These are tasks you can work on. You can combine any number of tasks, so they add up to a full- or part time project. You can also suggest your own tasks to supplement the ones below. Especially if they are related. Keep in mind that this is not about picking the easiest tasks, but learning something that will be useful later. It's best to pick tasks from a single project, so you can work with the same mentor the whole time.
Difficulty: Medium
Length: 90 hours
Skills required: Python, Qt, Unix
Description:
There is already some code to catch errors and display a simple dialog. However it isn't used consistently.
This task would add a dialog displaying the errors user friendly and a wizard for reporting an issue on Github. This task would also ensure we catch long Borg errors (maybe via return code).
Task outline:
Get familiar with the kind of errors encountered in vorta and how they are handled.
Plan out the use cases and the functionalities of the error dialog and report wizard.
Draw a mockup of the new GUI.
Implement the GUI as a Qt UI file.
Do the coding needed to make the GUI functional.
Adjust the existing vorta code to use/work with the new dialog.
Possible mentors: @real-yfprojects, @m3nu, @Hofer-Julian
Difficulty: Easy
Length: 90 hours
Skills required: Python, Qt, Regex, Linux
Description: Last year we added a new exclusion dialog. It allows to ship presets that users can easily exclude in their backups. E.g. to exclude all webdev files. This task would look into how to best add more built-in exclusions while keeping the proecess maintainable.
Task outline:
Research exclusion patterns that would be useful to include
Suggest ways to ship them in Vorta, but also keep them updated
Implement any changes needed in Exclusion dialog
Additional details: See discussion #1231
Possible mentors: @real-yfprojects, @m3nu, @Hofer-Julian
Difficulty: Medium-hard
Length: 90 hours
Skills required: Python, Unix
Description: Vorta was first built for user backups. Some users also want to run it as root and backup the whole system. There are minor blocking issues to achieve this. This task would resolve those and write documentation for it. This is a large task that needs additional Linux knowledge.
Task outline:
Research ways of running vorta as root on Linux and possible ways to only run borg subcommands as root (polkit, sudo, root install, ...).
Weigh up the advantages and disadvantages of the options and decide on a way to implement.
Implement support for root backups.
Additional details: See issues #801 and #1482
Possible mentors: @real-yfprojects, @m3nu
Difficulty: Medium
Length: 175 hours
Skills required: Python, Qt
Description: Currently Vorta just shows 0, 1 or 2 to inform about borg's return code and users have to wonder about what that could mean. It needs to offer a borg log file view, so that users can find more information about problems and also about successful runs. The log file view could colour the log lines according to their log level (e.g. display ERROR in red). A more advanced log line highlighter would be even better. The log viewer should also allowing filtering log lines by application session, borg command and other attributes.
Task outline: Plan out how the log file viewer would integrate into the existing GUI. Plan out how the log file viewer should work from a user perspective. Draw a GUI mockup. Implement the GUI as a Qt UI file. Implement the logic of the new dialog. Implement filtering logs. Implement log syntax highlighting. Write unittests throughtout the process.
Additional details: See issue #1483
Possible mentors: @real-yfprojects, @m3nu, @Hofer-Julian
Difficulty: Medium
Length: 175 hours
Skills required: Python, Unix, basic Qt
Description:
Vorta already allows specifying a custom command before and after borg create is run. We would like to have a more versatile way to run custom commands for different hooks. For example some users want to mount a drive before every borg command and unmount it afterwards. There should be hooks for these use cases. The custom commands could also be able to modify the behaviour of Vorta, e.g. through their output or by settings environment variables. The Vorta community did already thought about different ways of implementing such a feature.
Our favourite one includes a script the user can configure that is called before and after every command. Vorta provides the script with information about the profile, repository and the hook the script is called for. The user can freely configure which commands the script executes for a given hook or set of arguments.
Task outline:
Read this thread. Inform yourself about the use cases this feature might be able to help with.
Plan out the details of how this feature should work.
Implement GUI widget to configure the script.
Write code that makes vorta run the script for each hook and provides it with the correct arguments.
Write unittests for the implemented feature(s).
Write script template that allows users to get started quickly with configuring custom commands for common use cases.
Write documentation for this feature describing how to use it and how it works from a user perspective.
Additional details: See issue #379
Possible mentors: @real-yfprojects, @m3nu, @Hofer-Julian
You can also come up with own ideas to implement or choose to solve any other existing issue. Discuss your ideas with you prospective mentors. Some possible options:
- #1924 support BORG_EXIT_CODES=modern
- #1918 add UI for borg key backup
- #1852 RTL scripts support (a mentor with some experience with that would be good)
Difficulty: Easy
Length: 90 hours
Skills required: Python, Shell, reStructuredText
Description: For the next release of Borg, a few commands have changed subtly. This small task would update shell completions for popular shells, like Zsh. Borg's website shows a quick intro video that demonstrates how Borg works at a high level. There was already some work to automate the creation of this video, but still needs an update to cover the upcoming v2 release.
Task outline: Review existing completions and video and adjust and test them for Borg v2
Additional details: See this issue
Possible mentors: @m3nu, backup mentor: @ThomasWaldmann
Difficulty: Easy
Length: 175 hours
Skills required: Unix, Shell, Google research
Description: This would help users decide on the backup tool to use. Naturally there is no single best option for all use cases. This large task includes less programming and more research and qualitative work. You may need to write some scripts for benchmarking.
Task outline: First step would be to determine the features to use for comparing. Then do qualitative research to find them for each backup tool. Finally set up a benchmarking process for quantitative comparison.
Additional details: Some work on this was already done for our docs and a recent benchmark.
Possible mentors: @m3nu, backup mentor: @ThomasWaldmann
Expand Borg import tool (low priority)
Difficulty: Medium
Length: 350 hours
Skills required: Python, Unix
Description: This is a large task, but doesn't require in-depth Borg knowledge. It would involve adding support for additional hardlink-based backup tools. So those users can migrate their existing backups into Borg.
Task outline: Together with Borg maintainers you will decide on which backup tools to support for importing. Then research their format and add the import code and test cases.
Possible mentors: @m3nu, backup mentor: @ThomasWaldmann
Other issues tagged help wanted and good first issue
Difficulty: Easy
Length: 90 hours
Skills required: Python, Linux
Description: borgmatic is effectively a wrapper around Borg backup, providing additional features like a configuration file, database integration, etc. But borgmatic only wraps a fraction of the sub-commands that Borg provides. And for those that it does wrap, it doesn't necessarily support all command-line flags as borgmatic options. Users can always drop back down to running Borg directly for those missing sub-commands (or use the borgmatic borg action), but that doesn't provide all the conveniences of borgmatic and its configuration file.
Task outline: Implement borgmatic actions for all Borg sub-commands that are not yet implemented. For each Borg flag within those sub-commands, decide whether it makes sense to add a new borgmatic configuration option for it—or whether it would be more appropriate as a borgmatic action command-line flag. Also as part of this work, consider implementing missing flags/options on existing borgmatic actions.
Additional details: Not all Borg sub-commands make sense to wrap. For instance, Borg invokes borg serve internally, and there's likely not a good use case for running it via borgmatic. Similarly, some Borg flags like --info and --debug shouldn't be exposed directly via borgmatic configuration options or command-line flags, because borgmatic uses them implicitly (e.g. via --verbosity) without exposing them to the end-user.
Difficulty: Medium
Length: 90 hours
Skills required: Python, Linux, MySQL/MariaDB
Description: Today borgmatic supports dumping MySQL/MariaDB databases directly to Borg for backup purposes and also restoring them directly from Borg. However, borgmatic does not support the MySQL/MariaDB directory format for database dumps currently.
Task outline: Implement a new directory value (or maybe tab?) for the existing MySQL/MariaDB format configuration options. When that's set, pass the relevant option (--tab=...?) to mysqldump so that database dumps are dumped into a directory instead of as a single file. Note that this will have to bypass the existing streaming to named pipe logic since a directory can't be streamed that way. Also note that there are separate MySQL and MariaDB hooks, which would both need similar updates to support this feature.
Additional details: Look at the existing PostgreSQL hook for an example of a database hook that supports both streaming database dumps and non-streaming directory format database dumps. You can probably take a similar approach with this MySQL/MariaDB work. Also see the ticket.
Difficulty: Medium
Length: 90 hours
Skills required: Python, Linux, Docker/Podman
Description: borgmatic has a variety of hooks for dumping non-filesystem data sources like databases, but now there's also a need to dump containers or container volumes for backup.
Task outline: Implement a new borgmatic hook or multiple hooks for dumping and restoring containers or container volumes. Test against both Docker and Podman.
Additional details: See the ticket for design and implementation ideas.
Also see good first issues.
For the Borg+Borgmatic Docker image and Ansible role
Difficulty: Medium
Length: 90 hours
Skills required: Ansible, Docker, Linux
Description: Borgmatic's config file format has rapidly changed and our Ansible role hasn't fully caught up yet. This project would work on adding the new file format as template, adjust testing and add a condition on when to use which version. Also address smaller issues related to recent Borgmatic updates, like database hooks.
Task outline: Research how verification is done at similar projects. Then make a list of things we like to test. Then see which ones we can test in the context of a Docker test.
Additional details: See this issue for more.
Possible mentors: @m3nu
Difficulty: Medium
Length: 90 hours
Skills required: Ansible, Molecule, Docker, Linux
Description: We already use Molecule to test this role, but only do a few basic validations. This task would add more validations and maybe run a local backup to make sure the role works well.
Task outline: Research how verification is done at similar projects. Then make a list of things we like to test. Then see which ones we can test in the context of a Docker test.
Additional details: See this issue for more.
Possible mentors: @m3nu, @grantbevis (backup mentor)