This summer I had the opportunity to participate in Google Summer of Code, where I contributed to the Borg Collective organization under Python Software Foundation. Working on backup software like Vorta and Borgmatic allowed me to make an impact on tools people use to protect their data. Over the last few months, I implemented several enhancements to improve the user experience and reliability of these critical applications.
In this post, I’ll summarize my journey as a first-time GSoC contributor, including the projects I took on, what I learned along the way, and how this experience has positively impacted my growth as a developer. I’m grateful I got to spend my summer coding for open source while gaining real-world skills. Read on for a recap of my GSoC 2023 experience!
Links:
My contributions during this program
During the community bonding period, we rearranged some of the tasks and my actual work slightly differs from the proposed project. Here’s what I worked on during this program.
| Project | Name | Status | Issue | PR |
| Vorta | Integration Tests for Borg | Merged | #1711 | #1716 |
| Vorta | Diff and Extract View Filtering Options | In Review | #1674 | #1749 |
| Docker Borgmatic | Remove antiquated msmtp and ntfy images | Merged | #130 #144 | #240 |
My contributions prior to acceptance into this program can be found in my proposal which is also linked above.
Here is a summary of each of these tasks.
Integration Tests for Borg
This task involved improving testing for Vorta by implementing live integration tests using multiple Borg versions. The existing tests relied on mocked output for a single borg version. Although these tests are fine to test the functionality of various internal features and functions, there is no validation that Vorta works as expected when interacting with Borg. Different Borg versions can produce varied outputs or behave differently. So relying solely on mocked data leaves a gap.
To address this, I wrote new tests, some from scratch while some adapted logic of existing tests. I utilized nox to automate creation of test environment and execution of tests across multiple Borg versions.
I began working on this task by discussing the method to implement integration tests with my mentor on this program. He was the one who suggested nox to me. After testing the functionality of this tool, I worked on writing fixture for pytest to setup the borg repository with different file types while handling different borg versions and operating systems. This was a very important phase for this task, as having robust fixtures enabled me to focus on the test logic itself. Here’s the first commited code:
@nox.parametrize("borgbackup", ["1.2.4"])
def run_tests(session, borgbackup):
session.install(f"borgbackup=={borgbackup}"
cli_version = session.run("borg", "--version", silent=True).strip()
cli_version = re.search(r"borg (\S+)", cli_version).group(1)
python_version = session.run("python", "-c", "import borg; print(borg.__version__)", silent=True).strip()
session.log(f"CLI version: {cli_version}")
session.log(f"Python version: {python_version}")
assert cli_version == borgbackup
assert python_version == borgbackupNext, I moved on to authoring the integration tests themselves. This was the most critical and time-intensive portion. I needed to validate key functionality like archiving, extracting, mounting archives, and more against Borg. These tests provide strong validation that Vorta works correctly with various Borg versions.
The last phase involved automating everything through Github Actions which involved setting up test environment with all dependencies and creating job matrix to run nox for a combination of python and nox versions.
This was the most challenging and rewarding task during this program in terms of learning. It made me more proficient and confident in test-driven development and that’s invaluable.
Remove antiquated msmtp and ntfy images
For this task, I worked on the docker image for borgmatic (which is a very powerful backup software utilizing Borg and I personally use it a lot for servers). It primary included refactoring and extending the README file along with container entry script and discontinuing two old versions of the image as their functionality has been merged to base image with a new library, apprise.
I’ve built container images before but working on this taught me best practices around optimizing images for efficiency mentored by an experienced developer. Updating the documentation also made me think about clarity and maintainability.
I was also able to use this experience to create image for a small personal project, ResolveHost, and automate deployment through docker by implementing continous deployment.
Diff and Extract View Filtering Options
This was the last task in order of implementation and I was really looking forward to this. I added a search input in Diff and Extract Dialogs for archives. With large number of items in a repository, selected extraction and comparison of differences between archives become tedious. We aimed to add a search options through which users can easily find required files and folders with simple but powerful search syntax.
A fun part of this task was developing the syntax to allow various search options. I initially proposed a dropdown to allow selecting the type of search user wanted to perform (name, size etc) with a string input box. There were also several good recommendation from the community on the issue page. I later realized the flaws and issues with dropdown approach and we settled on search syntax.
I decided on using unix-like command syntax and passing parameters. I found it intuitive and easily comprehensible. My mentor also made me aware of the concept of user stories which was honestly a mind opener and very helpful during the implementation stage of syntax.
For implementation, I utilized argparse which is a primarily for handling command line options but it was surprisingly easy to use this for custom input.
What’s next?
The completion of my GSoC project marks an important milestone, but it is not the end of my involvement with Borg Collective or open source development.
For now, I plan to continue with my contributions and help fix bugs, improve documentations as my schedule allows. I’d also love to help out new contributors getting started with the project and/or getting familiar with the codebase. I also plan to continue honing my skills as an open source developer and potentially apply to GSoC again next year or guide new contributors.
Acknowledgement
I would like to express my deepest gratitude to my mentors yfprojects and Manu for their invaluable guidance, feedback and constant support throughout my Google Summer of Code project. yfprojects constantly reviewed my code, suggested improvement and overall helped me throughout the program. I couldn’t have asked for a better mentor. I learned a lot about Docker from Manu who also helped me write Documentation for Borgmatic Docker image.
I would also like to thank Dan Helfman for helping me with borgmatic contributions and his invaluable inputs and suggestions in writing my GSoC Proposal.
I’ve had a wonderful experience during this program and hope to continue contributing to the project in the future. My thanks to Google for organizing GSoC 2023.