Public Patch

Alright so for this project I found out that I needed to submit a public patch along with my GSoC application. Needless to say, I freaked out and went digging through the Bugs page of the Mailman page on Launchpad trying to find something that I could do without having to figure out the intricacies of the software. Yep, it didn't work. Thankfully Barry had mercy on me and gave me a project that involved adding support for deleting domains to the REST API.

Of course this discovery came in the middle of the last few weeks of my freshman year at College of Charleston, so I was busy beyond belief and was desperately looking for random time slots in which I could work on this project. I've never worked on open source software before, so the size of the source code was daunting and I began to struggle as I worked through several files line by line trying to figure out how everything works. After looking at lists.py for comparison, I was unsure as to whether I should be writing an entire module to support deleting of domains or writing a method in IDomainManager to perform the action. Barry came to the rescue again and led me to realize that the method I had (correctly!) intended to implement was already implemented by a model class...and that I had been looking at the interface definition for a DomainManager without realizing. Go figure.

Finally I got my hands on the relevant files to the project and after about an hour and a half, I had written the extension to the REST API that allowed for deleting domains through HTTP DELETE connections. After pestering Barry again, I found out that Mailman is developed using Test Driven Development philosophy; something I only knew about in theory. Now I knew that as soon as I figured it out I'd take it and run with it, so I jumped right into writing tests and, you guessed it, failed horribly. I really, really don't like it when I don't understand CS concepts, so over a span of about three days (random intervals due to school work) I worked to figure out how to build and run tests. I was confident that my delete method in domains.py would work, so I began to get incredibly frustrated that I couldn't get the tests to pass.

This morning (April 25th), I gave in and ping'd Barry again on IRC and with his patient guidance and my brute determination, I managed to push what I hope is a correct solution to the problem via Launchpad.

This whole process was a fantastic way for me to get to work with the software and become familiar with at least part of the functionality. It also gave me a chance to work with test driven development and Bazaar for version control (again, something I only knew about in theory) using tutorial I found to push my branch to Launchpad.

I'm pumped to get started on Mailman for the summer, it's going to be cool to dig into the documentation and figure out how everything works so that I can carry out my proposal as planned. In the meantime, I'm finishing up a little bit of development on my iPhone application before the artwork gets finished so I can push it to the App Store. It's going to be busy summer!

Project Proposal Version One

Mailman 3 Pipermail SQL Implementation and Mailman 2 Archive Uprade Script.

Mailman 2's Pipermail archiver is currently built on a persistence layer of
Python pickles, which is memory and performance inefficient.  The end result
of this project will be to rewrite the Mailman 3 Pipermail implementation to
be backed by SQL, using the Storm (storm.canoincal.com) ORM.  This will be
achieved by mapping the MM2 pickle schema to a new MM3 SQL schema, along with
an upgrade script to convert existing Pipermail archives to the new format.
This upgrade script could be run standalone, or as part of a general MM2->MM3
upgrade procedure.  APIs will be added and updated as needed to support both
the on-the-fly addition of new messages, as well as full archive
regeneration.  Upon completion of this objective, I will work on the Stable

URL project proposal by using the Message-ID hashes as keys for generating

stable URLs to the entries in the database.  As an added feature, a mapping
will be kept for the old URLs so that they can be redirected to the new stable
URLs, ensuring that current links will not be broken.

Milestones

These are initial goals, students may work with their mentor over the Summer to update these goals as needed. 

Start of Program (May 24)

My goal before I start is to spend time from now until the start of the actual coding to figuring out how the entire Mailman system works and how my Pipermail implementation needs to fit into the system. 

Midterm Evaluation (July 12)

Deliverables:

- Nearly complete if not fully working implementation of Pipermail using the Storm ORM API

- Nearly complete if not fully working upgrade script to move through the Mailman 2 archives and convert the Python Pickle Data to a SQL database using Storm

Final Evaluation (Aug 16)

List your proposed deliverables by the end of the Summer:

- Fully working implementation of the Pipermail backend using the Storm ORM API

-  Fully working upgrade script to convert Mailman 2 Pipermail archives from Pickle scheme to the Mailman 3 SQL scheme

- Stable URL generation for entries in the Mm3 SQL database using the Message-ID hashes as keys with support for mapping old URLs to the new stable ones