I just had a very satisfying free software experience!

Over the last few months, my non-profit organization (QuestionCopyright.org) has been having the sort of problem any non-profit would want: we’ve been getting lots of donations from many sources. The majority are directed donations to the Sita Distribution Project, so the money is used only there and not for general operating expenses. But the bookkeeping overhead is the same either way: the money has to actually get into the correct bank account somehow, a tax receipt has to go out to the donor (at least for donations over a certain amount), and both the donor and this donation need to be entered into our records.

That last part is more of a challenge than you might think. We started out with a manual process for recording donations: every time we got a payment (by Google Checkout, PayPal, check, or bank transfer), I’d create a new text file (based on a template) in a version-controlled directory, recording the details of the transaction. It was a poor substitute for a database — grep was our query mechanism :-). But it worked well enough, as long as there were only a few donations to process in any given week.

However, as the project got under way, the donation rate quickly rose while the average amount of each donation went down. In other words, we were getting a lot more donations, for smaller amounts. Pretty soon it was up to several a day, sometimes more than ten, with most of them coming in through online payment processors: Google Checkout or PayPal. My manual process for transferring information from the Google Checkout and PayPal web pages into text files quickly became totally untenable. If I spent the whole day processing donations, I might be able to keep up, but that’s hardly a good use of time.

Enter CiviCRM…

I’d heard about CiviCRM for a while, but never used it before. It’s a free software, web-based membership and donation management system designed for non-profits. It fits into popular content management systems like Drupal, which happens to be what QuestionCopyright.org uses, so on the recommendation of Donald Lobo — about whom more in a moment — I installed it and started poking around.

Within about ten minutes I knew I had the right tool for the job. It wasn’t the CiviCRM user interface that convinced me (though the interface is good enough that I could find my way around before reading any documentation), but rather the fact that the data entities were organized in the right way. For example: like most non-profits, we get repeat donations, and therefore want to organize things around the donor rather than the donation. In database-speak, donors stand to donations in a one-to-many relationship. While CiviCRM keeps track of each donation individually, donations are always attached to a donor, and the donor is the “primary” object — which makes sense, because the donor is usually much more interesting to the organization than any individual donation is. This and other things convinced me that in CiviCRM, our information would be organized in a sane way.

But even though I could see that CiviCRM would solve our problems in the long term, we still had an immediate, pressing need: how to get at the donations still pending in the Google Checkout and PayPal queues? By this time there were quite a few, as I’d had to stop doing manual processing while searching for an automated solution, just due to time constraints. As far as QuestionCopyright.org is concerned, those donations don’t exist until we process them. If we were going to switch to CiviCRM (which I very much wanted to do), then we had to have a way to get those donations into the system, and to continue pulling future donations from those sources in some semi-automated way.

It turns out that Donald Lobo, the same fellow who had pointed me to CiviCRM in the first place, works on CiviCRM full-time (CiviCRM development is supported by the non-profit Social Source Foundation) and had wanted to solve this problem anyway. When he heard that QuestionCopyright.org needed the feature, he wrote the ContributionProcessor.php script to solve the problem, and then — I’m not making this up — stayed up with me several nights in a row, after putting his kids to bed, to test and debug the script in real time so he could be sure it really solved the problem. He was in California, I was in New York, so we would get into the same IRC channel and walk through data pulls, tweaking the script until we saw that CiviCRM was pulling all the data it should be. The script works quite reliably now, and should be available in an upcoming release of CiviCRM.

I now run the script every week or so, once for Google Checkout and once for PayPal. By “run the script”, I just mean hit the script’s URL along with the correct parameters. Something like this:

   http://questioncopyright.org/\
   sites/all/modules/civicrm/bin/ContributionProcessor.php?\
   type=google&ppID=3&ppMode=live&start=30&end=1\
   &name=civicrm_api&pass=XXXXXXXX&key=YYYYYYYYYY

or this:

   http://questioncopyright.org/\
   sites/all/modules/civicrm/bin/ContributionProcessor.php?\
   type=paypal&ppID=5&ppMode=live&start=30&end=1\
   &name=civicrm_api&pass=XXXXXXXX&key=YYYYYYYYYY

In case you’re thinking of trying this yourself:

• The type field is google or paypal.
• The ppID is the CiviCRM Payment Processor ID associated with that online service. The Payment Processor is an entity you create in the CiviContribute section of CiviCRM (when you create each payment processor, you’ll tell it the appropriate login information for talking to the service it’s supposed to talk to). Although the IDs of the two processors I created happen to be 3 and 5, they could be different numbers for you.
• I typically set start and end to 30 days ago and 1 day ago. Even though I run it weekly, that’s okay — CiviCRM won’t double-process any donations; it detects duplicates and skips them. You’ll see the “skipped” notices in the script’s output.
• The name field is for the name of the Drupal user whom the script should run as. Typically one makes up a role user; I called it civicrm_api, which seems like a reasonable default.
• The pass is the Drupal password for that user.
• The key is the CIVICRM_SITE_KEY, usually specified in the file .../sites/default/civicrm.settings.php, like so: "define( 'CIVICRM_SITE_KEY', 'YYYYYYYYYY' );".

When we get the occasional donation by other means (check or bank transfer) I simply log into CiviCRM, create a new donor record, and then create a donation attached to that donor. After that, it looks just like any other kind of donation, and we can deal with it in the system the same way we deal with all donations.

Thanks to CiviCRM (and to Donald), a great weight has been lifted off my shoulders. We’re not dropping any donations on the floor, and I can start to explore CiviCRM’s other features. Next on my list: learning how to send out automatic acknowledgement emails (with tax information), and how to manage “campaigns” (as I think they’re known in CiviCRM) to make sure that that directed donations are recorded as such in a more formal way than the notes we’re currently attaching to them.

In the meantime, if you’re responsible for IT at a non-profit that doesn’t yet have a donor information management system, I strongly suggest checking out CiviCRM. Support is available, both from the volunteer community and from professional consultants.