My article “XML, API, CSV, SOAP! Understanding the Alphabet Soup of Data Exchange” is up at idealware.org.  This is intended as a primer for those of you trying to make sense of all of this talk about Application Programming Interfaces (APIs) and data integration. It discusses, with examples, the practical application of some of the acronyms, and suggests some recommended practices around data system selection and deployment.  Credit has to go to Laura Quinn, webmaster at Idealware, who really co-wrote the article with me, but didn’t take much credit, and our reviewers,  Paul Hagan, Steve Anderson and Stephen Backman, who added great insights to a pretty heady topic.

The article went through a lot of rewrites, and we had to cut out a fair amount in order to turn it into something cohesive, so I hope to blog a bit on some of the worthwhile omissions soon, but my day job at Earthjustice has been keeping me pretty busy.

Day 2 of the Salesforce Non-Profit Roadmap session was focused on refining plans and sharing information. We had sessions and reports from Salesforce Product managers and developers, and we discussed and demoed some of the creative things that our community has developed. The Salesforce guests showed off Apex, the new scripting language that will be available for live use sometime next year; and we had a fascinating (but non-discloseable!) peek at where the reporting is going.

A lot of the talk focused on ways that we can—or will be able—to get around Salesforce’s core assumption that we deal with companies and contacts when, in fact, donation management is about individuals and households. And a big topic was integration, with a lot of questions centered on what can or should be done in Salesforce and what should be programmed on top of it. Two technologies that popped up a lot were Facebook and Ruby on Rails. I learned about (and immediately grabbed) a Salesforce library that has been developed for rails, and Alan Benamer sang the praises of Facebook both as a compelling social network and a fundraising tool, via their new “Causes” feature. Facebook has been in the news for opening up a powerful API, which makes them pretty much the “Salesforce of Social Networks”.

In the afternoon, we got to th fun stuff – showing off what we’ve done. Six of the participant’s showed off projects big and small.

Ben Munat showed us ChipIn, a fundraising widget that currently is available as a wep page plug in, but will soon be integrated with Salesforce, Facebook, and other application platforms.

• Sonny Cloward showed us a very clean and elegant Salesforce template for fund development created using Salesforce’s Person object. The Person object, which can be used in lieu of Accounts and Contacts, was introduced late last year to a somewhat underwhelming response, the problem being that it’s an either/or choice. If you use Person objects, you can’t use Accounts and Contacts, and, in most cases, you have both companies and individuals among your constituents. All the same, Sonny’s template transformed Salesforce into a clean and simple CRM that would be far easier to teach and support, and maybe quite suitable for small organizations.


• Rem Hoffman demoed the very sophisticated case management system that his company, Exponent Partners, has put together. This was a real ooh and aaher, as he demoed how a Mental Health agency, swamped in paper, could use it to track cases and print all of the paperwork with about a quarter of the effort that had been required. I’m very intrigued by Rem’s work, as I believe that case management options in the workforce development industry are all pretty painful. As far as I know, Social Solutions is the only company talking about opening up their application; most are the worst examples of grabbing a company’s data and locking them out of it.


• Ryan Ozimak of PicNet demoed his Joomla/Salesforce integration, which is also very cool and clean, and promising. At present is is likely the fastest and easiest way to develop a web site with Salesforce Contact integration, and the next steps will open up other objects for clean integration. Ryan (who is sitting next to me as I type) has just let me know that this is around the corner.


• As usual, Steve Anderson of One/Northwest had an amazing demo, showing how he has developed Apex code that completely masks the Account/Contact model so that a user can easily add and remove individuals from households. This was very slick, as his automation made tasks that take multiple screen views and actions today and almost magically integrated them. For example, if you have the household of John Doe and the house hold of Jane Doe, and you want to combine them, then you add Jane Doe to John Doe’s household and – poof! – the household is automatically renamed to “John and Jane Doe” and Jane Doe’s household is deleted. This completely removes the limitation that use of Person accounts involves – you can still have accounts and contacts. The problem being that Apex is only available in the sandbox for now.


• Finally, Evan Callahan of NPower Seattle demoed a simple translator lookup app that he created for a client. What was cool about this was both that he put together a very intuitive and functional tool for finding a translator with the proper skills and availability, and he did it with some very simple code and a web form. In both Steve and Evan’s cases, they took innovative and undocumented approaches that produced powerful results. Must be something in that moist Seattle air.

Today we dive into how the Salesforce community can better operate as a cohesive support infrastructure and wrap up at noon. If you are a Salesforce license donee, keep your eyes open for a survey that will let you in on this critical input. And look for a bigger event next year—this was a great exercise for all parties.

Day one of the Salesforce Roadmap session was a well-crafted, but fairly standard run at typical strategic planning. Hosted by Aspiration’s ever-able Gunner (who I seem to run into everywhere lately), we had a group of about 40 people: five or six from Salesforce/Salesforce Foundation, five to six NP staff, and an assortment of Salesforce consultants. While I’m a consultant these days, I maintain a bit of a staff perspective, as my primary experience with Salesforce was to roll it out for SF Goodwill. The day consisted of breaking up into small teams and hammering out what works for our sector, what doesn’t, what could be done, and building all of this into a set of possible roadmaps that would address non-profit needs. The most striking thing about the outcome was that we had six groups design those roadmaps, and we largely all came up with the exact same things.

So, what are they?

Templates. In 2005, Salesforce developed a template for non-profits that everyone admits was pretty lame. Most of the consultants advised against using it. In 2006, Tucker MacLean, at the time a Fellow with the Foundation, redesigned it into something far more substantial – but still problematic, the problem being that non-profits are far too diverse in their structure and needs to fit a single template. The template in place transforms Salesforce into a donation management application. But I would argue that deploying Salesforce strictly as a fund development tool is short-sighted, and possibly disadvantageous when there are so many choices for software that is developed to that purpose, not twisted to it. The reason to deploy Salesforce is because it can handle the fund development and do so much more.

So, roadmap 1 is to move away from the one-size-fits-all template to something far more modular.

Road map 2 is around the community, or eco-system that supports the non-profit Salesforce adopters. And I think this is where the most meaningful changes can occur. This is about shared development—should NP Salesforce have an Appexchange of its own, one that acts more like Sourceforge? Can the consultant community adopt standards for how we deploy, and can Salesforce support us in any innovative ways? And can best practice, case studies, and non-profit specific training and documentation be collected in one place?

Third was the product itself, which I really don’t think non-profits can or should influence all that heavily. I don’t believe that our platform issues are unique. But we do want to see that new things (document management, Google Apps integration); we would really appreciate a customer portal and stronger ties to CMS’s and web sites, and stronger integration with our external applications.

What interests me is the dual need for this very open, malleable platform and the dire need non-profits have for out of the box functionality. Currently, Salesforce is a very worthwhile investment, but it’s not a light investment for a tech and cash strapped organization. The integrators working with it are frustrated by how much programming they have to do to support some very basic functionality.

But it says worlds that Salesforce is approaching this by inviting the community to advise them. This somewhat techy gathering will be followed up by a survey for the non-profit users at large. Ask yourself, how often does a large, corporate software company ask you directly to give input into their development? Or, if they do, do you think they actually listen? Once again, Salesforce is modeling an approach to doing business that has far more in common with the open source world than the for-profit. More on this later.

I’m attending a strategic planning session at Salesforce.com this week devoted to planning the roadmap for non-profit use of the product. This should be an interesting event and an exciting opportunity to help steer one of the most exciting applications to hit the industry in some time. I remember walking through the exhibitor booth’s at the “Science Fair” during the 2005 NTEN Conference in Chicago and noting, in the corner, the guy with a shaved head standing at a small booth titled “Salesforce.com” and wondering what, on earth, he was doing there. Wasn’t Salesforce that corporate application used by all those people trying to sell me enterprise software? The next year, in Seattle, Salesforce was a key sponsor of the show, and the whole gang from the foundation was there. I was a lot more educated as to why, as well – in the interim, my former organization had signed up and I had started work deploying it.

Salesforce appeals to me because it lives up to many of the standards I look for in an online database:

• It’s open. Any Salesforce customer can download their entire database into Excel pretty much at any time. There are no technical or contractual walls separating me from my information as a Salesforce customer.


• It has a community around it extending, developing and integrating the product. While Salesforce is far from the only commercial application with such a community, it is far more analogous to the open source communities around applications like Joomla and Drupal than it is like their commercial counterparts. Salesforce has provided excellent forums and support, nurturing their partners in ways that most commercial developers are far too guarded to allow.


• Sharing and philanthropy are part of the corporate ethic, fairly deeply ingrained. I like to joke that their stated policy of “one percent of people, product and profits goes back to the community” is not that big a deal, given that 100% of a non-profit’s revenues are recycled back into their missions, but the truth is that they do a lot more than just give away software, and I’m certain that it ends up being much more than 1%.


• Salesforce is audacious and ambitious in all the right ways. They want to do away with your infrastructure and change the way that technology is deployed, and they are by far the most sophisticated example of how that can and should be done. And don’t ever mistake them for a CRM company just because that’s what they’ve primarily been – they’re a shard data and computing platform, and the next few years are going to see them break out of the CRM neighborhood into a new role as a data management middleware provider. Store your data and build your processes, they’ll handle the hardware.

Finally, in this era, when internet business is shaking up traditional business models in dramatic fashions—just ask the RIAA, or the telecoms, or your local newspaper’s classifieds editor—Salesforce is the disruptor in our community. Blackbaud, Kintera and Convio, along with the other established donation-based business support vendors, are all rapidly changing their models to more closely match the open approach. And Social Solutions and the case management crowd are well aware that they’re next. This bodes well for the customers.

I’ll be blogging from the conference (as allowed) and hope to spread exciting news.

So, I came to this Rails conference looking for a few things. It’s not over, but I think I’ve got a good sense what I’ll walk away with tomorrow.

I started to learn a bit about Rails while considering joining a software start-up (in the non-profit space). I spent a month hammering away with a few O’Reilly books and a sample project, then got pulled away by real world concerns like starting up my new career fast so my family won’t starve. I got far enough to get the concepts and philosophy, master the innovative database management (activerecord), and start an app that I plan to finish and publish as part of Techcafeteria someday. Along the way, I loved the rapid development features and recognized Rails as a bit of a conceptual leap in programming/scripting, that values efficiency of following conventions over coding. Being oriented toward finding the fastest paths to the best results, I was also intrigued by how Rails builds Ajax functionality into the code (I just never bothered to get beyond the basics of Javascript, preferring server-side programming, I bias I now regret…) But I also grew concerned about the platforms speed and scalability, concerns that my friends at Social Source Commons (SSC) would second, I suspect.

So, the four areas that the conference could have helped me with, and how it did:

1. Learning more of the scripting language. Not so much—maybe a referral to the book I’m missing that will glide me right over that hump.


2. Ajax intro – pretty good. I attended a few sessions on Prototype and Scriptaculous that gave me a far better handle on how they work .


3. Ruby Scaling—an awesome session on the proxy cache and other options out there to speed up Rails, with pointers to what bottlenecks it. This was likely the most valuable thing, and I’ll be contacting Gunner to offer to take a look at the SSC platform and see if we can apply some of what I learned.


4. Where it’s going, as I reported on yesterday. Among web scripting languages, PHP and ASP/.NET are the kings today. My prediction is that Ruby on Rails will eclipse them, and gain broad adoption among web 2.0 developers and corporations looking for in-house app development tools. The main limitation – performance – is being addressed and will be fixed, no question.

The benefit of having a functional application roughly 60 seconds after you think of a name for it is phenomenal, and the developers are completely geared toward continuing to make it the out of the box solution for speedy delivery of standards-based, current tech web applications.

Day 2 at the Ruby on Rails conference – after the Keynote.

My main focus is on technology trends that allow us all to make better use of the vast amounts of information that we store in myriad locations and formats across diverse systems. The new standards for database manipulation (SQL); data interchange (XML) and data delivery (RSS) are huge developments in an industry that has traditionally offered hundreds different ways of managing, exporting and delivering data, none of which worked particularly well—if at all—with anybody else’s method. The technology industry has tried to address this with one size fits all options—Oracle, SAP, etc., offering Enterprise Resource Platforms that should be all things to all people. But these are expensive options that require a stable of high-paid programmers on hand to develop. I strongly advocate that we don’t need to have all of our software on one platform, but that all data management systems have to support standardized methods of exchanging information. I boil it all down to this:

It’s your data. Data systems should not restrict you from doing what you want to do with your data, and they should offer powerful and easy methods of accessing the data. You can google the world for free. You shouldn’t have to pay to access your own donor information in meaningful ways.

How can the software developers do this? By including open Application Programming Interfaces (APIs) that support web standards.

So what does this have to do with Ruby on Rails? At the Keynote this morning, David Heinemeier Hannson showed us the improvements coming up in Ruby for Rails 2.0. And he started with a real world example: an address book. Bear with me.

1. He created the project (one line entered at a command prompt).


2. He created the database (another line)


3. He used Rails’ scaffolding feature to create some preliminary HTML and code for working with his address book (one more line).


4. He added a couple of people to the address book.

At this point, with a line or so of code, he was able to produce HTML, XML, RSS and CSV outputs of his data. The new scaffolding in 2.0 automatically builds the API. I could get a lot more geeky about the myriad ways that Ruby on Rails basically insures that your application will be, out of the box, open, but I think that says it well.

Think of what this means to the average small business or non-profit:

• You need a database to track, say, web site members, and you want to further integrate that with your CRM system. With rails, you can, very quickly, create a database; generate (via scaffolding) the input forms; easily export all data to CSV or XML, either of which can be imported into a decent CRM.


• You want to offer newsfeeds on your web site. Create the simple database in Rails. Generate the basic input forms. Give access to the forms to the news editors. Export the news to RSS files on your web server.

This is powerful stuff, and, as I said, an instant API, meaning that it can meet all sorts of data management needs, and even act as an intermediary between incompatible systems. I still have some reservations about Rails as a full-fledged application-development environment, mostly because it’s performance is slow, and, while the keynote mentioned some things that will address speed in 2.0, notably a smart method of combing and compressing CSS and Javascript code, I didn’t hear anything that dramatically addresses that problem. But, as a platform, it’s great to see how it makes actively including data management standards a native output of any project, as opposed to something that the developer must decide whether or not to do. And, as a tool, it might have a real home as a mediator in our data integration disputes.

It’s Thursday morning, and I’m in Portland, Oregon at the 2007 O’Reilly Railsconf, all about the web programming language/environment/framework called Ruby on Rails. I was introduced to Ruby on Rails by a friend/associate who I hope to be doing some work with soon – we’re part of a group looking for funding to develop some applications. I program in a few languages, mostly PHP, but agreed to learn Ruby on Rails after being introduced to it.

Ruby on Rails, it turns out, is a controversial language, in a way that is very reminiscent of the Apple vs. everything else debate. Rails enthusiasts are very attached to the platform, and adherents of Java, C, and even PHP, tend to be very skeptical, with complaints that the structure is too rigid and that the language only goes so far. They might be right – I’m not fluent enough yet to know. But there are a few definite things that have me interested in Rails.

1. Rails abstrats the database creation and management process in a really fascinating way. Using the MVC framework—model, views, controller—you basically develop your database using plain english to describe the relationships between tables. This really works for me. To create the database, you write some very simple code that adheres to certain naming conventions, and then you can manage the database almost exclusively from the code.


2. Once the database is created, Rails uses a method called scaffolding to automatically create forms for database manipulation. With one line of code in your controller, you can very simply grab data from multiple tables using a simple syntax. Rails makes it all very, very easy.


3. I’m looking for a holy grail, of sorts, something that falls halfway between a programming language and a content management system (CMS), and this comes close. What can we use to rapidly develop interactive, web-based applications that doesn’t lock us into the type of assumptions that Drupal and (the current version of) Joomla do, but don’t require building the whole thing from scratch? Ruby on Rails is still a pretty complex thing for most techs at non-profits to budget the time to learn, but it’s intriguing, as is the move in the next release of Joomla to have it sit atop a Ruby on Rails-like framework (that, unfortunately, lacks the database routines).

I’m also looking at Javascript/ajax libraries – I’m in one right now on Prototype and scriptaculous, but the presenter is the developer of scriptalicious and his presentation style is somewhat coma-inducing…

Just to put this all together, I’ve written a F.A.Q. and a How-To on OpenID and added them to the OpenID offerings here at Techcafeteria which are, in a nutshell:

1. The OpenID-enabled Blog;


2. The OpenID server, which I’m committed to maintaining. Techcafeteria won’t be going away anytime soon!;


3. A new OpenID F.A.Q., which links to other OpenID resources;


4. and a new OpenID illustrated How-to, which uses the Techcafeteria server as an example but overviews how they all work.

Why am I harping on about this? I really do think that OpenID offers a solution to a very pesky problem. I have an encrypted file with all of the logins and passwords that I keep on a regular basis for web sites and services that I use. There are over 200 of them. I might be an extreme case, but I’m far from alone. And, from my years as a technology manager, I know that most people solve this problem by using the same password at multiple sites. So if those sites include your online banking, that’s a serious risk.

But, beyond the convenience and security, I look at it this way. My goal for Techcafeteria is to grow it into a real diverse offering of web-based services, in fitting with the name. Some of these, like the blog, will be based on third-party platforms, others will be things that I develop (I’m experienced with PHP/MySQL and I’m learning Ruby on Rails – I’m even attending O’Reilly’s big conference on it in Portland this week). My goal is single sign-on, via OpenId, for everything that Techcafeteria ever offers.

It’s not a big deal doing this on my web site. It would have been a huge deal if I could have accomplished it at the large non-profit or decent sized law fIrm that I served as an IT Director for. At both of those jobs, we had a variety of systems, all tied into Novell and/or MS networks, but we still had nothing but password soup to offer our users, because the apps weren’t standardized enough to allow for true single sign-on.

At Joomla Day on Saturday, I sat in on a session where one of the core developers (Sam) demonstrated a way to share authentication between Joomla and MediaWiki. Very cool, but somewhat easy because MediaWiki stores the password unencrypted. Assuming that most sites use standardized encryption protocols (MD5 being the big dog, that’s not an insurmountable challenge. But I couldn’t help thinking how much easier this will be via OpenID. It’s not just about this stuff being possible – it’s also about allowing Sysadmins who are not also programmers to implement it.

So, end of OpenID rants, for now. I’ll be doing some live blogging from the Rails conference, and I’ll try and include some context as to why I think Ruby on Rails is an important programming environment.

I’m posting this live from the first Joomla Day West conference being held at Google headquarters in Mountainview (so, yes, wireless is reliable!)

This is an interesting event – an “un-conference” as Ryan calls it, which falls somewhere in the territory of a traditional conference, a town hall meeting, and, maybe, the Phil Donahue show, as emceed by the always entertaining Gunner (of Aspiration fame).

It’s about halfway through the day, and continuing through tomorrow, but I won’t be able to come back, because that would incur the justified scorn of my son’s mother, who expects me to not be a computer geek on her day. There are 100 or so people here from many corners of the earth (well, the Americas and Europe are healthfully represented) and associations to Joomla that range from a tiny non-profit thinking about using it to the core development team. Joomla, for those who don’t know, is a popular open Source Content Management System (CMS) with a huge developer community, making it very powerful and popular. It has it’s roots in a CMS called Mambo.

The big topics are:

• The upcoming Joomla 1.5 release, which is a dramatic rewrite of the application that will make developers (like me) very happy. They have exposed a programming framework that could develop into an environment all it’s own, and they’ve made changes to the templating that allow for powerful customizations.


• The move to more strictly enforce GPL compliance. The GNU General Public License is designed to offer users of GPL applications much freedom,with restrictions on how the code can be redistributed that insure that the community will share in all enhancements. The Mambo/Joomla developer community apparently includes many add-ons that aren’t compliant with this, and the Joomla team hopes to (appropriately) bring them back to compliance.

This is a seriously fun event with group activities intersperced with break out sessions, and a kind of “this is being made up as it goes along” agenda. Next up: speed geeking! which Gunner describes as “like speed dating, but completely different”.

A quick addendum to my last entry:

First, my apologies if you’re trying to play. For some reason, the DNS change that will allow you to access openid.techcafeteria.com is taking a looonng time to propagate. I’ve asked my ISP about this. And it makes no sense to give you the ip or an alternate name – you need the actual name to get this working.

Don’t trust me to maintain techcafeteria.com 24/7 for as long as you may live? Good thinking! I’m hosting this on my home box, because I can’t hack PHP sufficiently in order to get it going on my ISPs system. So this is what’s cool about OpenID. It’s relatively easy to become an OpenID provider, if you have your own server. I think it took me two hours or so to get it all set up. So there will be plenty of providers out there. And OpenID gives you an option for setting up a permanent address on any server where you can create a simple page (regardless of whether it’s your system or if it has anything related to OpenID installed) and then referring it to your OpenID provider. So, if I take my system down (I do that about twice a year), you can register somewhere else and simply point your URL to their system. It’s very flexible, and you’ll have the instructions in front of you after you create your ID on my server.

In addition to OpenID.net here are two important resources:

OpenID Enabled is a wiki devoted to OpenID. Very thorough!

The OpenID Directory is an early stab at collecting all of the sites that allow you to log in via OpenID. It’s also an OpenID provider, if you’re looking for that backup.