August 13 2014

Hackcess To Justice

Hackcess to Justice LogoRegular blog readers know that landing my job at Legal Services Corporation, the single largest funder of civil legal aid to people in financial need, was not an accident.  The mission of providing representation to those who need it, but can’t afford it, is one that I targeted for over half a decade before getting this position. I’m passionate about the work of our grantees, because there is something about social and economic injustice that offends me at my core, and I consider it my responsibility and my privilege to be able to do work that attempts to alleviate such injustice.   That’s my best explanation, but you should hear my boss describe the problem. As a lawyer, he makes the case, but he makes it in plain, clear English, and he makes it powerfully.

The setting of this 13 and a half minute speech is completely appropriate for this blog.  The first “Hackcess to Justice” hackathon was held in Boston on August 7th and 8th at the annual American Bar Association meeting. The goal of the hackathon was to create apps that address the needs of people seeking representation or representing themselves in civil courts. The projects are based on the recommendations that came out of the technology summit that LSC held in 2012 and 2013.  The report, linked here, is a good — and not too lengthy — read. And the winners creatively met those goals, with apps that help write wills, determine whether you need legal help, and point you to legal resources in a disaster. Robert Ambrogi, one of the three judges, blogged about the winners, too.

I’m proud to work for an organization that not only thinks strategically about how we use technology, but strategize about how the world can use it to address the problems that we were founded to help solve. And I’m very happy to work at a company where the leadership gets it — technology is more than just plumbing; it’s an enabler.  Deployed correctly, it can facilitate solutions to extremely challenging problems, such as the severe justice gap in the United States.

Jim says it best, and I can’t recommend enough that you take the quarter of an hour (less, actually) to hear what he has to say.

Jim Sandman’s Opening Remarks at the Hackcess To Justice Hackathon, August 7th, 2014

Category: Access to Justice, NPTech, Web | Comments Off
January 26 2014

Making Your Website More Useful For More People

This post was originally published on the LSC Technology Blog in January of 2014. LSC is Legal Services Corporation, my employer.

At LSC, we’ve been taking a critical look at our web site, to see if we can make it a more useful web site by factoring in all of the ways that people might want to view or use our information. In these days of big data and small screens, we realize that we have to be much more attentive to the ways that we present data than we have in the past.

Identifying the different visitors who frequently use our site, we took a closer look at their needs, and how we could improve our delivery of information to them. For example, visitors to LSC’s web site could be:

  • reporters or Hill staffers looking for a quick cut and paste of data on the site that is hard to get out of a linked PDF;
  • general public looking for data to pull into a spreadsheet, who would also be disappointed to find that data in a PDF;
  • visually or physically impaired, and therefore not able to view web content that isn’t compliant with the standards that their specialized software requires;
  • accessing the site on a mobile device that doesn’t display flash or video and has no capability to display a PDF

The PDF Problem

Adobe has done great things with the Portable Document Format, opening it up as a public standard and continually improving the functionality of the format. But this is not an optimal format for web-based content, because PDFs require additional software in order to be viewed, and they need to be created with a solid understanding of how PDFs need to be prepared, so that they are compatible with accessibility standards. Our goal is to ensure content is delivered optimally, and in a format that makes it easy to access for anyone and everyone visiting our site.

In the past, we’ve relied heavily on publishing web content via PDF, and we now have a backlog of documents that aren’t as widely usable as we would like. Our plan is to immediately make two changes:

  1. Use PDF sparingly and thoughtfully as we move forward. Use PDFs as optional downloads for content that is also displayed in HTML, or as appropriate downloads for white papers and legal reports that aren’t the types of things that users will want to quote or edit; design PDFs that are compatible with the section 508 standards for web accessibility.
  1. Determine which of our existing PDFs need to be republished in more accessible formats and convert them. We don’t have the resources to fix everything, but we have good statistical data from Google Analytics to tell us which PDFs our visitors look at and a good idea how to prioritize this content.

Open Web

As a nonprofit that allocates federal funds, we have a responsibility to make data available to the public. But a commitment to open data means more than just making the data available; it needs to be available in formats that people can easily use. Data stored in an HTML table can be copied and pasted into Excel. Data in PDF and image formats can’t be, at least, not easily. As David Ottewell recently tweeted, a PDF of a spreadsheet is not a spreadsheet. These efforts dovetail with our broader efforts to make data available in manipulatable formats.

Wild, Wild Web

It is also important that our web site deliver the same user experience on smartphones or a tablets as it would when viewed on desktop or laptop browsers. This wasn’t high on our radar in 2011, when we redesigned our website in the Drupal content management system. At the time, we developed a mobile site as a separate, fractional copy of our main site.

Looking ahead

A  modest revamp of LSC.GOV is planned for second half of 2014 to improve the site navigation and responsiveness on multiple devices (e.g. one site that alters it’s navigational elements and appearance to properly utilize the screen that it’s displayed on). We also won’t forget the visitors that don’t have smart phones and how best to make information available to them.

Having a website that anticipates their diverse needs of our online visitors is our goal. What’s yours? What are your current challenges?

Category: Technology, Web | Comments Off
January 26 2014

Finding Aid To Improve Find Legal Aid

This post was originally published on the LSC Technology Blog in January of 2014. LSC is Legal Services Corporation, my employer.

FLA-example.PNG

Hands down, the most popular feature on LSC’s website is our Find Legal Aid lookup, which directs you to the LSC-funded legal services provider in your service area. I’m happy to announce that we’ve given this lookup a refresh while simplifying its use. But we didn’t do it alone, and the story of how we got this project going is one that I really want to share with our community.

As I’ve blogged about before, our service areas are a unique geography that doesn’t lend itself to easy data integration. This became a problem when we started looking at the possibility of sharing our data with the hacker community, in hopes that they would use it to develop apps that further equal justice goals. Simply put, our territories sometimes run within county and city boundaries, making it difficult to align them to standard geographical data. This also meant that our Find Legal Aid tool was a complicated piece of code that was never entirely accurate (it was right 99.8% of the time, and, otherwise, the people who answered calls could redirect someone to the proper legal services provider).

Our desire was to have Find Legal Aid work the same way that any major retailer’s “Find a Store” lookup would, with no more input required than a zip code. We didn’t have the internal expertise established to do this on our own. So we learned of a group called the DC Legal Hackers, and we introduced ourselves. DC Legal Hackers is one of a number of Legal Hacker groups in the US and Canada. Legal hackers work at the intersection of law and technology, looking for ways to improve public access and address inequities in the system via the web. Access to Justice is one of the areas that they focus on. When the group held their first hackathon, we pitched revamping our lookup as one of the projects. Glenn Rawdon, Jessie Posilkin and I attended the hackathon on a Saturday and assisted where we could. We watched as some brilliant people took the shapefiles that LSNTAP made of the LSC service areas and mashed them up in such a way that, by about 2:00 in the afternoon, we had a working prototype.

It took a bit more time for LSC staff members Peter Larsen, Christina Sanabria and Alex Tucker to take it from prototype to a fully-functional application. We gained a lot more internal expertise in working with mapping technology. It’s important to note, though, that this took time, building the skillset as we completed the application and kept up with other priorities. These projects work best when the deadlines are loose.

We did face some choices. The lookup does not return office addresses or info about branches. We assume that the service providers may prefer to start with telephone screening before directing the public to a particular office location. We are contemplating adding links to online intake systems and statewide web sites relevant to the results. And we’re looking to see if a SMS text-based version of Find Legal Aid might be easy to produce.

We’re grateful to DC Legal hackers for taking us halfway there, and over the programming hump that was beyond us. There’s a great community out there willing to work with us.

Category: Access to Justice, NPTech, Technology, Web | Comments Off
May 31 2013

Everything That You Know About Spam Is Wrong

Image: Vince LambImage: Vince Lamb

At least, if everything you know about it is everything that I knew about it before last week. I attended an NTEN 501TechClub event where Brett Schenker of Salsa Labs spoke on how the large mail services identify Spam emails.  It turns out that my understanding that it was based primarily on keywords, number of links and bulk traits is really out of date.  While every mail service has their own methods, the large ones, like GMail and Yahoo!, are doing big data analysis and establishing sender reputations based on how often their emails are actually opened and/or read. You probably have a sender score, and you want it to be a good one.

Put another way, for every non-profit that is dying to get some reasonable understanding of how many opens and clicks their newsletters are getting, Google could tell you to the click, but they won’t.  What they will do is judge you based on that data.  What this really means is that a strategy of growing your list size could be the most unproductive thing that you could do if the goal is to increase constituent engagement.

As Brett explained (in a pen and paper presentation that I sadly can not link to), if 70% of your subscribers are deleting your emails without opening them, than that could result in huge percentages of your emails going straight to the spam folder.  Accordingly, the quality of your list is far more critical than the volume. Simply put, if you send an email newsletter to 30,000 recipients, and only 1000 open it, your reputation as a trustworthy sender drops.  But if you send it to 5000 people and 3500 of them open it, you’ve more than tripled the engagement without soiling your email reputation.

I know that this goes against the grain of a very established way of thinking.  Percentage of list growth is a simple, treasured metric.  But it’s the wrong one.

Here’s what you should do:

  • Make sure that your list is Opt-In only, and verify every enrollment.
  • Don’t buy big lists and mail to them. Just don’t! Unless you have solid reasons to think the list members will be receptive, you’ll only hurt your sender score.
  • Put your unsubscribe option in big letters at the top of each email
  • Best of all, send out occasional emails asking people if they want to keep receiving your emails and make them click a link if they want to.  If they don’t click it, drop them.
  • Keep the addresses of the unsubscribed; inviting them to reconnect later might be a worthwhile way to re-establish the engagement.

Don’t think for a minute that people who voluntarily signed up for your lists are going to want to stay on them forever.  And don’t assume that their willingness to be dropped from the list indicates that they’ll stop supporting you.

Even better, make sure that the news and blog posts on your web site are easy to subscribe to in RSS.  We all struggle with the mass of information that pushes our important emails below the fold.  Offering alternative, more manageable options to communicate are great, and most smartphones have good RSS readers pre-installed.

One more reason to do this?  Google’s imminent GMail update, which pushes subscriptions out of the inbox into a background tab.  If most people are like me, once the emails are piling up in the low priority, out of site subscriptions tab, they’ll be more likely to be mass deleted.

March 15 2013

Google Made Me Cry

Well, not real tears. But the announcement that Google Reader will no longer be available as of July 1st was personally updating news.  Like many people,  over the last eight years, this application has become as central a part of my online life as email. It is easily the web site that I spend the most time on, likely more than all of the other sites I frequent combined, including Facebook.

What do I do there? Learn. Laugh. Research. Spy. Reminisce. Observe. Ogle. Be outraged. Get motivated. Get inspired. Pinpoint trends. Predict the future.

With a diverse feed of nptech blogs,  traditional news,  entertainment, tech, LinkedIn updates, comic strips and anything else that I could figure out how to subscribe to,  this is the center of my information flow. I read the Washington Post every day,  but I skim the articles because they’re often old news. I don’t have a TV (well, I do have Amazon Prime and Hulu).

And I share the really good stuff.  You might say, “what’s the big deal? You can get news from Twitter and Facebook”  or “There are other feed readers.”

The big deal is that the other feed readers fall in three categories:

  1. Too smart: Fever
  2. Too pretty: Feedly, Pulse
  3. Too beta: Newsblur, TheOldReader
“Smart” readers hide posts that aren’t popular, assuming that I want to know what everyone likes, instead of research topics or discover information on my own. There’s a great value to knowing what others are reading; I use Twitter and Facebook to both share what I find and read what my friends and nptech peers recommend.  I use my feed reader to discover things.
Pretty readers present feed items in a glossy magazine format that’s hard to navigate through quickly and hell on my data plan.
The beta readers are the ones that look pretty good to me, until I have to wait 45 seconds for a small feed to refresh or note that their mobile client is the desktop website, not even an HTML5 variant.

What made Google Reader the reader for most of us was the sheer utility.  My 143 feeds generate about 1000 posts a day.  On breaks or commutes, I scan through them, starring anything that looks interesting as I go.  When I get home from work, and again in the morning, I go through the starred items, finding the gems.

Key functionality for me is the mobile support. Just like the web site, the Google Reader Android app wins no beauty contests, but it’s fast and simple and supports my workflow.

At this point, I’m putting my hopes on Feedly, listed above as a “too pretty” candidate.  It does have a list view that works more like reader does.  The mobile client has a list view that is still too graphical, but I’m optimistic that they’ll offer a fix for that before July.  Currently, they are a front-end to Google’s servers, which means that there is no need to export/import your feeds to join, and your actions stay synced with Google Reader (Feedly’s Saved Items are Google’s Starred, wherever you mark them).  Sometime before July, Feedly plans to move to their own back-end and the change should be seamless.

July is three months away. I’m keeping my eyes open.  Assuming that anyone who’s read this far is wrestling with the same challenge, please share your thoughts and solutions in the comments.

 

 

Category: Mobile, Technology, Web | 1 Comment
September 27 2011

Is It Time To Worry About Cybercrime?

This article was originally posted on the Idealware Blog in September of 2011.

For the past decade, the bulk of unlawful web-based activities have been profit-motivated: phishing, spam, “Nigerian” money scams, and hacking to get credit cards. This year has seen a rise in politically motivated crimes, most widely exemplified by the loosely-knit group of hackers known as “Anonymous“.  Anonymous hackers attack the websites of organizations, be they government, corporate or otherwise that they deem to be repressive or unethical.  In addition to defacing the sites, they’ve also routinely exposed confidential user information, such as login names, passwords and addresses.  If we are now entering the age where political cybercrime is commonplace, what does that mean for nonprofits?  How can we defend oursleves when we already struggle with basic security on tight budgets and limited resources?

Two high profile victims were Sony, the gigantic electronics and entertainment conglomerate, and BART, the Bay Area Rapid Transit commuter service.

  • Sony was initially a target for Anonymous after they took legal action against a computer geek named George Holtz, who figured out how to reprogram a Playstation game device in order to play blocked third-party games on it.  This violated the Sony license, but the hacking and gaming communities felt that the license restriction wasn’t very fair in the first place. They considered the action against Holtz unwarranted and severe.  Sony also, famously, installed a hacker’s rootkit, themselves, on a number of music CDs with interactive computer features, and were sued for that crime.,  Could it be that the hackers were particularly annoyed that this mega-corporation will stoop to their tactics, but sue them for similar actions?
  • BART was targeted for more visceral actions.  Their internal police force shot Oscar Grant, an unarmed youth, in the back a few years ago, and then, again, recently, fired on a homeless man holding a knife, killing him. These actions drew the attention of the community and resulted in protests, some violent.  But BART only drew the attention of Anonymous when they took the step of blocking cell phone service at their four downtown San Francisco stations in order to quell communication about a planned protest.  This action is under investigation by the FCC and has been decried by the ACLU; it was quite likely illegal. Then it was revealed that, at a press conference to discuss the protests, they seeded the audience with BART proponents coached in what to ask and say.

Anonymous hacked a dozen or more Sony Websites and three BART websites in protest/retaliation for what they consider to be corporate crime. Here’s how easy it was for them: one of the Sony servers containing hundreds of thousands of user account records was running on an old, unpatched version of Apache with no encryption. The initial attack was simply accomplished using a hack (SQL Injection) that is ridiculously easy to block (by updating to a current software version, in most cases). The Administrator password to get into the BART police site was “admin123″.  The “hacker” who broke into that site reported that she’d never hacked a web site in her life, she just did a bit of googling and got right in.

These were corporate web sites, run by companies that take in vast amounts of consumer dollars every day, and they couldn’t be bothered to do even the minimum amount of safeguarding of their customer’s data.  They might not be the criminals, but is it wild to suggest that they were criminally negligent? This isn’t a matter of them not having the money, resources or available expertise to protect our data.  It was a matter of them not taking the responsibility to protect it.

What can nonprofit organizations, that aren’t obsessed with bottom lines, do to avoid the problems that BART and Sony have faced?

  • First and foremost, we need to protect constituent data.  If your NPO doesn’t have the weherewithal to do that internally, than your online data should be hosted with companies that have strong commitments to security and privacy of customer data.
  • Second, should breaches occur (and they do), your primary goal should be timely, open communication with the victims of the data breach.  We’re getting past the point where our constituents are naive about all of this (Sony has done a great job of prepping them for us).  So your first response to exposed constituent data should be to tell the constituents exacty what was exposed.
  • One uncomfortable situation like this won’t kill your credibility, but a history of bad or callous relationships will amplify it.  This is one of the reasons why good social media policies are critical — the people who can support or sink you when something like a data breach occurs are on Twitter and Facebook, and they’ll feed the media stream with support or slander, depending on how well you relate to them.
  • We promote causes online, but we admit faults there, too.  We don’t engage customers by lying to them, hiding things that impact them, or dictating the terms of our relationships with them.
  • Our supporters are people, and they have their motivations for supporting us (or not) and their ideas about how they should be doing it.  Their motivations and reasoning might be quite different from what we assume. Accordingly, we should be basing our assumptions — and campaigns — on the best feedback that we can coax out of them.  Long-held industry assumptions are suspect simply because they’re long-held, in a world where technology, and how we interact with it, is constantly changing.

 

If we ever needed reverse primers in how to manage constituent relationships, the Sony and BART fiascos are prime ones.  They are victims of illegal and unethical behaviour.  But by viewing their customers and constituents as threats, with callous regard for the people who keep them in business in the first place, they’ve created a public relationship that did nothing to stem the attacks. Sony has put far more money and effort into attacking and dehumanizing their customers with lawsuits and invasive, annoying copyright protection schemes than they have in listening, or trying to understand the needs and desires of their constituents.  BART has tried to block their ears so tightly to shut out public criticism of their violent, shoot first police force that they’ve crossed constitutional lines of conduct. We — nonprofits — know better. It’s a two way relationship, not a dictatorial relationship with our supporters, that will serve as our most effective firewall.

Category: NPTech, Web | Comments Off
March 26 2011

Do Nonprofits Spam?

This article was first published on the Idealware Blog in March of 2011.

Supporters at the gates

NPTech maven Deborah Elizabeth Finn started a blog last week called “No Nonprofit Spam“.  As a well-known NPTech consultant, Deborah is far from alone in finding herself regularly subscribed to nonprofit email lists that she has never opted into.  But, as opposed to just complaining about what is, in anyone’s definition (except possibly the sender’s) unsolicited commercial email; Deborah took the opportunity to try and educate.  It’s a controversial undertaking. Nobody likes spam.  Many of us like nonprofits, and aren’t going to hold them to the same level of criticism as we will that anonymous meds or mortgages dealer; and the measures that we take against the seamy spammers are pretty harsh.  Even if nonprofits are guilty of the spamming crime, should they be subject to the same punishments?

Spam, like beauty, is in the eye of the beholder. So, for the purposes of this conversation, let’s agree on a definition of nonprofit spam. Sending one email to someone that you have identified as a potential constituent, either by engaging them in other media or purchasing their name from a list provider, is, at worst, borderline spam, and not something that I would join a campaign to complain about.  If I delete the message and don’t hear from the NPO again, no big deal.  But subscribing me to a recurring list without my express buy-in is what I consider spamming.  And that’s the focus of Deborah’s blog (which is naming names) and the action that goes from email engagement to email abuse, for the purposes of this post.

In my post to the No Nonprofit Spam website, I made the point that we’re all inundated with email and we can only support so many orgs, so NPOs would do better to build their web site and their Charity Navigator rating than to push their messages, uninvited, into our inboxes. It’s a matter of being respectful of constituent priorities.

There are two motivations for overdoing it on the emails. One is the mildly understandable, but not really forgiveable mistake of overenthusiasm for one’s mission.  Believing that the work you do is so important that subscribing people who have expressed no interest to your list is warranted.  That’s a mistake of naivety more than anything else.

The less forgivable excuse is the typical spam calculation: no matter how many people you offend, enough people will click on it to justify the excess.  After all, it’s cost-justified by the response rate, right?

The downside in both cases is that, if you only count the constituents you gained, then you’re missing something of great important to nonprofits and little import to viagra salesman.  The people you offended might have otherwise been supporters. The viagra spammer isn’t going to pitch their product through other avenues.  It’s a low investment, so any yeild is great gain.  But you likely have people devoting their full hearts to your cause.  You’re in the business of building relationships, not burning them.  And you will never know how many consttuents that you might have gained through more respectful avenues if you treat them callously with your email initiatives.

Worse, the standard ways that individuals deal with spam could be very challenging for an NPO to deal with.  In the comments to my No Nonprofit Spam post, some people advocated doing more than just marking the messages as spam, but also reporting the offending orgs to Spamcop, who then list them with Spamhaus, the organization that maintains block lists of known spammers that large ISPs subscribe to.  By overstepping the bounds of net courtesy, you could not only alienate individuals, but wreak havoc with your ability to reach people by email at all.  My take is that reporting NPOs — even the ones who, by my above definition, spam — is unusually cruel to organizations who do good in the world.  But I’m a nonprofit professional. Many of the people that we might be offending aren’t going to be so sympathetic.

So, what do you think? Is spam from a nonprofit any different from spam from a commercial vendor?  Should nonprofits be held to the same level of accountability as viagra spammers? Are even single unsolicited emails spam, or are they permissable? I searched for some nonprofit-focused best practices before completing this article, and didn’t come up with anything that differentiated our industry from the commercial ones, but I think there’s a difference. Just as nonprofits are exempt from the Do Not Call lists, I think we deserve some exemptions in email.  But I could be wrong, and what would serve us all well is a clear community policy on email engagement.  Does anyone have any to recommend?

Cartoon borrowed from Rob Cottingham’s Noise To Signal collection.

Category: NPTech, Web | Comments Off
December 26 2010

Delicious Memories

This article was originally published on the Idealware Blog in December of 2010.

Like many of my NPTECH peers, I was dismayed to learn yesterday that Delicious, the social bookmarking service, was being put to pasture by Yahoo!, the big company that purchased the startup five years ago.  Marshall Kirkpatrick of ReadWriteWeb has written the best memorial,  But the demise of Delicious marks a passing of significant note to our community of nonprofit staff that seek innovative uses of technology.  So let me talk quickly about how Delicious brought me into this community, and, along the way, a bit about what it meant to all of us.

In 2002, I was wrapped up in my job as VP of Information Technology at San Franciscco Goodwill.  At that time, the buzz term was “Web 2.0″, and it was all over the tech press with about a thousand definitions.  We all knew that “Web 2.0″ meant the evolution of the web from a straight publisher to consumer distribution medium to something more interactive, but nobody knew exactly what. Around that time, I started reading columns by Jon Udell about RSS, technology that would, as a simpler, subset of XML, helps us share web-based information the way that newspapers share syndicated content, such as comic strips and columns.  I was really intrigued.  The early adopters of RSS were bloggers, and what I think was very cool about this is that RSS was free technology that, like the web, advanced the opportunities of penniless mortals to become global publishers.  People who couldn’t tell an XML feed from an XL T-Shirt were championing an open standard, because it served as the megaphone in front of their soapboxes.

I kept my eye out for innovative uses of RSS,a nd quickly discovered Joshua Schacter’s del.icio.us website.  This was a social bookmarking service where, by adding a little javascript link to your web browsers bookmark bar (or quick links, or whatever), you could quickly save any web page you enjoyed to an online repository for later retrieval.  That repository was public, so others could see what you found valuable as well.  But this is where Schacter jumped the gun, and championed two information technology strategies that have, since that time, significantly changed the web: tagging and rss.

Tagging

In addition to the link and a brief description, you could add keywords to each bookmark, and then later find related bookmarks by that keyword.  You could just find the bookmarks that you tagged with a word, or you could find the tags that anyone using Delicious tagged with that word.  So, if you were studying the russian revolution, you could search Delicious for russia+revolution and find every bookmark that anyone had saved,   This was different than searching for the same terms in Google or yahoo, because the results weren’t just the most read; they were the sites that were meaningful enough to people to actually be saved.  Delicious became, as Kirkpatrick points out,  a mass-curated collection of valuable information, more like wikipedia than, say, Yahoo Directory.  Delicious was the lending library of the web.

RSS

In addition to searching the site for tags by keyword and/or user, any results your searching found could be subscribed to via RSS.  This was crazy powerful! Not only could you follow topics of interest, but, using PHP add-ons like MagpieRSS or aggregation functions like those built into Drupal, Joomla, and pretty much any major Content Management System, you could quickly incorporate valuable, easily updated content into your website.  I immediately replaced my static “Links” page on my website to one that grabbed items witha  particular keyword from Delicious, so that updating that Links page was as easy as bookmarking a site that I wanted listed there.

NPTECH

I wasn’t the only nonprofit strategist taking note of these developments.  One day, while browsing items that Delicious termed Popular (e.g., bookmarks that multiple people had saved to the site), I noted a blog entry titled “The Ten Reasons Nonprofits Should Use RSS“.  The article was written by one Marnie Webb of CompuMentor (now better known as TechSoup, where she is one of the CEOs).  A week or so later, while following the office email mailing lis for Delicious, I encountered Marnie again, and, this time, emailed her and suggested that we meet for lunch, based on our clearly common interest in nonprofits and RSS.  Marnie told me about the NPTech Tagging Project, and effort she started by simply telling her friends to tag websites related to nonprofit technology with the tag “nptech” on Delicious, so that we could all subscribe to that tag in our RSS readers.

Marnie and I believe that what we started was the first mass information referral system of this type.  In 2005 we took it up a level by creating the nptech.info website, which aggregates items tagged with nptech from Delicious, Twitter, Flicker and numerous other sources across the web. Nptech.info is now more widely read via it’s Twitter feed, @nptechinfo.

I think it’s safe to say that the nptech tagging project grew from a cool and useful idea and practice into a community, and a way that many of us identify who we are to the world.  I’m a lot of things, but nptechie sums most of them up into one simple word.  I know that many of you identify yourselves that way as well.

An offshoot of meeting Marnie on the Delicious mailing list was that she introduced me to NTEN, and brought me into the broad community of nptech, and my current status as a blogger, writer, presenter, Idealware board member and happy member of this broad community ties directly back to the Delicious website.  I stopped using the site as a bookmarking service some time ago, as efforts that it inspired (like Google Reader sharing)  became more convenient.  But I still subscribe to Delicious feeds and use it in websites.  It’s demise will likely be the the end of nptech,info.  Efforts are underway to save it, so we’ll see.  But even if this article is the first you’ve heard of Delicious, it’s important to know that it played a role in the evolution of nonprofit technology as the arbiter of all things nptech.  It’s ingenuity and utility will be sorely missed.

Category: NPTech, Technology, Web | Comments Off
July 31 2010

The Years Of The Kat

goodhunting

This is a memorial post for Krazy.com, a domain that I registered in February of 1995, back when Network Solutions was the only domain registrar and the annual registration fee was $0. I had recently closed my computer bulletin board system, which was called the Coconino County BBS, after the home of George Herriman‘s classic comic character, Krazy Kat. In it’s place, I put up a web site that grew to be the most complete and best known source of information on the now somewhat obscure, but dearly loved early 20th century newspaper strip about a Kat, named Krazy, who loved a Mouse, named Ignatz. This Ignatz found Krazy quite silly, and showed his disdain by throwing bricks at his/her head (Krazy’s gender was never identified). Offisa Pup, the local Kanine Konstable, who was in love with Krazy, arrested Ignatz every time he caught the mouse in the act. And all of this action took place against a surreally fluid landscape of mesas, monuments and moons inspired by Herriman’s love for eastern Arizona Navajo country, with it’s painted desert and monument valley.

pupslept
As my nptech crowd knows, I just got too busy over the years with other things to properly grow and manage this web site. As much as I love Krazy Kat (and my son’s middle name is Ignatz, no lie!), I have to prioritize my current pursuits. I am blessed with the opportunity to do meaningful work at Earthjustice, to blog, and to help out the nonprofit community where and when I can, as a board member at Idealware, a contributor to Techsoup, and a steadfast supporter of NTEN. There are only so many hours in a day.

Krazy.com had the distinction of being a short, catchy, .com domain name, which means that it’s sale value ain’t hay, and, while my life’s pursuits are pretty rich, I’m not. I got an offer that matched what the domain is professionally valued at, and I couldn’t afford to turn it down. It’s a melancholy moment — one of those decisions that isn’t difficult to make, but is sad all of the same, like trading in a beloved car that will cost too much to keep running.

In the more than 15 years that Krazy.com got steady traffic, from visitors that included Herriman’s great grand-daughter and Krazy Kat book cover artist Chris Ware, I built my career, got married, had a child, built a house, and lived a life that continues to be happy and rewarding. Krazy Kat is fond of singing “There is a heppy lend, fur, fur away”. My heppy lend is right here, and I’m sorry that I have to move away from my beloved Coconino County.

PLAQUE

Category: Life, Recommended Posts, Web | Comments Off
April 22 2010

Adventures In Web Site Migration

This post was first published on the Idealware Blog in April of 2010.

I recently took on the project of migrating the Idealware articles and blog from their old homes on Idealware’s prior web site and Google’s Blogger service to our shiny, new, Drupal-based home. This was an interesting data-migration challenge. The Idealware articles were static HTML web pages that needed to be put in Drupal’s content database. And there is no utility that imports Blogger blogs to Drupal. Both projects required research and creativity.

The first step in any data migration project is to determine if automating the task will be more work than just doing it by hand. Idealware has about 220 articles published; cutting and pasting the text into Drupal, and then cleaning up the formatting, would be a grueling project for someone. On the other hand, automating the process was not a slam dunk. Database data is easier to write conversion processes for than free form text. HTML is somewhere in the middle, with HTML codes that identify sections, but lots of free form data as well.

Converting HTML Articles with Regular Expressions

My toolkit (of choice) for this project was Sed, the Unix Stream Editor, and a generic installation of Drupal. Sed does regular expression searching and replacing. So I wrote a script that:

  1. Deleted lines with HTML tags that we didn’t need;
  2. stored data between title and body tags;
  3. and converted those items to SQL code that would insert the title and article text into my Drupal database.

This was the best I could do: other standardized information, such as author and publishing date, was not standardized in the text, so I left calling those out for a clean-up phase that the Idealware staff took on. The project was a success, in it that it took less than two days to complete the conversion. It was never going to be an easy one.

Without going too far, the sed command to delete, say, a “META” tag is:

/\<meta/d

That says to search for a literal “less than” bracket (the forward slash implies literal) and the text meta and delete any line that contains it. A tricky part of the cleanup was to make sure that my search phrases weren’t ones that might also match article text.

Once I’d stripped the file down to just the data between the “title” and “body” tags, I issued this command:

s/\<title\>(.*)\<\/title\>.*\<body\>(.*)\<\/body\>/insert into articles (title, body) values (‘\1′, ‘\2′);/

This searches for the text between HTML “title” tags, storing it in variable 1, then the text between “body” tags, storing it in variable 2, then substitutes the variable data into a simple SQL insert statement in the replacement string. Iterating a script with all of the clean-up commands, culminating in that last command, gave me a text file that could be imported into the Drupal database. The remaining cleanup was done in Drupal’s WYSIWYG interface.

Blog Conversion

As I said, there is no such thing as a program or module that converts a Blogger Blog into Drupal format. And our circumstance was further complicated by the fact that the Idealware Blog was in Blogger’s legacy “FTP” format, so the conversion options available were further limited.

There is an excellent module for converting WordPress blogs to Drupal, and there were options for converting a legacy Blogger blog to WordPress. So, then the question was, how well will the blog survive a double conversion? The answer was: very well! I challenge any of you to identify the one post that didn’t come through with every word and picture intact.

I had a good start for this, Matthew Saunders at the Nonprofits and Web 2.0 Blog posted this excellent guide. If you have a current Blogger blog to migrate, every step here will work. My problem was that the Idealware blog was in the old “FTP” format. Google has announced that blogs in their original publishing format must be converted by May 1st. While this fact had little or no relationship to the web site move to Drupal, it’s convenient that we made the move well in advance of that.

To prep, I installed current, vanilla copies of WordPress and Drupal at techcafeteria.com. I tracked down Google’s free blog converters. While there is no WP to Drupal converter, most other formats are covered, and I just used their web-based Blogger to WordPress tool to convert the exported Idealware blog to WP format. The conversion process prompted me to create accounts for each author.

To get from WordPress to Drupal, I installed above-mentioned WordPress-import module. As with the first import, this one also prompted me to create the authors’ Drupal accounts. It also had an option to store all images locally (which required rights to create a public-writeable folder on the Drupal server). Again, this worked very well.

With my test completed, I set about doing it all over again on the new Idealware blog. Here I had a little less flexibility. I had administrative rights in Drupal, but I didn’t have access to the server. Two challenges: The server’s file upload limit (set in both Drupal and PHP’s initialization file) was set to a smaller size than my WordPress import file. I got around this by importing it in by individual blogger, making sure to include all current and former Idealware bloggers. The second issue was in creating a folder for the images, which I asked our host and designer at Digital Loom.com to do for me.

Cleanup!

The final challenge was even stickier — the posts came across, but the URLs were in a different format than the old Blogger URLs This was a problem for the articles as well. How many sites do you think link to Idealware content out there? For this, I begged for enough server access to write and run a PHP script that renamed the current URLs to their former names — a half-successful effort, as Drupal had dramatically renamed a bunch of them. The remainder we manually altered.

All told, about two hours research time, three or four hours conversion (over a number of days) and more for the clean-up, as I wasted a lot of time trying to come up with a pure SQL command to do the URL renaming, only to eventually determine that it couldn’t be done without some scripting. A fun project, though, but I’d call it a success.

I hope this helps you out if you ever find yourself faced with a similar challenge.

Category: Blogging, NPTech, Technology, Web | Comments Off