Guide to Evaluating and Purchasing Major Software Systems

8/11/2019 Guide to Evaluating and Purchasing Major Software Systems

1/10

The Perfect Fit: A Guide to Evaluating and

Purchasing Major Software Systems

By Peter Campbell, September, 2008

A major software package shouldn't be chosen lightly. In this detailed guide, Peter Campbell

walks through how to find software options, evaluate them, make a good decision, and then

purchase the system in a way that protects you.

A smart shopper evaluates the item they want to purchase before putting money down. You

wouldnt shop for shoes without checking the size and taking a stroll up and down the aisle in

order to make sure they fit, would you? So whats the equivalent process of trying on a

software package will size? How can you make sure your substantial software purchasewont leave you sore and blistered after the cash has been exchanged?

Thats the goal of this articleto provide some guidance for properly evaluating major

software investments. Well walk through how to find potential software options, gather the

detailed information you need to evaluate them, make a solid decision and purchase a

package in a way that protects you if it doesnt do what you hoped it would for you.

Is it A Major Software System?

The evaluation process described here is detailed, so its probably not cost effective to apply

it to every software tool and utility you purchase. How do you know if the package youre

considering is major enough to qualify? Major systems have a dramatic impact on your

ability to operate and achieve your missionthey arent measured by budget, theyre

measured by impact.

To help identify a major purchase, ask yourself:

Will the application be used by a significant percentage of your staff?

Will multiple departments or organizational units be using it?

Will this software integrate with other data systems?

If this software becomes unstable or unusable once deployed, will it have significant

impact on your nonprofits ability to operate?

Giving significant attention to these types of major purchases is likely to save your

organization time in the long run.

Taking Preliminary Measurements
http://www.addthis.com/bookmark.phphttp://www.idealware.org/print/1463http://www.idealware.org/printmail/1463


2/10

Prior to even looking at available software options, make sure you thoroughly define your

needs and what the application you select should be able to do for you. Nonprofits are

process-driven. They receive, acknowledge, deposit and track donations; they identify, serve

and record transactions with clients; and they recruit, hire and manage employees.

Technology facilitates the way your organization manages these processes. A successful

software installation will make this work easier, more streamlined and more effective. But anew system that doesnt take your processes and needs into account will only make running

your organization more difficult.

So its critical that, before you begin looking for that donor database or client-tracking

system, you clearly understand the processes that need to be supported and the software

features critical to support that work.

This is an important and complex area that could easily be an articleor a bookin its own

right. We could also write numerous articles that delve into project management, getting

company buy-in and change managementall critical factors in organizational readiness.

However, for the purposes of this article, were focusing on the process of evaluating andpurchasing software once youve already identified your needs and prepped the organization

for the project.

Finding the Available Options

Once you know what you need and why you need it, the next step is to identify the pool of

applications that might fit. An expert consultant can be a huge help. A consultant who knows

the market and is familiar with how the systems are working for other nonprofits can save

you research time, and can direct you to systems more likely to meet your true needs. While a

consultant can be more expensive than going it alone, money spent up front on the selection

and planning phases is almost always recouped through lower costs and greater efficiency

down the road.

If a consultant isnt warranted, take advantage of the resources available to the nonprofit

community, such as Idealware,Social Source Commons,Techsoups forumsorNTENs

surveys.Ask your peers what theyre using, how they like it and why. Ideally you want to

identify no less than three, and probably no more than eight, suitable products to evaluate.

Considering an RFP

With your list of possible software candidates in hand, the next step is to find out more about

how those packages meet your needs. This is traditionally done through a Request for

Proposal (RFP), a document that describes your environment and asks for the information

you need to know about the products youre evaluating.

Well-written RFPs can be extremely valuable for understanding the objective aspects of large

software purchases. For example, if you are looking for a Web site content management

system (CMS), questions such as does the blogging feature support trackbacks? or Can the
http://socialsourcecommons.org/http://socialsourcecommons.org/http://socialsourcecommons.org/http://ttp//www.techsoup.org/http://ttp//www.techsoup.org/http://ttp//www.techsoup.org/http://www.nten.org/http://www.nten.org/http://www.nten.org/http://www.nten.org/http://www.nten.org/http://www.nten.org/http://ttp//www.techsoup.org/http://socialsourcecommons.org/


3/10

CMS display individualized content based on cookie or user authentication? are good ones

for an RFP.

What you want from the RFP is information you can track with checkboxes. For example, It

can/cant do this, It can/cant export to these formats: XML, SQL, CSV, PDF, or They

can program in PHP and Ruby, but not Java or Cold Fusion. Questions that encouragevendors to answer unambiguously, with answers that can be compared in a simple matrix,

will be useful for assessing and documenting the system capabilities.

An RFP cant address all the concerns you're likely to have. Subjective questions like How

user-friendly is your system? or Please describe your support are unlikely to be answered

meaningfully through an RFP process.

Certainly, you can arrange for demonstrations, and use that opportunity to ask your questions

without going through an RFP process. But while the formality of an RFP might seem

unnecessary, there are some key reasons for getting your critical questions answered in

writing:

You can objectively assess the responses and only pursue the applications that arent

clearly ruled out, saving some time later in the process.

A more casual phone or demo approach might result in different questions asked and

answered by different vendors. An RFP process puts all of the applications andvendors on a level field for assessing.

The RFP responses of the vendor you select are routinely attached to the signed

contract. An all-too-common scenario is that the vendor answers all of your questions

with yes, yes, yes, but the answers change once you start to implement the software.

If you dont have the assurances that the software willdo what you require in writing,

you wont have solid legal footing to void a contract.

Structuring Your RFP

RFPs work well as a four section document. Below, we walk through each of those sections.

Introduction

The introduction provides a summary of your organization, mission and the purpose of theRFP

Background

The background section provides context the vendor will need to understand your situation.

Consider including a description of your organizationfor instance, number of locations,

number of staff and organizational structure, the processes the system should support, and

such technology infrastructure as network operating system(s) and other core software

packages. Include any upcoming projects that might be relevant.

Questionnaire


4/10

The questionnaire is the critical piece of the documentyou want to be sure you ask all of

the questions that you need answered. In preparing these questions, its best to envision what

the vendor responses might look like. What will have to be in those responses for you to

properly assess them? Consider asking about:

Functionality. In order to get answers youll be able to compare, ask your questionsat a granular level. Does a CRM support householding? Does a donor database have a

method for storing soft credits? Can multiple users maintain and view records of

donor interactions? Can alerts or notifications be programmed in response to

particular events? Use the results of your business requirements work to focus in on

the functions that are critical to you and your more unusual needs.

Technology specifics.Make sure the software will integrate properly with other

applications, that the reporting is robust and customizable by end users, and that the

platform is well-supported. Ask which formats data can be exported to and imported

from, how many tables can be queried simultaneously and what type of support is

availableboth from the vendor and third parties. Ask for a data dictionary, which a

technical staffer or consultant can review, because a poorly designed database willcomplicate reporting and integration. And ask for a product roadmap. If the next

version is going to be a complete rewrite of the application, you might want to ruleout the current version for consideration.

Company information. Think through what youll want to know about the company

itself. How big is it? Do they have an office near you? How long have they been in

business? Are they public or private? Can they provide some documentation of

financial viability? Who are the staff members that would be assigned to your project?

References from similar clients with similar-scope projects can also be very useful.

For more information on this area, see Idealwares articleVendors as Allies: How to

Evaluate Viability, Service, and Commitment.

Pricing and availability. What are their hourly rates, broken down by role, if

applicable? What are their payment terms? What is their total estimate for the project

as described? How do they handle changes in project scope that might arise during

implementation? What are their incidental rates and policies (travel, meals)? Do they

discount their services or software costs for 501(c)(3)s? How long do they estimate

this project will take? When are they available to start?

While its important to be thorough, dont ask a lot of questions you dont plan to actually use

to evaluate the systems. Asking questions just in case increases the amount of informationyoull need to sift through later, and increases the possibility that vendors might decide your

RFP isnt worth the time to respond to.

Instructions

Close with a deadline and details about how to submit replies. For a sizeable RFP, allow a

minimum of four to six weeks for a response. Remember that this isnt a confrontational

processa good vendor will appreciate and want to work with a client that has thought

things out this well, and the questionnaire is also an opportunity for them to understand the

project up front and determine their suitability for it. Respect their schedules and give them

ample time to provide a detailed response.
http://www.idealware.org/articles/vendors_as_allies.phphttp://www.idealware.org/articles/vendors_as_allies.phphttp://www.idealware.org/articles/vendors_as_allies.phphttp://www.idealware.org/articles/vendors_as_allies.phphttp://www.idealware.org/articles/vendors_as_allies.phphttp://www.idealware.org/articles/vendors_as_allies.php


5/10

Include an indication as to how additional questions will be handled. In general, if one vendor

asks for clarification or details, your answers should be shared with all of the RFP

participants. You want to keep things on a level playing field, and not give one vendor an

advantage over the rest. You might do this via a group Q&A, with all the vendors invited to

participate in a meeting or conference call after the RFP has been sent to them but well before

they are due to respond. With all vendors asking their questions in the same room, you keepthem all equally informed. Alternatively, you can specify a deadline by which written

questions must be submitted. All participants would then receive the questions and answers.

Evaluating the Answers

Once you receive RFP responses, youll need to winnow down your list to determine which

packages youd like to demo.

If you asked straightforward, granular questions, youll now reap the benefit: you can set up acomparative matrix. Create a table or spreadsheet with columns for each vendor and rows for

each question, summarizing the responses as much as possible in order to have a readable

chart. You might add columns that weight the responses, both on the suitability of the

vendor's response (e.g. 1, unacceptable; 2, fair; 3, excellent) and/or on the importance of the

question (for instance, some features are going to be much more important to you than

others).

Going through the features and technology sections, youll see the strong and weak points of

the applications. In determining which fit your needs, there will likely be some trade-offs

perhaps one application has a stronger model for handling soft credits, but another has more

flexible reporting. Its unlikely that any will jump out as the perfect application, but youll be

able to determine which are generally suitable, and which arent.

For example, if youre looking for software to manage your e-commerce activities, inventory

management might be a critical function for you. If a submitted software package lacks that

feature, then youll need to eliminate it. As long as you understand your own critical needs,

the RFP responses will identify unsuitable candidates.

You might rule out a vendor or two based on what the RFP response tells you about their

availability or company stability. Take care, though, in eliminating vendors based on their

RFP pricing information. RFP responses can be very subjective. Before determining that avendor is too pricy based on their project estimate, dig deeperother vendors might be

underestimating the actual cost. If you feel you have a solid grasp on the project timeline, use

the hourly rates as a more significant measurement.

The RFP responses will tell you a lot about the vendors. Youre asking questions that are

important to your ability to operate. Their ability to read, comprehend and reasonably reply to

those questions will offer a strong indication as to how important your business is to them,

and whether theyll consider your needs as the software is implemented and into the future. If

they respond (as many will) to your critical questions with incomplete answers, or with stacks

of pre-printed literaturesaying, in effect, the answers are in here--then theyre telling you

they wont take a lot of time to address your concerns.


6/10

Keep in mind, though, that a weak sales representative might not mean a weak vendor,

particularly if they're representing a product that comes recommended or looks particularly

suitable on all other fronts. It's acceptable to reject the response and ask the vendor to

resubmit if you really feel they have done you, and themselves, a disservicebut temper this

with the knowledge that they blew it the first time.

Trying It All On for Size

At this point the process will hopefully have narrowed the field of potential applicationsdown to three-to-five options. The next step is to schedule software demos. A well-written

RFP will offer important, factual and comprehensive details about the application that might

otherwise be missed, either by too narrow a demo or by one the vendor orchestrates to

highlight product strengths and gloss over weaknesses. But the demos serve many additional

purposes:

Evaluating look and feel. As good as the specs might look, youll know quickly in a

demo if an application is really unusable. For instance, an application might

technically have that great Zip code lookup feature you asked about in the RFP, but it

may be implemented in a way that makes it a pain to use. Prior to the demo, try to

present the vendors with a script of the functions you want to see. It can also be useful

to provide them with sample data, if they are willingevaluating a program with data

similar to your own data will be less distracting. Be careful not to provide them with

actual data that might compromise youror your constituents'privacy and security.

The goal is to provide a level and familiar experience that unifies the demos and puts

you in the driver's seat, not the vendor.

Cross training. The demo is another opportunity for the vendor to educate you

regarding the operating assumptions of the software, and for you to provide them with

more insight into your needs. A generic donor management system is likely to make

very good assumptions about how you track individuals, offer powerful tools for

segmentation and include good canned reports, because the donor-courting processes

are very similar. But in less standardized areasor if you have more unusual needs

the model used by the software application can differ dramatically from your internal

process, making it difficult for your organization to use. Use the demo to learn how

the software will address your own process and less conventional needs.

Internal training. Even more valuable is the opportunity to use the demos to show

internal staff what theyll be able to do with the software. Demos are such a goodopportunity to get staff thinking about the application of technology that you should

pack the room with as many people as you can. Get a good mix of key decision-

makers and application end-usersthe people who design and perform the business

processes the software facilitates. The people who will actually use the software are

the ones who can really tell if the package will work for them.

Making the Decision

With luck, your vendor selection process will now be complete, with one package clearlyidentified as the best option. If key constituents are torn between two options or unimpressed


7/10

with the lot, senior decision-makers might have to make the call. Be careful, however, not to

alienate a group of people whose commitment and enthusiasm for the project might be

needed.

If none of the applications you evaluated completely meets your needs, but one comes close,

you might consider customizations or software modifications to address the missing areas.Note that any alterations of the basic software package will likely be costly, will not be

covered in the packaged documentation and help files, and might break if and when you

upgrade the software. Be very sure there isnt an alternate, built-in way to accomplish your

goal. If f the modification is justified, make sure its done in such a way that it wont be too

difficult to support as the software is developed.

Before making a final decision, you should always check vendor references, but take them

with a healthy grain of salt. An organizations satisfaction with software depends not only on

how well it meets their needs, but how familiar they are with their optionsthere are a lot of

people who are happy using difficult, labor-heavy, limited applications simply because they

dont know there are better alternatives.

If you still have a tie after RFPs, demos and reference checks, the best next step is to conduct

on-site visits with an existing customer for each software package. As with demos, bring a

representative group of management, technical staff and users. Assuming the reference can

afford the time to speak with you, the visit will highlight how the software meets their needs,

and will give you a good, real world look at its strengths and weaknesses. Youll also likely

walk away with new ideas as to how you might use it.

Signing on the Dotted Line

Youve selected an application. Congratulations! You might be tired, but you arent finished

yet. You still need to work with the vendor to define the scope of the engagement, and an

agreement that will cover you in case of problems. A good contract clearly articulates and

codifies everything that has been discussed to date into a legally binding agreement. If, down

the road, the vendor isnt living up to their promises, or the software cant do what you were

told it would do, then this is your recourse for getting out of an expensive project.

Contract negotiations can take time. Its far more dangerous to sign a bad contract in the

interest of expediency, though, than it is to delay a project while you ensure that bothpartiesyou and the vendorcompletely understand each others requirements. Dont start

planning the project until the papers have been signed.

A software contract should include a number of parts, including the actual agreement, the

license, the scope of work and the RFP.

The Agreement

This is the legal document itself, with all of the mumbo jumbo about force majeure and

indemnity. The key things to look for here are:


8/10

Equal terms and penalties.Are terms and penalties equally assessed? Vendors will

write all sorts of terms into contracts that outline what you will do or pay if you dont

live up to your end of the agreement. But theyll often leave out any equivalent

controls on their behavior. You should find every if this happens, customer will do

this clause and make sure the conditions are acceptable, and that there are

complementary terms specified for the vendors actions. Reasonable cancellation penalties. If there are penalties defined for canceling a

consulting or integration contract, these should not be exorbitant. Its reasonable for

the vendor to impose a limited penalty to cover expenses incurred in anticipation of

scheduled work, such as airfare purchased or materials procured. But unless this is a

fixed cost agreement, which is highly unusual, dont let them impose penalties for

work they dont have to dofor example, for a large percentage of the estimated

project cost.

Agreement under the laws of a sensible state. If the vendor is in California, and

youre in California, then the agreement should be covered by California laws rather

than some random other state. In particular, Virginias laws highly favor software

companies and vendors. In most cases, you want the jurisdiction to be where you live,or at least where the vendors headquarters actually are.

The Software License

The license specifies the allowed uses of the software youre purchasing. This, too, can

contain some unacceptable conditions.

Use of your data. A software license should not restrict your rights to access or workwith your data in any way you see fit. The license agreement will likely contain

conditions under which the software warranty would be voided. Its perfectly

acceptable for a commercial software vendor to bar re-engineering their product, but

its not acceptable for them to void the warranty if you are only modifying the data

contained within the system. So conditions that bar the exporting, importing,

archiving or mass updating of data should be challenged. If the system is hosted, the

vendor should provide full access to your data, and the license should include

language providing that client shall have reasonable access for using, copying and

backing up all customer information in the database. There should be no language in

the contract implying that the vendor owns your data, or that they can use it for any

additional purposes. Responsibility avoidance.Software warranties should not include blanket software

provider is not responsible if nothing works statements. This shouldnt need to be

said, but, sadly, there are often warranty sections in license agreements that say just

that.

Back doors. The license should not allow for any post-sale reversals of licensing,

such as language stating that the contract will be void if the customer uses the

software in perfectly reasonable ways they dont anticipate. For instance, if you want

to use the CRM functions of your donor database to track contacts that arent potential

donors, you shouldnt sign a contract limiting use of the software to fundraising

purposes. Also, there should not be any back doors programmed into the

application that the vendor can maintain for purposes of disabling the software.


9/10

The Scope of Work

The Scope of Work (SOW) describes exactly what the project will consist of. Its an

agreement between the vendor and the customer as to what will happen, when, and how longit will take. Good scopes include estimates of hours and costs by task and/or stage of the

project. The scope should be attached as a governing exhibit to the contract. Usually, this is

negotiated prior to receiving the actual contract. By having it attached to the contract, the

vendor is now legally obligated to, basically, do what they said they would do.

The RFP

Like the Scope of Work, the RFP should also be attached as a governing document that

assures that the software does what the vendor claimed it would.

In Conclusion

For big ticket purchases, it's well worth having an attorney review or assist in negotiations.

Keep in mind that the goal is to end up with a contract that equally defends the rights of both

parties. True success, of course, is a solid contract that is never revisited after signing.

Litigation doesn't serve anyone's interest.

Bringing It Home

Theres a lot of talk and plenty of examples of technology jumpstarting an organizations

effectiveness. But if someone were to do the tally, there would probably be more stories of

the reverse. All too often, organizations make decisions about their software based on

uninformed recommendations or quick evaluations of the prospective solutions. Decisions are

often based more on expediency than educated selection.

Rushing a major investment can be a critical error. Learn about the available options,

thoroughly assess their suitability to your needs and prepare your staff to make the most of

them. Then, sign a contract that protects you if, after all else is done, the application and/or

vendor fails to live up to the promises. Finding the right application and setting it up to

support, not inhibit, your workflow is a matter of finding something that really fits. You cantdo that with your eyes closed.

For More Information

Vendors as Allies: How to Evaluate Viability, Service, and Commitment

An Idealware article on how to think specifically about the less-concrete aspects of software

selection.
http://www.idealware.org/articles/vendors_as_allies.phphttp://www.idealware.org/articles/vendors_as_allies.php


10/10

How To Find Data-Exchange-Friendly Software

An overview of how to ensure you're going to be able to get data in and out of a software

package. (For much more detailed considerations, see ourFramework to Evaluate Data

Exchange Features.)

Peter Campbell is the director of Information Technology at Earthjustice, a nonprofit law

firm dedicated to defending the earth, and blogs about NPTech tools and strategies at

Techcafeteria.com. Prior to joining Earthjustice, Peter spent seven years serving as IT

Director at Goodwill Industries of San Francisco, San Mateo, and Marin Counties, and has

been managing technology for non-profits and law firms for over 20 years.

Robert Weinerand Steve Heye also contributed to this article.
http://nten.org/blog/2007/10/23/how-to-find-data-exchanage-friendly-softwarehttp://nten.org/blog/2007/10/23/how-to-find-data-exchanage-friendly-softwarehttp://www.idealware.org/data_exchange/download.phphttp://www.idealware.org/data_exchange/download.phphttp://www.idealware.org/data_exchange/download.phphttp://www.idealware.org/data_exchange/download.phphttp://www.rlweiner.com/http://www.rlweiner.com/http://www.rlweiner.com/http://www.idealware.org/data_exchange/download.phphttp://www.idealware.org/data_exchange/download.phphttp://nten.org/blog/2007/10/23/how-to-find-data-exchanage-friendly-software

Documents

Guide to Evaluating and Purchasing Major Software Systems