CSPro for Data Entry Workshop

Carolina Population Center

May 16-21, 2011

Useful Links

U.S. Census Bureau websitehttp://www.census.gov/ipc/www/cspro

CSPro Usershttp://www.csprousers.org

Twitterhttp://twitter.com/cspro

Mailing list for CSPro questionscspro@lists.census.gov

History

Census Bureau Involvement

U.S. Census Bureau has a long history of using technology to process censuses, going back to Hollerith and the 1890 census

International Programs Center has been helping countries process census data for decades

Off-the-shelf software in the past was not affordable, not easy to use, or otherwise was not practical for some countries where IPC worked

Integrated Microcomputer Processing System (IMPS) Census Bureau developed and supported, with funding

principally from USAID DOS-based, text-based, used a COBOL compiler to enhance

functionality Several modules:

Entry (CENTRY) Editing (CONCOR) Tabulation (CENTS) Variance Calculation (CENVAR) Operational Control (CENTRACK)

CSPro does not have the latter two modules (yet)

Combination of IMPS and ISSA USAID was funding the development of two statistical

processing software packages: IMPS ISSA: Integrated System for Survey Analysis

Developed by SERPRO and Macro International

Both software packages were in need of an update to enter the Windows/GUI world

CSPro combines the best features of both software packages: The ease-of-use of IMPS The advanced functionality and logic language of ISSA Called: Census and Survey Processing System

CSPro in One Slide

CSPro is a free software package used by hundreds of organizations and tens of thousands of individuals for entering, editing, tabulating, and disseminating census and survey data. CSPro is designed to be as user-friendly and easy to use as possible, yet powerful enough to handle the most complex applications. It can be used by a wide range of people, from non-technical staff assistants to senior demographers and programmers. The funding for CSPro comes from USAID.

Release History

CSPro 2.0 (the first version) released on May 5, 2000 CSPro 3.x (2005) included a much more powerful tabulation

system CSPro 4.0 (2008) added support for PDA devices Most recent version (CSPro 4.1) released on May 5, 2011 CSPro 4.1 (2011) added additional tools for CAPI operations

on portable computers such as tablets

Worldwide Use

Used in over 160 countries by: National Statistical Offices NGOs Universities Hospitals Militaries Businesses

Has processed the census data for countries as varied as Djibouti (less than one million people) to Bangladesh (over 150 million people)

CSPro Processes…

Censuses (population and housing; agriculture; and economic) Demographic and labor force surveys Household income and expenditure surveys Major international projects such as:

Demographic and Health Surveys (DHS), Macro International Living Standards Measurement Study (LSMS), World Bank Multiple Indicator Cluster Survey (MICS), UNICEF

Future?

IMPS to CSPro marked the transition from DOS to Windows With low-cost computers proliferating (tablets, smartphones),

will there be another paradigm shift in the future?

Strengths and Weaknesses

Cost

CSPro is free and will be forever Some software packages charge both designer and

implementer fees Some software packages charge by number of data collection

points

Support

Census Bureau provides free support for CSPro via email:cspro@lists.census.gov

There are CSPro consultants available for hire, though not many

Many applications have been created, though few available on the Internet

Very few Internet websites devoted to CSPro

Platform

Works on any modern machine with Microsoft Windows Data entry can be done, one question at a time, on Windows

Mobile PDAs (versions 5 and 6, not Windows Phone 7) The growth and activity in low cost tablets and smartphones

is not centered around Windows No way to turn applications into Internet applications

Robustness

Has been tested and used for over ten years Can handle census-sized applications Can export data to major statistical software formats, and

often is used in combination with other programs Most major bugs have been fixed, though some known bugs

(generally minor) remain

Data File

Flat file, text-based format Advantages:

Human readable Possible to create and modify without a data entry application

Data File (continued)

Disadvantages: Large file sizes, though smaller when compressed Extra files (e.g., an index) must be stored as separate files Modifying an entry can force the whole file to be rewritten Fixed width fields mean that changing a dictionary means

reformatting existing files Inability to store variable-length text Processing is slower Not compliant with common database programs and languages,

e.g., SQL Only one user can write to a file at any given time

Keyed Data Entry

CSPro was specifically designed for keyed data entry Can track the rates of keyers A verification operation can be established Has two modes that dictate how much control a keyer has

over an operation: operator- and system- controlled modes Operational control of files can be challenging to new users,

particularly if managing an operation with many keyers Not ideal for keying letters unless they are Latin letters Key-from-image not supported natively

CAPI Data Entry

Supports CAPI controls (e.g., radio buttons) but as popup windows, not built into the form

No UNICODE support, though kludges mean CSPro can support data entry in most languages

Subject-oriented, non-linear, data entry is difficult to program Designing the dictionary, forms, and data entry is done in

three places, rather than all integrated

Logic Language Robust, and designed specifically for data entry and processing Easy tasks are simple to program and require few programming

skills Some tasks could be programmed more cleanly in an object-

oriented framework Simple debugging (tracing) is possible, but dynamic

modification of the code is not, requiring a rebuild of the application for any small change

The syntax of features added later (e.g., CAPI functions) is sometimes awkward

Can run slowly as it is an interpreted language Inability to include standard modules across many applications

Editing and Tabulation

Editing module is specifically designed for census editing, an uncommon task

Tabulation is simple, drag and drop, and can create complex tables with some programming

Area processing Inability to perform complex analysis via the tabulation

module Editing, and especially tabulation, are slow when run on large

data sets

Missing Modules

Census variance Operational control system for data entry, and for application

design Cleaner interface with industry standard GIS products, rather

than the built-in MapViewer