Data and Evaluation/Measurement

This page has resources for utilizing publicly available data, recommendations for compiling participation data from departments, and collecting data to guide improvements or expansion of BPC activities.

Actions To Take

Data is essential for motivating proposed activities and evaluating/measuring the impact of implemented activities at achieving desired outcomes. Data can be collected directly from BPC activity participants, from the institution, region, or field. Data can also be used to track student and faculty retention.

Track Changes in Participation and Performance

  • Departments should partner with offices of Institutional Research to collect, track, and report on the following demographic data annually:
    • Faculty (by rank),
    • PhD students (enrolled, degree recipients),
    • MS students (enrolled, degree recipients),
    • BA/BS students in a computing major (enrolled, degree recipients),
    • Computing minors (enrolled, degree recipients),
    • Introductory students (CS1, CS2, CS3), and
    • Students earning a D or F, or withdrawing from particular computing courses (typically called DFW rates)
  • Consider identifying retention problems because points with low retention likely warrant additional data collection and intervention. Additionally, improving retention magnifies the impact of recruiting efforts.
  • Disaggregate data when possible to help identify specific opportunities for BPC activities. Departments may benefit from disaggregating data by:
    • Race/ethnicity
    • Gender
    • Disability
    • Economically disadvantaged or first-generation college status (no parent/guardian with a 4-year college degree)
    • Intersections of race/ethnicity, gender, disability, economically disadvantaged, and/or first-generation college status
      • Note: For the purposes of BPC Plans, only race/ethnicity, gender, and disability are included in the NSF definition of underrepresentation and all BPC activities included in BPC Plan need to focus on people who are underrepresented in computing because of these dimensions of their identity at the minimum but can include other groups.

Collect Feedback from Department Members (faculty, staff, and students) to Identify Opportunities for BPC Activities

  • Collect basic data such as participation through surveys, focus groups, or providing specific processes for feedback from department members, to help identify opportunities for improving or expanding BPC activities.
  • Analyze student performance by gathering and comparing data from previous years to identify if there are persistence gaps in the program by gender or race/ethnicity. This includes analyzing (1) rates for CS1 earning a D, F, or withdrawing from the course, (2) attrition rates after CS1, and (3) attrition rates after CS2.
  • Give student advocacy groups the opportunity to collect data for an annual wellness survey and allow them to present the results of the collected data at department meetings.
  • Measure rates of accommodation by partnering with your institution’s Office of Accessibility to monitor the representation of department members that require accommodations.
  • Collect and present data on the representation among undergraduate and graduate students graduating from your institution in comparison to peer institutions and regional K-12 schools. Learn more: University-specific graduation data, annual compilation of comparison data from BPCnet.org, NCWIT, Taulbee, ACM NDC, K-12 demographic data, workforce data)
  • Survey students in the department. The Data Buddies Survey is CRA’s free survey that is administered every fall and provides departments a report of their student responses compared to the responses of students from similar institutions as well as a breakdown of their data by gender and race/ethnicity. NCWIT’s Student Experience of the Major is another free survey that departments can customize and administer to their students.
  • The following resources can help departments evaluate and measure ongoing BPC activities – NSF’s The 2010 User-Friendly Handbook for Project Evaluation; SRI International’s Online Evaluation Resource Library; NCWIT’s Catalog of Evaluation Tools and Learning about Evaluation; and CSEdResearch’s Catalog of Evaluation Instruments.
  • Evaluate/measure the success of recruiting activities. Make use of NCWIT’s Pre-Post Survey for Outreach Programs, a free set of surveys to use before and after students participate in an outreach program, and Entry Survey, another free and customizable survey that departments can use to identify the impact and reach of their recruiting activities.

Use Publicly Available Data Repositories

Use Publicly Available Data Compilations