View Project

G2S Project Code: 2021-MT-89193
State: Montana
Fiscal Year: 2021
Grantee
Montana State Library

Project Director
Director Name: Cara Orban
Director Phone: 406-444-5350
Director Email: corban@mt.gov
General Information
Title: Five-Year Plan Evaluation
State Project Code:
Start Date: 07/01/2021
End Date: 09/30/2022
Abstract: Montana State Library hired a third-party evaluator to measure progress toward the goals outlined in Montana's 2018-2022 Five-Year Plan. Through a competitive bidding process, QualityMetrics was awarded the contract. The evaluator assessed the efficacy of the State Library's activities in advancing its goals and developed key findings and recommendations to help shape the next Five-Year Plan.
State Goal: Community Leadership
Budget Information
LSTA
MATCH-State
MATCH-Other
Total
$23,800.00
$0.00
$0.00
$23,800.00
Intent(s)
Improve library operations.
Research & Statistics
Activities
Activity Details
Title: LSTA Grants to States Program Evaluation
Narrative: QualityMetrics deployed a multi-faceted and rigorous mixed-methods protocol for data collection. 
Through their research, QualityMetrics found that Montana had achieved or partially achieved all five goals, noting that additional progress would be made in the remaining two years of the plan. Montana State Library shared key findings and recommendations with the State Library Commission and Network Advisory Council during their regular meetings. Coordinators also reviewed results at library federation meetings, and the State Librarian discussed the evaluation via a webinar targeted at the library workforce and vested partners called Webside Chat.
Intent: Improve library operations.

Activity: Planning & Evaluation
Mode: Retrospective
Format: Third party


Quantity
Number of evaluations and/or plans funded: 1
Number of funded evaluation and/or plans completed: 1


Partner Information
Organization Type of Partner Organization(s):
Libraries: No
Historical Societies or Organizations: No
Museums: No
Archives: No
Cultural Heritage Organization Multi-type: No
Preschools: No
Schools: No
Adult Education: No
Human Service Organizations: No
Other: No


Legal Type of Partner Organization(s):
Federal Government: No
State Government: No
Local Government (excluding school districts): No
School District: No
Non-Profit: No
Private Sector: No
Tribe/Native Hawaiian Organization: No


Beneficiaries
Is the activity directed at the library workforce: Yes
For a targeted group or for the general population: General


Locale
Is the activity state-wide: Yes
Specific Locations: No
Library Types
Public Libraries: 0
Academic Libraries: 0
SLAA: 1
Consortia: 0
Special Libraries: 0
School Libraries: 0
Other: 0
Question 1: I believe the planning and evaluation addresses library needs.
Strongly Agree: 0
Agree: 0
Neither Agree nor Disagree: 0
Disagree: 0
Strongly Disagree: 0
Non-Response: 0
Question 2: I am satisfied with the extent to which the plan or evaluation addresses library needs.
Strongly Agree: 0
Agree: 0
Neither Agree nor Disagree: 0
Disagree: 0
Strongly Disagree: 0
Non-Response: 0
Question 3: I believe the information from the plan or evaluation will be applied to address library needs.
Strongly Agree: 0
Agree: 0
Neither Agree nor Disagree: 0
Disagree: 0
Strongly Disagree: 0
Non-Response: 0
Project Outcomes
Project Outcomes
List any important outcomes or findings not previously reported:
Based on its assessment, QualityMetrics made recommendations for Montana's next Five-Year Plan: 1) Continue to invest in technology for libraries such as hot spots, wiring upgrades, and broadband.  2) Provide more outreach to libraries to help them understand their options for services such as E-Rate and other opportunities.  3) Provide access to electronic resources for all types of libraries.  4) Address the challenges of staffing limitations and capacity at libraries.  5) Recruit schools into the lifelong learning programs.  6) Conduct user experience studies of MSL’s website resources.  7) Provide technology support for small libraries.  8) Invest more in courier services.  9) Help libraries expand their collections. 
Please briefly describe the importance of these outcomes and findings for future program planning:
The Montana State Library considered recommendations from QualityMetrics, in conjunction with feedback from library directors, Core Services Committees, and library federations, for drafting its 2023-2027 Five-Year Plan.
Explain one or two of the most significant lessons learned for others wanting to adopt any facets of this project:
When selecting an evaluator, determine what outputs they can provide to assist with developing the next Five-Year Plan. For example, QualityMetrics provide a one-page summary, but more in depth recommendations could be beneficial for some libraries.
Do you anticipate continuing this project after the current reporting period ends:
No
Do you anticipate any change in level of effort in managing this project:
No
Explain:

Do you anticipate changing the types of activities and objectives addressed by the project:
No
Explain:

Was an evaluation conducted for this project:
Yes
Was a final written evaluation report produced:
Yes
Can the final written evaluation report be shared publicly on the IMLS website:
Yes
Was the evaluation conducted by project staff (either SLAA or local library) or by a third-party evaluator:
Third-Party
What data collection tools were used for any report outcomes and outputs:
Administrative Records
Review Surveys
Direct Observation
Interviews
Focus Groups
Did you collect any media for the data:
What types of methods were used to analyze collected data:
Statistical Methods
Qualitative Methods
Other:
How were participants (or items) selected:
Targeted Sample – We selected based on a desired characteristic, e.g. age.
What type of research design did you use to compare the value for any reported output or outcome:
Pre-post comparison for a reported output or outcome: The evaluator used a retrospective assessment to measure the state's progress toward the goals outlined in the 2018-2022 Five-Year Plan. Thorough reviews of demographic data, quantitative PLS data, and SPR data formed the basis of the evaluation. Additional data gathering to triangulate the research included: 1) interviews with agency staff members; 2) interviews with project managers of various SPR-reported projects and activities; 3) library websites; and 4) responses to three web-based surveys.
Exemplary: No
Exemplary Narrative
Project Tags: