June 2024 - August 2024
Product Designer
1 product owner,1 delivery lead, and 1 business developer.
Development is in progress, and the launch is planned for May 2025.
Overview
Problems
Challenges educators face when accessing student college readiness data
College readiness refers to a student's preparedness to succeed in postsecondary education without the need for remedial courses.
However, educators face many challenges in accessing students' college readiness levels. For example,
Data Fragmentation
College readiness data often comes from various sources. Merging and making sense of fragmented data can be time-consuming and confusing.
Resource Constraints
Accessing, analyzing, and using this data effectively requires training and often software resources that may be limited in under-resourced schools.
Various CCR metrics
Each state develops its own standards and benchmarks to evaluate students' readiness for college and careers.
Background
Rally Analytics Platform and California College Roadmap Report
Education Analytics is advancing this area by supporting educators in accessing student data. EA has developed the Rally Analytics Platform to help educators analyze student data effectively in one platform. Additionally, EA’s college roadmap projects aim to build a predictive analytics model to assess college readiness for students and parents in CORE districts, California.
Rally is used by districts in South Carolina, Texas, New York, and other states.
College Roadmap Report is for California.
Project Objectives
Develop the college roadmap feature in Rally Platform
To better support educators in helping 7th to 12th grade students prepare for college, the project aims to build a college readiness feature into the Rally platform, leveraging the advantages of the data integration process in Rally and the previous predictive analytics model in California. This project took 10 weeks to complete the design process, from research to testing.
Results
This feature has been adopted by the South Carolina Department of Education
The South Carolina Department of Education has shown great interest in the college roadmap feature and has adopted it. My team is currently developing the feature to tailor it to South Carolina's college readiness indicators and to process students through Ed-Fi, with a planned launch in May 2025.
Research
Secondary Research
Different college readiness indicators across states
I compared the college readiness indicators across 10+ states, including South Carolina, New York, California, Vermont, Texas, and others that have adopted the Rally platform. I also examined high school graduation requirements, state college admission requirements, and scholarship criteria. 👉Click here to access the report
Common CCR Indicators
end-of-course examinations
AP/IB course completion and minimum scores
ACT/SAT scores
FAFSA completion
dual enrollment courses
state required assessments
chronic absenteeism rates
HS graduation and state college admission requirements
Most high school graduation credit requirements are more comprehensive compared to state college admission requirements.
Stakeholder Interview
Gathered insights from different perspectives.
The company did not have a plan to conduct user research at this point, so I conducted stakeholder interviews with data engineers, data analysts, researchers, and project leads to gain more insights into use cases, public data, data sourcing, data coverage, and other related topics. We also discussed the possibilities, challenges, and feasibility of this project, which helped me a lot with my design later.
Use cases
Rally primary users are teachers, counselors, and administrators in classroom settings.
Low-to-no training needed for highly intuitive data pages
Data
Student data is processed through Ed-Fi system built by EA, with updates occurring almost daily, while assessments may take longer to complete.
Student skills or interests could be included, but data sourcing and coverage need to be considered.
Predictive analytics model
The current predictive analytics model is very California-specific. At a high level, most components can be transferred to Rally.
User Storyboard
Visualized use cases and built empathy with users
Counselor David
Teacher Sarah
Design Challenges
Jobs-to-be-Done for counselors and teachers
Determine Students' College Readiness Levels
Enable users to quickly assess college readiness levels for each student, saving time on analysis and understanding readiness indicators.
Visualize Students' Growth Over Time
Provide a clear visual representation of students' academic growth, allowing users to easily track individual progress and performance trends over time.
Identify Students Falling Behind
Alert users to students who may be struggling, enabling timely and targeted support to keep students on track for college readiness.
Ideation
Sketches
brainstormed ideas and concluded the value propositions
Predictive Analysis Model
Calculate students' readiness levels.
Growth Tracking and Visualization
Track individual student progress over time, allowing to monitor performance.
Progress Comparison to Benchmarks
Compare how far students are from/meeting each requirement.
Overview of All Students Performance
Identify areas that need attention to provide support.
A new tab was added to Rally, along with an entry on the home page. The green sections indicate the new additions.
I transferred ideas from the sketches to wireframes and then conducted critiques within the team. We discussed the pros and cons of each version. The highlighted parts indicated the selected directions which includes readiness level, course performance, assessments.
Design
Data Visualization
Explored the visualization methods for each type of data
For selected sections, I created various data visualizations to present the data effectively and intuitively. For course grades, readiness levels, advanced courses, and assessments, there are different approaches to visualizing the data.
Iterations
Optimized the page structure for improved usability and navigation
High-Fidelity Prototype
Completed the class and student levels of views
Class Level
Users can view the distribution of students' readiness levels in the class. On the readiness page, they can find how many students participate in AP/IB courses, FAFSA completion rates, and detailed information in the table, along with assessments data compared to district levels.
Student Level
Users can view student readiness performance over time on the homepage. On the readiness page, they can find GPA, UGP GPA, credits earned, course progress compared with high school graduation requirements, AP/IB exam results, and comparisons with scholarship criteria.
Testing
Stakeholder testing
Evaluated design through different perspectives
I conducted testing sessions with stakeholders to gather their insights and feedback, evaluating the look and user interactions. This is helpful for collecting input on design aesthetics and usability.
Iterations
Optimized the design based on feedback.
3/5 stakeholders found the graph confusing for college readiness lanes. Since the predictive analytics model is still being researched, I used badges to represent the lanes. I also added a goal for the year and included an explanation of how the score was calculated.
It will be difficult to obtain district-level data for SAT or ACT assessments, and 4/5 stakeholders mentioned they didn’t understand what the colors represent. I removed the comparison and labeled the scores for each color. When users hover over a section, they will also see the student’s name and score.
Final Deliverables
Effectively helps educators understand the college readiness of students
This feature has been adopted by the South Carolina Department of Education 🎉
Conclusion
Next step
Conduct user interview
We're planning conduct user interview in November to gather insights directly from end-users in South Carolina to understand their needs, preferences, and challenges.
Connect with clients to gather specific requirements
Collaborate with clients to clarify detailed requirements about college readiness metrics in South Carolina.
Run Data Analysis on the Technical Side
Work with the technical team to understand the actual data format and tailor the design to implementation standards.
Explore Additional Possibilities
Consider incorporating features like interest assessments, skills tracking, scholarship matching, personalized recommendations, and other areas that can add value for users.
Key Takeaways
Don’t hesitate to ask questions or share ideas
Asking questions and presenting ideas, even early in the process, can uncover valuable insights and prevent misunderstandings. It’s essential for aligning the team and ensuring a clear, shared vision.
Listen to stakeholders and advocate for users
Stakeholders provide context and business priorities, but it’s equally crucial to be the voice for users. Balancing both perspectives helps create solutions that meet user needs while aligning with strategic goals.
Early feedback is essential for improving the design
Gathering feedback early allows for iterative improvements and helps catch potential issues before they become costly to fix. It also ensures that the design evolves based on real needs and insights.
Be flexible and adapt to new information
Projects often shift as new information emerges. Flexibility and a willingness to adapt ensure that the final product is relevant and effective, even if it diverges from the initial plan.
Document findings and decisions for future reference
Keeping detailed records of insights, decisions, and rationales provides clarity and continuity, especially as the project progresses or new team members join.
