One Enterprise System to Unite Multiple Organizations

Associated organization

Nova Scotia department of Transportation & Infrastructure Renewal
4 months (2019)

Context

The injured workers, post-healing, faced a major delay in returning-to-work.

Separate paper-based assessments introduced perplexity in comparison between 3 different government organizations because each had native comparison methods.

To eliminate people scratching their heads, a standardized B2B enterprise system had to be created to satisfy users from diverse demographics, expertise and mental models.

My Role

UX Researcher & Information Architect

Responsibilities

• Led a team of 6
• Champion of UX across multiple cross-functional teams
• End-to-end UX research
• Experience strategy for enterprise product
• Designed high-fidelity prototypes rich in information architecture

Results

92%

participants scored >9/10
in perceived user comfort.

87%

reduction of
average filling time.

62%

reduction in
average reviewing time
.

Project Overview

Process

• Agile UX + Lean startup cycle

Generative research

Organizational research:
• Remote stakeholder interviews
• Organizational ethnography - Focus on culture & operation styles
• Executives, laymen and team alignment workshops
• Flat-organization workshops

Ethnographic user research:
• Contextual inquiries
• Task analysis
• Work study

Strategical research & synthesis:
• User personas
• SWOT
• Empathy map
• Buy-a-feature (NEW)

Design

• Information architecture (Content strategy & Site map)
• Paper sketches
• Wireframes
• Rapid Prototyping - Medium fidelity prototypes

Evaluative research

Information architecture testing:
• Card sorting
• Tree testing

Formative testing:
• A/B testing.
• Formative moderated usability testing (In-person & Remote)
• Summative usability testing pipeline (Time & Efficiency)

Due to conformance with a non-disclosure agreement, I have recreated, obfuscated and omitted the actual works to exclude confidential information. 

All information disclosed in this case study are my own and doesn't necessarily reflect the views of  Nova Scotia Department of Transportation and Infrastructure Renewal.

Block Quote

the situation

Challenges

• Distinct user demographics and knowledge of expertise
• Tech-averse audience
• Low budget
• Big scope but short time

The team

• UX designer & researcher (myself)
• Front-end developer
• Back-end developer
• Kinesiologist
• Ergonomist
• Healthcare and safety specialist

Position of UX in the project

A direct causation of improving the assessor's experience resulted in making the assessment process efficient and cost-saving (direct & indirect). This project acted as a seed for an enterprise system in the long run.

Project methodologies: Agile UX + Lean startup cycle

methodologies

Why?
- Dynamic requirement/need updates.
- Need for weekly deliverables.

Results:

• Achieved large goals in short time.
• An MVP with visible improvements boosted internal trust.

Agile UX

Since the product updates (especially feature sets) were volatile, to cope up with the product road map, I led the team to follow partially-agile methods for the course of this project. No specific standards were adopted, but the core principles were brought in.

Lean startup cycle

To bring out the novel product into full functioning ASAP, the lean startup cycle was combined with agile methods. This made swift delivery of the Minimum Viable Product (MVP) possible.

As a result, the craving for immediate benefits, for the client organization as well as users, was achieved in a short span. Significantly, under the given time, it revived the possibility of large work under short time.

1. Organizational Research

Why?
- Understand the enterprise culture.
- Identify viable communication channels.
- Find out 'What outcomes speak?'

Results:

• Understand exactly how can an enterprise system integrated.
• Conclude on business needs, scope & objectives.

To set the scope and nature of the product, a complete understanding was achieved by exploring the organizational ideologies, communication standards, organizational structures, project expectations and delegation mechanisms.

Setting product vision and goals

I aligned the team and the stakeholders by devising measurable goals through workshops, meetings, knowledge-sharing sessions and presentations. The creep of the product’s scope was eliminated by completing a thorough organizational research.

Team alignment workshops

1st discussion: Bring in new paper-based forms
2nd discussion: Can attach photos in the forms
3rd discussion: Cost benefits of human-centered tech
4th discussion: A paper-based form + an electronic form
5th discussion: A print-friendly electronic platform (DECIDED)
6th discussion:  ***The surprise awaits at the end of case study***

Getting to know from bone to skin

Unraveled the actual organizational needs by conducting:
• Remote stakeholder interviews
- What is the level of UX maturity within the organization?
- What future does the management wishes to build?
- How significant is this project to satisfy internal expectations?

• Plan & strategy back tracking (bottom-to-top)
- How does authority and task delegation work?
- What does the ground-zero workers think of the superiors?
- What loopholes are present in the existing system?

• Flat organization meetings
- Easiest way to find common grounds.
- A storytelling session of distinct personas to empathize daily responsibilities.
- A passive social that significantly reduces the impact of Hawthorne effect.

The butterfly effect

I established a good rapport with the business executives and our team by leading & conducting a daily scrum. Basking in the trust I created, positive side effects that were due, unfolded as follows.

The daily status updates created transparency in our work, approach and effects towards solving problems. This paved way to hear out our recommendations regarding change management and controlled communication at the executive level during deployment.

Business & organization needs identified

• Establish a standardized and efficient system.
• Create the product to act as a communicable bridge.
• A flexible product for the longer run (MVP).

2. User Research

Why?
- Identify 'What we can provide to the users?'
- Spot the loopholes in the existing system.
- Evidence to synthesize strategy & design.

Results:

• Raw, unfiltered & unbiased (to a certainty) qualitative data to expose the current state.
• Quantitative data directed to establish metrics for the project.

To understand the types of users who would benefit from the product, the user's mental models, company-specific informal communication methods, user personas & their behaviors and the indirect stakeholders were studied.

Setting UX goals

The organizational research helped me find a varied distribution of users. The target users squared up were mechanics, insurance assessors, healthcare workers, doctors, managers & executives.

Techno-cultural understanding & expectations

Identified wants, needs and hidden requirements through:
 User interviews
- An introductory session to get to know the users.
- Found the symptoms, users' wants and visible pain points.

 Contextual inquiries
- More like a 'formal date' with an individual user.
- Played a vital role to understand users in-depth and find actual problems.
- As a moderator, I addressed a checklist of topics while striking a natural conversation.

• Task analysis (observe, record & organize)
- Simultaneous with contextual inquiries.
- The kinesiologist and I switch roles in asking and assessing.
- Played a vital role in deconstructing activities to micro-level.
- Helped the team in drafting information architecture.

• User surveys
- To confirm and validate qualitative data remotely from a large sample.
- The questions also identified individual needs & accounted multiple perspectives.
- Resulted in confirming the hypotheses and the numbers helped to increase trust on UX from management.

 Work study
- Teamed up with ergonomist and kinesiologist in identifying needs of the validators.
- An analytic session with frequent ground-zero visits helped me, as a UX researcher to understand the substance of the processes.

Every data gathering methods were conducted based on ethnographic research methods because the focus was to on collect raw, unfiltered and unbiased qualitative data.

User needs finalized

• Easier, quicker, comfortable and a clear B2B system.
• Flexible content and error-resistant architecture.
• Make the switch to the new product efficient & effortless.

The most common tech platform used by all user personas was MS Excel. Keeping the users' comfort in mind and to achieve a smooth transition of change in the longer run , the initial prototype was built on MS Excel and tested .

Success metrics
The combined results from organizational and user research led me to devise and validate the following success metrics:

1. Perceived user comfort (To be increased)
2. Assessment time (To be reduced)

Since the reduction in cost associated can't be disclosed, the penultimate factors that drive the cost reduction are established as product performance indicators.

3. IDEATION

Why?
Educate myself and in turn educated my team & the stakeholders to understand the end-users.

Results:

• Goals established to solve the problems that aligns with the experience strategy.

User Personas
The research was analyzed by clustering the qualitative data through SWOT & empathy maps.

3 distinct user personas & respective task flows (after conducting red route analysis) were created.

The fictitious user personas generated from original data, carried the uniqueness and intricacies as a lucid end-user. This further convinced the business team and the executives to completely believe in the sense of direction of the project.
Empathy map of a user persona - Assessor (Manager/Engineer)

3. Ideation

Why?
- Testing ideas and IA at the early stages exposes strong and weak user flows.

Results:
• >95% success rates in all tests
• Precise feature sets at the intersection of budget, feasibility and requirements.

Skeuomorphing the process for info scent
To avoid stressing the mental models of the users during change, the entire structure and content of the information architecture was derived from the existing process-based-skeuomorphism.

In other words, the IA  was derived from the order and structure of observations from task analyses & contextual inquiries (inclusive of all worker categories). This was decided to help users find expected information at expected positions & order.
Site map

3. Ideation

Why?
- Testing ideas and IA at the early stages exposes strong and weak user flows.

Results:

• >95% success rates in all tests.
• Precise feature sets at the intersection of budget, feasibility and requirement.

Card sorting, Tree testing & Buy-a-feature
Card sorting: To generate the idea of IA and hypothesize the users' mental models based on results.

Tree testing:
To validate the conformance content with the users.

Buy-a-feature:
A gamified process to prioritize feature sets. (Contact me if you want to know more on this)

Results from testing ideas

The hypotheses generated from the research results were validated by creating, testing & iterating the ideas. The final plan after 2 iterations resulted in the following measures:

• Success rate >95% in all the tests.
• Conformation of info architecture with users’ mental models.
• Finalized feature set, content and info arch for the creations.

An executive report consisting of product road map (only MVP), experience strategy, process methodology, feature set, IA and content marked the end of ideation phase.

4. SYNTHESIS

Why?
- Transforming ideas into reality.

Results:

• Introduced inclusion in creations through user friendly, accessible & simple designs.

Sketching all the way
The creative processes were carried out to achieve the following goals:

Paper sketches - Visual representation of the idea.
(Pen & Paper)

Wireframes - Analysis of spacing and content.
(Axure RP & Adobe Comp)

Prototypes - Interaction tests & real-time content interpretation.
(Adobe XD & MS Excel)

Pattern libraries - To help with developers following atomic CSS process and to enhance design sustainability.

Elements of the pattern library were semantically named in developer's class context i.e. atoms --> molecules-atoms --> organisms-molecules-atoms --> templates-organisms-molecules-atoms . (Did not create for pages)

Design principles:
User-centered design (EID)+ Ecological interface Design (EID)
Initial sketches

4. SYNTHESIS

Why?
- Champion a clear social cause as a team lead.

Results:
• Established accessibility as a basic need at the structural level of designs.

Making way for accessibility
A main aspect focused on this product was accessibility. Introduction of users in early research processes and testing in ideation phase revealed 3 color blind users and 5 users with partial hearing loss in the sample population.

A monochrome color scheme, alt texts, alert sounds, simple content language and design for errors helped to counteract this problem. This did not only improve the experience of the fore-mentioned individuals but it improved the overall user experience and supported every user facing situation-based disabilities.
Creation of visually accessible designs in auto-generated mapping feature

(My apologies for the contrast mismatch perception in number '5' because of image enlargement.
The same field in the actual designs & product were validated 'PASS' for WCAG 2.0 AA at a ratio of 4.81 : 1)

4. SYNTHESIS

Why?
An interface used when taking notes on observation must be swift to learn and (simple + easy) to use.

Results:
• Created personalized dashboard and user flows for distinct user personas.

Reducing cognitive load
The interface was divided into different sections to progressively disclose information according to the user category using the system.

Different forms of data structures to interpret information were created to ease the cognitive loads. Few of the significant features were, an auto-generated map, auto-generated report, overall score sheet and a auto-generated most safe/moderate/vulnerable activity list. (Please contact me to know more details)
Intermediate working prototype handover to customers for pilot testing (lookalike)

5. Test & validate

Why?
- Identify defects and feedback fixes ASAP.

Results:
- Tests performed at the end of each creative processes accelerated the iterative design process.
- Created an error preventing evaluative testing architecture.

All-round user testing
The only way to validate my creations was to test them.

To support the agility of the creative processes and avoiding piling up of defects in the final designs, the testing had to be done at the end of every creative process.

The availability of the users facilitated multiple testing as a feasibility at every creative checkpoint.

Since, the number of users (sample population) in user testing pools were low, a significant assumption was made. For every test conducted, a metric of 80% success rate marked to progress to the next stage.

Two major user testing processes conducted were:
• A/B testing
• Moderated in-person usability & performance testing
A/B testing
Why?      
Home page testing  for positioning of ‘clear all data’ CTA.

When?    
After creating mid-fidelity mockups.

RESULTS:  
Reduction in unintentional user errors in home page by ~94%.
A/B testing with 'Type B' layout winning the contest

5. Test & validate

Goals achieved:
- Find, fix and re-test errors.
- Create a transparent workflow model that helps within and outside the team.
- Communicate the product's value in terms of numbers of significance.

Moderated usability testing & user performance testing
Why?    
To find usability errors and measure filling time, review time,  & perceived user-comfort. (Test population– 13 direct & indirect stakeholders)

When?  
During flat organization workshops and on-boarding workshops.

RESULTS:  ***Interesting outcomes awaits in the next section***
User performance testing - 'Reviewing time' comparisons of existing and new system
User performance testing - 'Perceived user comfort' comparisons of existing and new system

6. RESULTS

The final product achieved the following numerical outcomes:

Perceived user comfort
~92% participants scored >9/10

Filling time
~87% reduction of average filling time

Reviewing time
~62% reduction in average reviewing time

Workshops & support

I also conducted on-boarding workshops which consisted of demo runs, walk-throughs and Q&A sessions to help the users familiarize with the new product. A user manual was created & distributed for making their experience on the long-run (in reference to the product road-map explained in the next section) fruitful.

To ensure acceptance of the new product,our team provided in-person & over-the-call support during the final week.

The consultant who keeps giving

As a delight, I surprised the executives with two perks:

1.
 An overview of the future product road-map. This explained the need for a cloud-based environment and the necessity of B2B SaaS enterprise system. The road-map's walk-through was appended with the strategy to ease change management. The proposal was happily accepted with thanks and it is soon-to-be-implemented.

2. A workshop for assessors to facilitate a self-initiated summative usability testing pipeline to measure success metrics. I created assessment forms to address user issues as a feedback pipeline to help designers and developers once my contract is over. Also, I taught the inclusion of such insights in the areas in executive reports.

LEARNING OUTCOMES
FOR PROJECT
-FAILURE: 'Testing diverse users' remained in successive sprint backlogs because of non-accessible initial designs.
+SOLUTION:
Strive for accessible designs right from mid-fi prototyping. It helps all users facing situational / temporary / permanent disabilities.


-FAILURE: Confusions in deciding on testing strategy for a brand new product.
+SOLUTION:
Testing after every creative process minimized queuing of defects, and provides validity to direct decisions.


• Testimonials gained more user trust than advertisements or preaching.

• Early collaboration with transparency in communication is the key to team chemistry.
FOR SELF
- FAILURE: Team burn out on nearing the end of the project.
+ SOLUTION: Distribution of responsibilities must be team and member-specific.


-FAILURE: Started the project without assessing the level of UX knowledge of the client organization.
+SOLUTION:
Initially, ask leading questions to figure out the UX maturity of each stakeholder. Communicate accordingly.


• For a product, the aesthetics paves way for ‘love at first sight’ but a good IA builds ‘a happily ever after’.

• Communicate using comfortable & understandable terms (tone, metrics, jargons, humor etc.) that the receiving end craves for.