Improving Releases at JOOR

Reduce product support call volume, increase customer satisfaction for new releases.

“You can’t release anything during Market…” said our VP of Global Customer Experience. The Director of Global Product Support nods his head enthusiastically. They both have fear in their eyes as I discuss the dates I want to launch the new global navigations for our four customer groups. (They’re referring to the fashion week calendar where Retail Buyers get to view and purchase Brands’ collections.) The thing is, our users are in Market eight months out of the year.

They had every right to be cautious; every major release had been a nightmare for them and their teams. The most recent release, a new UX/UI for the Order Review page, had over 108 product support cases from Pro Brands (paying customers) just in July, with over 54 users asking to opt out of the test. By the end of Dec, there were over 400 Tier 1 incidents when we ended the test. Any way that you looked at it, it was a complete disaster.

The redesign for the navigation I wanted to launch was a fraction of the complexity of Order Review, but I had to respect my colleagues and their experience. I agreed to start my beta test for the new navBar for the same user group, Pro Brands, during the slowest time of the year: Nov/Dec. But this just wasn’t a good way to work. How could I get the feedback from customers I needed if no one was logging in? I couldn’t do usability tests; only internal folks at JOOR had tested the new designs. I needed to learn how the new navBar would perform from a usability standpoint, and because language comprehension was a big question, I also needed to test our translations.

JOOR’s typical beta release process was to choose a group of accounts (not by user, but their entire user base) and force them into the beta. We’d send out email communications preparing those users for the update and then automatically launch a Pendo guide to walk them through the changes when logged in. I started following this example, except I needed to target users, not entire accounts. I wanted to prioritize anyone who had changed their default language on our settings page, used the language switcher, or used a language other than English at the browser level. I then had to weed out anyone upset by the Order Review beta or any customers who were at risk. That gave me around 4125 users from 415 Brands. From December 4, 2023, to January 1, 2024, less than half of those users (2008) logged in. I had a meager incident rate, with only five users contacting Product Support. My second beta group was 4553 users from 628 Brands, but only 1773 users logged in from December 15, 2023 - January 1, 2024. We had 3 Product Support incidents. As everyone returned to work the first week of January, I had 4897 users in the beta test who had not seen the new navigation and one last beta group to test with 956 of our most vocal users and Milan Men’s Fashion Week starting on January 12. Things were about to get real, or so I thought. Both lead from Customer Support and Product Support wanted to stop the remaining beta test and did not want to launch Global Acceptance.

Two other projects depended on the launch of the new navigation, and I needed a way to convince my stakeholders to go forward with the release plan. The only thing I could think of was creating a way for users to choose to experience the new nav and opt out of the new experience if it was too much of a disruption, taking the weight off our support teams. I needed a way for users to have the ability to turn the feature flag on and off themselves.

So I did just that. I proposed an Opt-in/Opt-out widget that would live on the home page, and whatever page had new functionality. Here’s a video of it below:

So far, we’ve received no product support issues on the Opt-in/Opt-out widget itself, and we’ve put two redesigns into the widget for beta testing: the new navBar and a refresh of the manage styles page.