How we migrated Freshworks’ critical monorepo tackling tech debts

At Freshworks Marketplace, we have leveraged a single codebase to power two distinct experiences: a unified in-product gallery and a public website. Despite their differing natures, we have designed them in a manner that ensures a consistent UI across both experiences.

Our gallery experience is developed as a single-age application using Vue.js powered with Vue CLI, Vue Router, Vuex, and deployed to AWS S3. On the other hand, our website is a server-side rendered application powered by Nuxt.js and deployed to S3.

You can read the full story behind our decision to use a single codebase here.

While we have enjoyed the benefits of using a monorepo to help with this unique problem, it has also presented its own set of tradeoffs. One such challenge we encountered was migrating our entire codebase to the latest versions of the components in our tech stack. To make things more interesting, around the same time we recognized the need to improve the user experience through a major revamp. Our latest revamp features prominent filters, upgraded search capabilities, eye-catching colors, better use of available real estate, and a streamlined user interface that’s intuitive and easy to use. With these enhancements, our platform now provides an even more user-friendly experience than ever before. This post summarizes our learnings and the approaches that worked for us during this journey. 

Getting started

On February 7, 2022, Vue 3 was announced as the new major version for Vue.js. Having heavily invested in Vue.js since 2018, we were eager to learn about the new upgrades and advancements from this community-led project. 

Prior to this launch, back in February 2021, the first stable release of Vite JS caused a stir on Twitter. On one of the regular knowledge-sharing sessions we hold internally, we covered Vite and studied its potential very closely. This session prompted a proposal from one of our engineering managers to explore its use for our platform. 

Since then, we have actively sought opportunities to integrate Vite into our workflow. It was a pleasant surprise to discover that Vite has been adopted as the official build toolchain for Vue 3. Meanwhile, we noticed that other important tools like Pinia for state management and Vitest for testing paired up with Vue 3 after gaining popularity as the defaults in many developer toolchains. This momentum presents a promising opportunity for us to become early adopters of Vue 3 and grow together with the community.

We wasted no time and began our experimentation immediately. 

What made this upgrade critical to us?

“Vue 3 is faster, smaller, more maintainable and it’s easier to target native.”

         — Evan Vue

As stated earlier, Vue 3 was set to become the new default version, and it is expected that support for Vue 2 will end by December 31, 2023. Vue 3 introduces several unforeseen modifications to its fundamental principles and APIs, diverging from previous updates. This necessitates progress in all other areas of the framework, which would soon push our current toolset into maintenance mode. This, in turn, could result in a growing number of security issues that are challenging to handle. Additionally, the latest upgrade pledges significant performance enhancements and a complete shift in API methodology, further emphasizing its significance.

Additionally, Vue 3 offers the following advantages that we particularly cared about:

  • Smaller Vue core: Size improvement reached by the combination of greater code modularity, a smaller core runtime, and a tree-shaking-friendly compiler. Thus the new core would be reduced from about 20KB to 10KB gzipped
  • Better rendering performance: The Vue 3 renderer improves the performance of the rendering process by using optimizations such as the hoisting and inlining of static parts and implementing Vue 3’s reactivity with ES6 proxies
  • Blazing fast, Vite-powered build toolchain
  • More ergonomic Composition API syntax via <script setup>
  • Improved TypeScript IDE support for Single File Components (SFCs) via Volar
  • Command-line type checking for SFCs via vue-tsc
  • Simpler state management via Pinia
  • New dev tools extension with simultaneous Vue 2 / Vue 3 support and a plugin system that allows community libraries to hook into the dev tools panels

You can read more about what’s new in Vue 3 here.

All the above factors provided enough persuasion to take on the challenge of being an early adopter for Vue 3. As an early adopter, with few proven examples to benchmark against, we also recognized the responsibility with which we must undertake this exercise. 

It is important to keep in mind that there are likely to be some compatibility issues with existing code and third-party libraries during any major upgrade. It is therefore essential to conduct thorough research and take gradual steps. Also, staying current with the latest framework updates and changes is vital, as they are subject to frequent modifications from early feedback.

Planning the migration

“By failing to prepare, you are preparing to fail.” 

Before embarking on this large-scale migration, it was important to have a plan. We needed to get down to the level of tasks and their estimated timelines because we had planned marketing activity around the revamped design. The plan is particularly crucial in our case, as our repository is a monorepo that supports two distinct user experiences and therefore requires due consideration of how different components interplay across the stack.

We eventually came up with the following list of high-level items as part of the migration plan:

  • Upgrading from Vue version 2 to Vue version 3 (first for the in-product experience)
  • Upgrading from Nuxt version 2 to Nuxt version 3 (for the website)
  • Upgrading the build tooling from Vue CLI to Vite
  • Migrating from the Options API to the Vue Composition API
  • Switching from Vuex to Pinia for state management
  • Eliminating the use of Element UI in several places on the UI in favor of Freshworks’ own design library Crayons

As many of the above-listed items are relatively new to the community, the learning curve will undoubtedly be steep. Therefore, meticulous planning and prior experimentation were deemed to be necessary.

Consequently, we opted to begin with a comprehensive analysis of the migration and its impact on the existing codebase. Through this process, we were able to provide an approximate estimation of the required efforts and timelines.

Analysis 

Although we have compiled a list of the action items, it is crucial to examine and comprehend the following points:

  • How would each item benefit us?
  • Why is it necessary to take each of these items into account, and are there any items that we can do without?
  • The level of effort required for each item, and whether we can accommodate it in terms of timelines

In order to address these questions, we conducted a comprehensive analysis of each topic and documented our findings in a clear and concise manner on our internal wiki. This served as a learning center for future developers who would work on this upgraded codebase and facilitated a comprehensive peer review.

We also created a comprehensive checklist, which included a table estimating each task, and shared it with the team for initial feedback. This exercise allowed us to visualize the resources required and potential challenges and opportunities at the outset. As a result of this exercise, we made some helpful discoveries.

  • We identified noncritical UI updates that could wait for subsequent releases, allowing us to better allocate our time 
  • We compiled a list of backend requirements in advance, providing clarity on each item and enabling our backend team to work on the list in parallel
  • This also provided an opportunity for our team to suggest some long-standing tech debt items that could potentially be paired if deemed relevant and beneficial

The experimentation

“A journey of a thousand miles begins with a single step.”

      – Chinese proverb

Now that we had completed the analysis phase, the next step was to experiment with the code. Our Marketplace boasts over a thousand apps built by developers from around the globe. Before we rocked this boat, we decided to conduct a proof of concept (POC) prior to committing to a switch.

As part of the POC, we evaluated the following factors:

  • The overall development experience with the new APIs
  • The ease of transition between the Options and Composition APIs
  • Any blockers related to the current bundling mechanism, SSR, etc
  • Would TypeScript be beneficial at this scale?
  • Potential pitfalls in migrating from Vue CLI to Vite
  • How Crayons fits in with SSR, and any potential blockers
  • How the new state-management library works with our current data model

While we were experimenting with the changes, the state of Nuxt was not entirely stable. Nuxt 3 was in the Release Candidate stage, and as a result, we found ourselves in a position to decide whether to adopt Nuxt 3 immediately or wait for the stable release.

The problem with not updating the Nuxt version is that we would lose the flexibility of having components in common between our in-app and website experiences. The monorepo architecture is intended to reuse components as much as possible, but if our website continues to use Nuxt 2, it will not be able to understand the Vue 3 APIs that are part of our reusable components.

We had two options:

  1. Experiment with the Nuxt Release Candidate and proceed with the migration
  2. Create a duplicate of common components that will remain in Vue 2 to be consumed by the website service

The second option is clearly not scalable in the long term and will result in unnecessary rework in the future.

We engaged three committed engineers who would travel with us through this release journey. One of these engineers, who had previously presented a Vite experiment in our internal knowledge-sharing session, would be responsible for conducting the POC, which would be supported and reviewed by two of our leads. The POC was carried out in stages.

Phase 1: We experimented with Vue 3 APIs, focusing solely on the in-product experience. We migrated only the homepage as a standalone app to better evaluate the DX and identify any potential issues. 

Phase 2: This involved experimenting with Nuxt 3 RC to determine its suitability. Since the framework was not yet fully stable, conducting an in-depth experiment was crucial. 

We created a separate clone for this POC in a private repository, and our lead and architects thoroughly examined the code. We also included it as a demo item in our weekly sprint demo and collected feedback from the entire team.

Outcomes of the POC

Phase 1:

  • APIs and store: Our homepage successfully utilized multiple API calls and a significant portion of our data store, covering almost all new APIs including Composition API, Pinia store, async API calls, etc. No blockers or disparities were encountered with existing features
  • Third-party dependencies: Certain unnecessary third-party dependencies were identified and removed, reducing our node-modules burden. Essential dependencies such as Fuse.js and Vue I18n were fitting in as expected. Deprecated dependencies such as Node Sass were also identified and replaced with alternatives such as Dart Sass

Phase 2:

  • Nuxt 3: The experiment of using Nuxt 3 RC with Vue 3 APIs under the hood was relatively simple and encountered no major issues
  • Fetch API: We identified an opportunity to replace Axios with simple HTTP fetch APIs as our calls mainly involve reading data from simple static JSON files
  • Third-party dependencies: We evaluated critical third-party dependencies being used in the Nuxt project, and most of those turned out to be working as expected
  • Nuxt Bridge: Nuxt Bridge (a forward-compatibility layer that allows you to experience many of the new Nuxt 3 features by simply installing and enabling a Nuxt module) was also evaluated but found to be unsuitable for our purpose, as it caused issues in the build process

Stages of migration

Although we were now quite confident about the migration thanks to the learnings from the POC, it was crucial to plan the stages in which the migration would occur. As a team, we try to avoid big-bang changes and deployments unless there is no alternative available.

The idea was to gradually ship the migration in stages to ensure a smooth transition without any breakage in production. The stages we came up with are represented in the picture below:

tech debtPhase 1: The first stage involves migrating the code to Vue 3 without affecting the UI. This can be done separately from the UI revamp, which takes longer and requires a separate release. 

Phase 2: After implementing the Vue 3 code, we had the option of gradually upgrading the UI, but we ultimately decided to release the updated in-product experience all at once. We made this decision based on the quick completion of the earlier phase, which allowed us to save bandwidth and avoid inconveniencing users with inconsistent UI differences.

Phases 3 and 4: The website experience remains unchanged until this phase, where the migration from Nuxt 2 to Nuxt 3 is carried out, similar to the in-product migration in Phase 1.

Phases 3 and 4 were eventually done in parallel, building on the knowledge gained in the earlier stages to fast-track progress. The refreshed website experience can be viewed at https://www.freshworks.com/apps/.

Tech debts 

In addition to the aforementioned steps, we recognized the potential for improvement in other areas such as productivity, performance, and build times.

Build setup: To optimize our build pipeline and accommodate our growing scale, we transitioned from a multi-branch to a single-branch repo structure, adopting the proven trunk-based development methodology for monorepos. Our repo now consists of a single master branch, with tags created for each environment. This allows us to leverage trunk-based development and deploy changes incrementally across different environments.

IDE Setup: We upgraded our IDE setup to align with the new set of tools introduced. Previously, we relied on vetur in conjunction with VSCode. The recommended IDE setup now is VSCode with the Volar extension. This extension supports syntax highlighting, TypeScript, and IntelliSense for template expressions and component props. Additionally, Volar’s VSCode extension provides out-of-the-box formatting for Vue Single File Components, which made it an ideal choice for our setup.

Security: Also as a result of this upgrade, the number of security vulnerabilities decreased significantly, with critical vulnerabilities virtually eliminated.

Metrics on DX improvements 

From a developer productivity perspective, we noticed significant gains from this upgrade, which are shared below for reference.

Comparison: In-product

comparison in-product

 

 

 

 

 

Comparison: Website

comparison website

 

 

 

 

 

Vulnerability audit: In-product + website monorepo

Before upgrade:

Vulnerability audit: In-product + website monorepo

 

 

 

 

After upgrade:

Vulnerability audit: In-product + website monorepo

 

 

 

 

Learnings and takeaways

Our team managed to accomplish all of these improvements within a quarter by planning and experimenting beforehand. Despite facing a steep learning curve, we succeeded in delivering by remaining focused on our goal, sticking to the plan, learning quickly together, and staying aligned as a team. 

For anyone planning a similarly major codebase migration, we recommend considering the following major takeaways from our experience.

  • We suggest analyzing each task item to determine priorities and understand the return on investment
  • We also recommend conducting the migration in stages to avoid disrupting the pipeline and user experience
  • Experiment with proofs of concept in areas that are unclear to minimize surprises
  • Document and audit each stage to make the learning process easier for others on the team

Good luck with your major migration!

References

Related Posts