One codebase, two experiences

Sometime back in 2018, we started building a new Freshworks Marketplace App Gallery designed to serve a unified experience for all our Freshworks products. Shortly after we had finished building the baseline, we started building a Marketplace Website to deliver the same experience on the internet. This is the story of how we built both of these experiences — one an in-product SaaS experience, and another an internet-facing website experience — from a single codebase.

On the Freshworks Developer Platform team, we build an SDK, offer PaaS capabilities and power an App Marketplace to combine forces with developers, and solve complex business problems together for our customers. Developers can quickly build various Apps on top of our highly scalable platform and help businesses that use Freshworks products to cater to their needs. Our Marketplace proudly lists over a thousand apps today built by developers around the world.

Freshworks’ customers access the Marketplace within a signed-in product experience as shown below, allowing them to discover, explore, and manage their apps.

On the other hand, potential Freshworks customers browse, discover, and explore our Marketplace on the internet through our App Website.

 

We had already invested heavily into our in-product experience when our team was tasked with the goal of quickly building and launching a new Apps website in time for Refresh 2019, London.

The designs for the in-product gallery and the app website were very similar. The similarities included on how apps are listed and how information is shown to the user. However, there were some functional differences:  

1. Unified experience: The website is responsible for listing apps from multiple Freshworks products. Apart from apps, there are classifications such as collections, categories, trending apps, popular apps, etc. These list apps from multiple products too. Whereas, the in-product gallery is responsible for listing these only for a specific Freshworks product at a time.
2. Installation experience: The installation experience is different from the website and the in-product gallery. The in-product gallery does the heavy lifting for handling the complex logic of installing an app whereas the website just redirects to the in-product gallery for installation.
3. Admin experience: The in-product gallery has additional features and pages that help an admin to check and maintain apps that are installed in a product. These features are not made available to a public-facing website.

Since the experiences for listing apps within the product and the website were  very similar, we found out that a large part of the in-product gallery could be reused. We were motivated to look for ways that the existing code base for the in-product gallery could be reused to build the website as well.

🤔 The challenges

When we started building the new Freshworks Apps Website, we identified the following requirements:

1. Re-use existing VueJS code as much as possible  —  It was essential that we re-used the in-product gallery code as much as possible. Since the experience was fairly similar, we would reuse components. Also, features that are added in the future to one of these versions of our store-front should easily show up in the other.
2. SEO — Discoverability is important for a public website. It is important for a public website to be search engine-friendly so that web-crawlers can crawl it without hiccups. Crawlers should be able to read HTML and extract data for indexing search results and then rank the website. Only a few crawlers like the Googlebot can parse Javascript but that is not very reliable. In a Single Page App, a crawler without Javascript enabled will only be able to parse the root file of the page, which generally has no data since all the data is rendered via Javascript. Server-Side Rendering helps to solve this issue as the page is rendered on the server and sent to the browser. This makes it available for crawlers to parse. The Marketplace website needs to be Search Engine-friendly so that it ranks well in search engines and users can easily find and stumble upon our site.
3. Responsive Design — The Freshworks Marketplace is visited by users from all kinds of devices(desktops, mobile phones, tablets, etc.) with various screens and form-factors. It was important to design the website in such a way that we provide a great adaptive and responsive experience for all screen sizes.
4. Performant — Frontend performance is very important for public-facing websites because this might be one of the first interactions a potential customer has with our products. We wanted to provide a seamless experience to visitors even if they are on a slow network.
5. Scalable — As a public website, it needed to cater to any kind of visitor traffic — even those driven by campaigns that generate a lot of it. We wanted a solution that effortlessly scales and provides consistent availability to give users a true Freshworks experience.

👋 Hello, NuxtJS!

Since we wanted to reuse most parts of the code from our in-product gallery, we needed to find a solution built around VueJS itself. As described above, our SEO needs also drove us toward a server-side rendered website.

A quick search revealed that VueJS can be rendered on the server. However, doing this involves a lot of effort in rendering each component. This is when we stumbled upon NuxtJS!

NuxtJS is a framework to build production-ready apps with Vue. It does all the heavy lifting to run a website/web-app and keep it performant. VueJS recommends using NuxtJS for these use cases.

With NuxtJS, we got server-side rendering out of the box. The framework also has multiple plugins for SEO. We could now use our VueJS components to build the website and additionally get SEO as a freebie.

To help us build a highly available site, we explored JAM Stack. JAM is a popular method of building websites where the entire website is pre-rendered and then enhanced via Javascript and APIs to make a performant, secure, and highly scalable website.

NuxtJS can pre-render the entire website instead of hosting it on top of a node app. So, instead of rendering a page on the fly based on a request, it can render all the pages during the build time. Data in the marketplace does not change often since apps are published or updated a few times a day. Also, this avoided the need for a live server that serves requests on the fly, and helps in serving at scale. Pre-rendering our website was a good fit for our use-case. It was okay for the website to lag a little bit behind the gallery to optimize for serving at scale.

This meant that with the right configuration our site could be performant and highly scalable. As the website serves mostly static data for apps, this was an excellent fit for our use-case.

With so many out-of-the-box features and advantages over other solutions, we decided to go with NuxtJS to build our website and serve a great User-experience.

🙌 Building a Mono-repo to serve two experiences

NuxtJS follows the principle of “Convention over Configuration”. This means that it follows a convention of placing files in specific directories in the project to automatically create web pages. There are specific folders for pages, middleware, store(for State Management), etc. that will help in building the website with minimal configuration. As a bonus, NuxtJS can be used as a dependency for any existing project! 🤩

Our existing project was built with Vue CLI and looked like the following:

We kickstarted it all by adding NuxtJS as a dependency in our project.

npm install –save nuxt

The proposed UI for the website was slightly different from the in-product gallery. The differences were mainly around the layout while using similar components. We planned to maximize the reuse of code while giving enough flexibility to build features exclusively for one of the experiences.

It is very important to have maintainable code especially when your code is responsible for delivering multiple experiences at scale. Over the years, the codebase tends to get complex with multiple features integrated by multiple people on the team. A good base architecture for the codebase helps in keeping it maintainable.

To help us manage the code better across our two store-fronts, we had to make a few early choices:

1. Specific Layouts for Pages: Pages would be specific to each experience while components can be common to both. As the layout in the website differs from that in the in-product gallery, it makes sense to decouple them and only reuse the building blocks(Vue components).
2. No global state in components: Components should not have access to the global state of the app directly. Each component should be a unit independent of the global state so that it does not cause conflict when used across the experiences. This makes components truly reusable and avoids coupling. Each component can store its own copy of the state via props that will be passed from the parent component.
3. Manage APIs specific to each experience: The in-product gallery and the website will have their APIs managed differently to decouple the data layer for the apps. This was important as the website and the in-product gallery would talk to different APIs with different authentications to fetch and store data. A tight coupling of data fetching logic in both experiences makes it hard to incrementally add features to one of the experiences. 
4. Customizable component styles: Each common component, although functionally similar, might have different styles based on where it is used. Styles may be different to match the style of the experience it renders in. For example, the same card that displays short information about the app may look different on the website and on different products. Also, it may look different on smaller screens like mobiles and tablets. The component should be built in a way to accommodate these customizations. A component should be able to adapt to the global context and apply styles specific to where it’s rendered.
5. Understand the impacts of a code change: The code should be written in a way so that it is easy to identify parts that are common and parts that are different in the two experiences. We can then preemptively find out the impact of a code change.

With these in mind we came up with the following structure:

root

|–src/

|—-common/ /* All common components, apis, styles etc. go here */

|—-gallery/ /* All gallery code goes here */

|—-website/ /* All website code goes here */

|– /* other folderrs and files. */

|–nuxt.config.js

|–vue.config.js

With the above convention in place, we started adding pages, store, components, and other directories to the website folder. We then specified that our root for the NuxtJS project would be “src/website/”. This will tell NuxtJS to build the project using this folder.

This clear segregation also helps our QA team to find the impact of a code change. For example, if a change occurs in the common folder, both the in-product as well as the website experience may be affected. A code change in the website folder, on the other hand, will not impact the in-product gallery.

We were finally able to come up with a structure to maintain a clean and flexible repository that has continued to serve us well for over a year.

👍 Secure and always up!

It was of prime importance that we maintain a stable website with almost zero downtime. As stated earlier, this is the reason we went with JAM stack approach.

The website we were planning to build was mostly a static one that lists various apps we host. We deliberately focused on minimizing dynamic components that would fetch data in realtime. This allowed us to pre-render the entire website and store the same as assets in Amazon S3.

This gave us a number of advantages:

1. Cost-effective: No additional computation cost to run a live server.
2. Scalable: As the website can be hosted via S3 without the need for live APIs, it is highly scalable with no impact to growing traffic. The Amazon CDN handles it all gracefully.
3. Secure: This solution is immune to security vulnerabilities such as DDoS attacks as it is just a read-only site completely protected by the CDN.

🏗 Building the pipeline

All our projects use Continuous Deployment pipelines to automatically deploy our apps to production. This helps us in shipping features faster and makes the process less error-prone. We already had a CI/CD pipeline for deploying the In-product gallery. The website demanded a separate pipeline as the build process was very different for NuxtJS.

As described earlier, the experience for the website may not change every day but the data it renders (the app listing) changes at least once a day. So, we decided to build the web pages every day at a certain time. A new app that went live would appear in the next day’s build. One always had the option to trigger a build on-demand if the new listing needed to be reflected or fixes or improvements needed to be deployed. We configured a CRON job that would trigger a build every day to keep the data as fresh as possible.

A brief look at how the pipeline operates is shown below:

👩‍🎓 Learnings and takeaways

We made the marketplace Website live in April 2019 and it has been running without any downtime ever since.

We have continued to add a number of features on top of the website and In-product gallery. The following are a few key learnings: —

1. JAM Stack proved to be powerful for fetching and aggregating data from multiple sources. As an example, when we added App Recommendations as a feature on the website, the source of the data was originally a spreadsheet generated using automation from a data science tool.
2. De-coupling the UI and the data layer is a good practice that avoids complications as changes pile up over time. All the components have their own interfaces as props or attributes. So, data is passed through props and data is emitted through events. This also helps in maintaining clean code.
3. Server-Side Rendering(SSR) boosts First Contentful Paint. It’s easier with NuxtJS to SSR the webpage and send the Javascript later to hydrate the app. This makes subsequent navigation on the site very much like a single-page application. This, truly, is the best of both worlds.

So we were able to deliver and maintain two experiences effortlessly using the pipeline. NuxtJS is really easy to pick up and build a scalable website. With over a year of working with this, we are yet to hit any serious bottlenecks. We will certainly consider using JAM for our next static project! It’s a cost-effective and secure way to deliver stunning experiences!

[An earlier version of this blog by Ahmed originally appeared on Medium.]