How we aced the challenge of building consistent multi-product APIs
At Freshworks, we proudly build SaaS applications in the CX (Customer Experience), CRM (Customer Relationship Management), and ITSM (IT Support Management) spaces. Our multi-product offering ranges from Freshdesk, our flagship product, to Freshworks CRM (FCRM), our newest addition. Currently we have more than 200 Public APIs across all our products, documented in our developer portal and used heavily by our customers for their automation needs.
Public APIs, like diamonds, are forever – Joshua Bloch
The Problem Statement
As we expanded from a single SaaS application (Freshdesk), to multiple products across different product domains, we had already invested in ensuring the user experience across all our products was consistent. We similarly needed the developer experience of using our APIs to be consistent, reliable, easy to use and fault-tolerant. The following sections detail our engineering efforts in building better multiproduct SaaS APIs.
Building consistency across a multi-product engineering setup, within a fast-growing company like Freshworks, is easier said than done. We needed a strategy that is simultaneously comprehensive, long-term and incremental and could scale to meet the needs of any engineer within Freshworks building and releasing APIs.
At the outset, we decided to focus on the following areas when it comes to API design, development and consumption that were otherwise handled in myriad different ways within engineering silos at Freshworks.
In addition to web-based interfaces served through browsers, our customers interact with products via Application Programming Interfaces (APIs). Like the web interface, the APIs also have User Experience (UX) elements associated with them. Making the APIs isomorphic, having a similar look and feel is one of the core tenets of designing intuitive APIs.
To improve and facilitate easier consumption, the APIs should be easily discoverable and today our developer portal solves this well. The portal is the source of truth and a one stop shop for finding available API resources along with details regarding the request and response for each API endpoint. We needed a similar approach for APIs that we choose to keep private, and facilitate faster integration through internal discovery, better reuse and maintenance.
Lingua Franca for API definitions
Teams within Freshworks each happened to use different mechanisms to document and share their APIs. Some of them relied on the public API documentation published on our developer portal, while others used specifications like JSON API and OpenAPI for communicating their structure. Besides realizing the need to establish a standard here, we also realized that communication across teams would become much simpler, once everyone spoke a single language (format) when dealing with APIs.
Anyone who ever published an API knows all too well that once the API is public, one has to keep supporting it for eternity, while making sure any updates are always backward compatible. This often risks incurring a recurring development and maintenance cost that can continue for years. To give you a perspective, Freshdesk moved to API V2 a couple of years ago, but we continue to have some of our clients still using our deprecated V1 APIs.
While this might be the most obvious form of API maintenance overhead, it extends equally well to our Private APIs. Our experience shows that there is a high probability that a private / internal API will be made public tomorrow. This goes to prove that our Private APIs also need to have the same level of rigor as our Public APIs. Considering this cost of building APIs and the long term costs of maintaining them, the best time to make sure our APIs have the desired UX is even before they are implemented (API-First Design Approach) .
The API 360 Process
The strategy to solve the above issues required creating a governance program around building and maintaining APIs, that focussed heavily on automation, tooling and defining standards. This resulted in the creation of the Freshworks API 360 initiative. Let’s take a closer look at the important aspects of this program that are in place today and applied to every new API coming out of Freshworks Engineering.
API Contract Language
As discussed earlier, we wanted to start by establishing a lingua franca for API specifications at Freshworks. The basic requirement was for a format which is simple and yet versatile enough to cover all our API needs. After considering multiple alternatives, we settled on “Open API 2.0” for writing and communicating our API specifications. Being a mature, open standard, OpenAPI 2.0 also had advanced tooling support built around it. We chose “2.0” Open API standards as the latest version (“3.0”) at that time was still in its infancy and the standard was not finalised.
Single Source of truth
Like most organizations trying to solve this problem, API Discoverability was one of the core problems we faced while scaling the program to all our products and platforms. Having a single system to capture the specifications is just one side of the coin. The other side is to make sure all of them are available in one place. The API specifications our engineers write in OpenAPI 2.0 are committed to Github and via our continuous integration (CI) pipelines get published internally.
Common definitions and styles
The Contact entity exists in both Freshdesk and Freshservice and has a similar behavior in both the products. A developer consuming this resource from Freshdesk, Freshservice or Freshworks CRM should expect similar contracts, naming conventions, behaviors when building an application around this resource. To facilitate this kind of consistency in resource definitions, we extracted common objects from high-level resources and established what the common specifications ought to be within Freshworks. Entities created by products would in turn use these definitions and build higher order resources on top of them.
Because the products now share these interface definitions, the resource APIs as seen by the developer user will remain consistent across all our products. This brings in familiarity in the resource structure and has a positive effect of flattening the developer learning curve as they move across our product suite. Any application built on top of these common resources can be easily migrated to other products, if required.
Treating our API specifications in line with standards for our source code, we decided to also introduce an API review process to help catch most of the errors and anomalies at an early stage. An internal API group reviews all the specifications before they are committed. As part of this review, we make sure the APIs follow standards like Open API best practices, Freshworks resource conventions and proper usage of common objects.
This paves the way for improving the API UX in the design phase itself, even before the implementation begins. An important side-effect of this review process is that most of the mundane contracts are codified as lint rules which get executed as part of the CI pipeline, processing each API update and sending early signals back to the API developer even before humans get involved.
Once the API definitions are available in a single place and a single format, we have internal tooling (built on top of open source offerings) to generate test suites, mock servers and mock clients from them. The Front end engineers and backend developers and SETs don’t just have a contract to work with, they are also able to use these generated artifacts to build their applications on top and test them.
As an example, a front-end engineer can use the mock server to simulate a backend while building their web application UI. The mock server returns the required (mock) responses, which facilitates the FE development cycle. Once the feature is complete, the FE artifacts are integrated with the production implementation to complete the cycle. The SET team also benefits from the tooling, by building automation tests on top of the generated test suites, rather than building them from scratch.
These tools help each team to work in parallel when building their components and also test them locally, by integrating with this mock artifact. This increases engineering velocity, enabling integration issues to be caught earlier in the lifecycle and to deliver the features built around these APIs into the hands of the customer faster. The deployments are now equipped with a fully integrated automation suite, even before they are deployed.
We find a growing appetite for not only our products at Freshworks, but also their APIs, as we expand our customer and developer base. As explained in this post, our API governance process has been critical to extending a uniform experience—both externally to our customers and internally to our engineers—around the APIs that power our SaaS products and applications built around them. While this post focussed on the high-level problem statement and how our API 360 process has attempted to solve it thus far, we plan to follow up with a series of posts, explaining in greater detail, the open source ecosystem we rely on and the internal tooling we built on the way during our “API 360” journey. Stay tuned!
Subscribe for blog updates
Thank you for subscribing!
OOPS! something went wrong try after sometime