Cracking machine learning to ace sales forecasting

Why is sales forecasting important?

“The most vital element of every business is Sales. It’s what keeps a business moving.”

Thousands of businesses are using Freshsales, the CRM software from Freshworks, to manage their end-to-end sales processes, including identifying new customers, pitching their products and services, negotiating a sale, and closing deals. Given that sales operations directly impact inventory management, manufacturing, and several other aspects of a business, companies need to plan upcoming sales activities well ahead in time. Sales, anticipated well in advance, can help a business optimize their GTM operations and build an efficient sales pipeline.

As a CRM provider, we care the most about helping our customers meet their sales targets, and moreover, unlock new efficiencies in their sales processes. While doing so, we realized the crucial requirement of sales forecasting for our customers, and predicting the time of closure of sales deals came up as one of the first problems to be addressed.

Since sales of a business can get affected by many factors in the world, including both internal and external, making an accurate prediction is a challenging problem, and we have built a solution for this as part of Freshworks’s AI platform Freddy.

 

What are the challenges in forecasting?

“Data is the problem, Machine Learning is just a tool.”

  1. The most fundamental problem in sales forecasting is not knowing about the factors that actually affect a sale. Although we can look at different kinds of data based on historical sales, predicting future sales and revenue accurately is challenging. A simple example would be the Covid-19 pandemic that has had the worst-ever hit on the global economy.
  2. Another major issue to draw predictions in CRM is the uncertainty of data. Though Artificial Intelligence is now a common term in many sectors, it’s not very mature in CRM. This domain mostly involves manual effort in entering customer-centric data. We don’t get all the required data in every scenario, and sometimes when we do get it, the quality of data is questionable.

To address these problems, we collect as much data available in CRM and draw different relations using it. That is when the term Big Data came into the picture for us.

 

What is deal forecasting?

A sales deal is a set of activities between sales agents and potential customers. To predict the sales revenue of a business, it is crucial to analyze all existing sales deals of that business. Hence, we formulated a sales deal as the unit of interest and built a system to forecast whether a deal is going to close in the next week, month, or quarter.

This also normalizes our data challenges, because every insight is drawn concerning a deal. And this gave us a semi-structured way to look at the data we already possess.

 

How do we make data useful?

“No data is useless, perhaps you are not looking at it correctly.”

Sales agents use a CRM platform to perform their daily sales routines e.g., identifying new leads, following up with contacts, negotiating a deal, etc. Every action they take to make a successful sale is captured and stored in the data lake. Following are the basic engineering exercises we perform to make this whole data valuable:

Activity associations

Since a sales deal consists of one or more leads and contacts, every activity associated with these leads and contacts can be associated with their corresponding deals. To make this clear, let’s consider an example:  John, a sales agent from Freshworks, gets a sales lead Bill, an IT manager from XYZ who is searching for an ITSM software for his team. So John tries to make a deal with Bill for Freshworks’ ITSM product “Freshservice”. Accordingly, Bill is added as a contact in Freshsales and a Deal-K is started. Now every email conversation that happened with Bill (Contact) becomes part of Deal-K.                                                                                                                                                                                                                                                                                                     

Weekly feature extraction

We divided our data into three major categories:

  1. Sales activities: conversations, status updates of deal, tasks, etc.
  2. Static and history data in CRM: product, deal amount, sales agents’ previous sales, etc.
  3. External data: demographics, company information from third party data sources

Thousands of sales activities are recorded every second in the CRM and it impacts a deal’s progress the most. These activities are aggregated at a weekly level to accommodate massive activity data in limited size. Several other dependent features of deals e.g., deal’s acceleration and degree of conversations are also derived on a weekly basis. 

For example, if John and Bill share 5 emails in a week, John also completes 2 tasks of quotations and moves the deal into the negotiation stage. All these activities become part of one feature record identified by the combination of sales deal and week number.

Sales trends of a product or a region, information about a client involved in the sale, track record of sales agents also play crucial roles in the deal’s closure. However all of this data is not available in every deal, and we use inputs from third parties to enrich CRM. All this data is combined with the activity data to get final features, which we term as weekly sales features.

Text sentiment analysis

Most of the insights for a deal can be derived from the conversations between sales agents and their potential customers. Sales agents also keep notes to track the deal’s progression and their commitments for the same. After the authorization from our customers, we use this textual data to create more influential features.

Freddy’s NLP platform and deep learning models are exclusively trained on a large variety of text data and are able to extract human sentiments from text. This platform aggregates text data for a deal and generates sentiment analysis towards a plausible closure. The NLP platform also retrieves hidden information about deal’s static data if not available directly in the CRM.

Since data in CRM is very sensitive to our customers, we make sure that the data is secure with the best standards, and it is only used to serve themselves but not anyone else. 

 

How do we engineer data at scale?

“The only way to learn about big data is to work with it.”

Extracting features from different data sources is a complex data pipeline and consists of multiple stages that require continuous monitoring. We needed to process 3 TB of data that was already present in CRM to build the foundation of the forecasting system, and then 100 GB of new data every week in order to generate weekly features repetitively. To accomplish this, we built our data engineering pipeline using Apache Spark.

The aforementioned data engineering pipeline consists of following building blocks:

  • Retrieve leads and contacts associated with deals and their active time periods. At Freshworks, we use terminology of sales journeys to define these associations.
  • Associate activities in sales journeys with the deal. Filter out redundant activities and summarize data of every week. Collect deals’ progression over its journey and derive dependent features. 
  • Read static and history data from CRM and external data sources. Combine this with activity features to generate weekly features.
  • Retrieve text data related to deals and derive sentiment features on the same. Once sentiment features are combined to weekly features, this data is dumped to a distributed file system to train ML-models and generate batch predictions.

Building a scalable and highly efficient Spark pipeline requires a number of optimizations and configuration tunings. After such optimizations, our data engineering pipeline, which took 18 hours in the initial phase, now completes within 2 hours. Details about spark optimizations are beyond the scope of this blog, so I will be covering those in a separate article.

 

How do we manage multi-tenancy in ML systems?

“A feature is substandard, if it doesn’t support customization.”

Every business follows different practices to meet its financial goals, and so, deal forecasting is unique for each of them. Besides, business accounts in Freshsales vary across a diverse range of industries, regions, and employee sizes, which can change the sales behavior of a company.

To provide the most reliable forecasts to our every customer, we planned to build a unique and customized ML system for each of them. But the biggest challenge was managing thousands of ML workloads together. It’s crucial to monitor every ML job and validate the models before deployment, also to make repetitive improvements in every model without disturbing the customer experience.

As we explored different ways to tackle these situations, AWS Sagemaker seemed to be a perfect choice. In addition to this, the serverless architecture came as an additional advantage.

The overall ML-pipeline can summarized as follows:

  • Spark pipeline generates datasets for every CRM account which undergoes customized preprocessing, training and validation in AWS Sagemaker.
  • Upon successful validation, the model is deployed in Sagemaker for both batch and real-time predictions. Predictions are sent to the CRM portal via Redis but are also stored in a SQL database for analytical purposes. 
  • Every business account’s ML jobs are run concurrently. As a result, the overall runtime of the ML-pipeline is brought down to 3 hours from the initial benchmark of 36 hours.

 

How do we make forecasting real-time?

“A product is driven by the response time of its features.”

As soon as sales agents make progress in a deal, Freddy’s forecasting reflects an immediate change so that our users can make quick decisions to achieve their sales targets. Following steps outline the overall process of real-time predictions.

  • Deals’ weekly features are stored in a NoSQL database to handle high scalability and low-latency retrieval of data for real-time processing. 
  • Activities performed by sales agents and corresponding changes in deals are captured in the Freshworks’ central Kafka. Once the payloads in Kafka topics are consumed, deal features in the database are updated.
  • All account’s ML-models are deployed in AWS Sagemaker as a multi-model endpoint. As soon as a feature is updated, this endpoint sends a new prediction to the CRM. 

 

How do we monitor the performance of ML models?

“Continuous improvement is the best strategy towards perfection.”

  • Online validation: We monitor forecasting results sent to users and deal closures in near-real time. Performance metrics of all the deployed models are updated using this data and are then sent to analytical dashboards. With this, we make sure that our customers are served highly efficient ML systems.
  • Feedback listener and A/B testing: We use Freshworks’ in-house feedback listener platform to analyse the business impact of Freddy’s sales forecasting. This platform records deal commitments, closures, and reviews from sales agents and builds a profile for the background ML system. It also supports A/B testing when multiple models are served for the same business. This helps us to compare different models using their overall returns on the businesses.

 

What’s next in sales forecasting?

  1. One of the best ways to address forecasting is to explore new data sources and extract hidden features from them.
  2. Every business invests in financial planning. Hence, all our customers deserve this feature, whether they are SMBs, MMs, or enterprises.

The Freddy AI/ML team at Freshworks is continuously engaged in efforts to make AI more accessible to businesses and we thrive on the challenges associated herein. The sales forecasting initiative is one step towards achieving this goal.