Transforming the lives of sales agents 1 second at a time
As a sales agent, Sam had gotten used to waiting. It was one of the attributes of his job, to wait and win. It wasn’t easy, it wasn’t the most productive way to go about his tasks, but Sam had resigned to it. Until recently.
What typically happens is when prospective customers visit his company’s website or discuss the company’s products with a chatbot, a customer relationship management tool gathers their details. The CRM then evaluates the prospects and assigns them scores indicating the likelihood of them converting into paying customers. The high-scoring ones are priority. Sam and his colleagues would then need to chase those prospects in a hurry.
Now let’s transport you into Sam’s chair. You are a high-performing salesperson, a go-getter. And there’s Jenna, who is searching online for a business product that your company happens to be selling. Her search leads her to your company’s website. She’s interested and would like to know more about your product. So she fills out a form on your website. A prospect, yay!
But you have to weigh your choices from among the hundreds of prospects who come your way every day via channels such as emails, phone calls, chatbots, or your website. You need to prioritize. But what happens is once Jenna’s contact details are fed into the CRM, you end up waiting a few hours for it to throw up her score.
Jenna’s not twiddling her thumbs meanwhile. She’s looking at the other products her search threw up and is filling out more forms. If she’s serious about buying that product, you need to know in a hurry.
In short, that’s what this blog is about. How we helped Sam become more productive and highly efficient by enabling him to pitch to Jenna in real-time, making a significant impact on his sales pipeline.
There’s always a problem to solve
Let’s talk about how those scores are arrived at. It’s called ‘contact scoring’, a process by which the CRM evaluates various inputs to predict the likelihood of a prospect, or a ‘contact’, converting into a customer. It can be done in several ways.
One is the so-called rule-based score, a simplistic process that does not factor in behavioral patterns. Machine learning-based models are more reliable. But these require complex computations and are typically run as batch processes, which means predictions are refreshed periodically and cannot be delivered in real-time. This deteriorates user experience.
Real-time is hard
Machine learning-based contact scoring in real-time was a priority feature when we were building Freshworks CRM, our most advanced and most comprehensive customer engagement tool yet. Freshworks CRM captures the entire sales journey of a potential customer in a single place instead of capturing them in silos across sales, marketing, and other functions.
Contact scoring in Freshworks CRM is based on two types of scores:
- Fit score, which relies on factors such as a prospective customer’s name, job title, his/her company’s business type, country.
- Interest score, which is based on the interest shown by a prospect, say, in terms of the number of calls or email inquiries.
Since both ‘fit’ and ‘interest’ scores are required to compute the final contact score, every contact score has two machine learning model predictions. Say hundreds of contacts are from the same company and its property (e.g., its headquarters) changes, this single update will result in hundreds of contact score predictions. The mere number of predictions for every change in data makes contact scoring a very complex system.
A CRM registers hundreds of emails and phone calls from potential customers every day. To calculate the interest score for a prospective customer, our machine learning model on Freshworks CRM considers their interactions over the previous 6 months. This is done for every single prediction. For these predictions to be made in real-time requires a complex and data-heavy machine learning system—a huge challenge.
Building multiple ML models
The sales cloud of Freshworks CRM, launched during our annual user conference, Refresh, is expected to have around 100,000 customer accounts in its first couple of months.
Contact scoring on Freshworks CRM has customer-specific machine learning models (fit and interest) for every customer who has enough historical data to warrant their own models. To make the contact scoring feature ‘globally available’, there are global machine learning models for accounts that do not qualify to have their own models. These global models are built from data across multiple accounts covering multiple industry verticals.
As new data keeps entering the CRM, all these different machine learning models need to be rebuilt to make the predictions more accurate and relevant to the recent data trends. When multiple models are built in parallel, monitoring every model’s progress and retrying and alerting on failures become basic requirements. Monitoring and retrying should be fully automated because of the sheer number of machine learning models that are re-trained and re-built.
SageMaker to the rescue
In contact scoring for Freshworks CRM, we use AWS SageMaker for model-building, which helps in completely automated multi-model builds.
Once the online prediction system is up and running, refreshing machine learning models with the newer models should be seamlessly done without affecting online predictions. Once the model in s3 is replaced with the new model, the SageMaker endpoint automatically picks the new model without having to do much. If not, a regular and periodic refresh of machine learning models is not possible.
In contact scoring, we use SageMaker for online predictions as well. Depending on which account’s contact score needs to be refreshed, different account-specific model-based predictions need to happen in parallel. For real-time predictions using multiple models, the requirement is for the models to be always deployed and for the frequently used models to be cache, which is done by SageMaker.
Streaming-based online database instead of data lake
It is mandatory that the real-time system is aware of any changes in the data immediately. Changes take time to reflect in a data lake. So a data lake cannot work as a backend for real-time predictions. The contact scoring system uses the Kafka streaming platform directly for its data requirements. A new message is sent in Kafka for any change in contact data. The contact scoring system looks for such messages in Kafka and displays a score on the contact’s user interface within a second of its creation!
When the dependence on the data lake is less, the system is not limited by factors such as resource availability or delays in data sync. To have a datastore that is not shared, we built a centralized online database, based on Amazon Web Service’s Relational Database Service, that we call Freddy DB. (Freddy is Freshworks’ AI platform.)
This is a dedicated database for Freddy projects and allows for optimizations for Freddy use-cases. The system uses the data lake only for one time backfill of data. The data is then kept up to date using Kafka.
It is imperative for data changes to be persisted in Freddy DB before they are picked up by the contact scoring system or any other Freddy system. So we created our own Freddy topic in Kafka which contains messages that have been persisted in Freddy DB.
So all Freddy projects can now consume messages from the Freddy Kafka topic instead of processing directly out of the topic to which Freshworks CRM’s sales cloud produces. Because of such a design, all errors in the data get filtered in the Freddy DB system instead of propagating to multiple Freddy systems.
To keep data generic across various Freddy projects, both Freddy topic and Freddy DB follow the same schema as the Freshworks CRM sales cloud.
Challenges using a streaming platform in building a database
Depending on a streaming platform for data comes with its own challenges. If the source of the message is not idempotent-enabled, the contact scoring system at the receiving end should be able to handle the same message being replayed due to retries at multiple levels–producer, Kafka server, consumer, etc. To make the contact scoring system idempotent, the updated time of every record is stored in Freddy DB. If the message being processed is older than the record in the database, it is dropped.
Another challenge with a distributed streaming platform is that it can only guarantee the order of messages at the source. While consuming, the order of the messages can be maintained within a partition but not across partitions.
How can we handle out-of-order messages? Isn’t dropping the older messages enough to take care of this too? No! Not with a simple CRUD-based DB. While different properties of a particular contact are updated, it is possible that they end up in different partitions. Dropping older messages could mean losing updates on a few properties.
The contact scoring system utilizes the fact that all contact properties are sent in Kafka for updates too. For every update, the entire record in the database is replaced. This way, dropping older messages is completely fine and out-of-order payloads are taken care of.
Stay away from noise
Freshworks CRM has a lot of marketing emails sent by sales agents to potential customers. These emails are huge in number and can suddenly storm the Kafka message queue. If the system were to treat them like any other email and refresh the scores on receiving each of those emails, valid updates will also start queuing up.
For that particular period, the system won’t be in real-time. These emails do not represent the contact’s interest in the product. So the contact scoring system skips all marketing emails without refreshing the scores upon receiving them.
A strong support system
Because contact scoring is a real-time system, any issues or errors have to be identified at the earliest. There are multiple levels of monitoring and alerting deployed for both the online and the offline training system. New Relic is the monitoring platform for contact scoring. New Relic alerts are set on error percentage, throughput, latency, etc.
Apart from this, AWS Cloudwatch alerts are set on the CPU and the memory of the machine in which the application is deployed. Alerts are set on a few specific error logs in Haystack. At the fourth level, an alert is also set on the lag of the Kafka message queue. All these different alerts make the contact scoring system reliable at all times.
When a basic user of Freshworks CRM upgrades to a paid plan, the system has to be ready to start predicting scores immediately. To support this, the contact scoring system doesn’t differentiate between different plans in the backend. Scores are computed for contacts of all accounts irrespective of the plan. The restriction to show scores only for paid plan accounts is applied on the UI level. So the contact scoring system not only handles contact updates but also subscription upgrades in real-time!
Scale, scale, scale
A real-time system should be able to scale instantaneously with any increase in load. When more customers sign up or more prospects are entered into the CRM, the contact scoring system should easily adapt to the increase in input traffic. Freddy DB is shard aware and the addition of a new shard is as simple as clicking a button.
Freddy DB is sharded based on a customer’s account. New accounts are automatically mapped to the latest shard available. When the number of shards has to be increased, another shard just needs to be added to the configuration and the system directs all new accounts to the newly created shard.
To take on the scale of Freshworks CRM, all Freddy DB tables are partitioned and indexed. Because Freddy DB uses RDS, the addition of read slaves to handle the increase in read load can be done instantaneously. If the rate of incoming payloads increases in the future, it is possible to increase the number of partitions in the Freddy topic and correspondingly increase the number of consumers, thus scaling seamlessly whenever necessary.
And we did it
In the Freshworks CRM contact scoring system, extracting fields from Kafka messages, reading the related data from the database, cleaning the data, computing complex machine learning features from the raw fields, making two machine learning model predictions, determining the final score, and sending the score back to Freshworks CRM—everything is kept within a second in 75% of the cases.
System stats for one hour
The future: Scaling up for heavy improvements
We plan to improve the predictions by taking signals from other related clouds in Freshworks CRM such as the marketing cloud. This would mean that messages from many more Kafka topics would have to be consumed. Our Freshworks CRM contact scoring system is designed to scale to that amount of traffic. The sentiments of the contact based on conversations on chats, call logs, and as recorded in agent notes, are also to be included in the predictions in the future. Text cleaning and processing is a time-consuming process. But the plan is to keep the contact scoring system real-time even with these heavy improvements.
