My Work on the Recipe Recommender System for the Thermomix on Cookidoo

I built the first iteration of the recipe recommendation system for Cookidoo, the recipe platform for the Thermomix, with a fantastic team. Cooking is such an important and emotional topic for many people, and I was excited about the idea that some 5 million people could get inspiration from the system we built.

These articles were first published on the Alexander Thamm blog: Part 1 and Part 2. Both parts are reproduced below. They describe the system and our plans at the time of writing in 2022, not Cookidoo today.

• Part 1: Model Selection
• Part 2: Candidate Selection, Scoring & Ranking

Part 1: Model Selection

Screenshot of the “For You” section on Cookidoo.de, taken on December 7, 2021.

In the screenshot, you can see some of our personalized recommendations on Cookidoo, the recipe platform for the smart cooking device Thermomix. In this post, I want to provide you with practical insights on building the recommender system for Cookidoo. To do this, I’ll walk you through the steps and describe the challenges we faced, the choices we made, and the lessons we learned from building the first iteration of our recipe recommender system.

The Challenge

Cookidoo is the recipe platform for Vorwerk’s digitally powered cooking appliance Thermomix. At the time of writing, over 5 million users could find over 80 thousand recipes in twelve different languages on the Cookidoo platform. One of the central values of the platform is to enable users to find and cook recipes they love. Cookidoo has an excellent team of content creators and some simple rules for curating recipes automatically, such as top trending recipes. But users have different tastes and highly value personalized experiences (also because they are used to recommendations from services such as Spotify and Netflix). Having personalized recipe recommendations at their fingertips can be the difference between a platform the users love and a suboptimal experience where users feel that “there are no recipes for me.” Furthermore, a well-crafted personalized experience can reduce meal choice and preparation efforts during busy days, inspire users with new culinary ideas, and support a healthy diet. Accordingly, our team’s vision is to make your daily “What to cook?” decision easy and joyful.

General Structure of Recommender Systems

People are often under the impression that recommender systems are end-to-end machine learning models that you throw data into to get out polished recommendations. That couldn’t be further from the truth. Modeling is just one of many steps in building recommender systems. Also, surprisingly, these steps usually include a large number of rule-based filters and business logic.

For our recommender problem, it helped us to divide our system into three components conceptually:

• Model Selection: Define training data and find the best model for your use case.
• Candidate Selection: Select items that you consider a candidate for a recommendation.
• Scoring & Ranking: Score all user-item combinations and create an individual item ranking for each user.

Next, I’ll go through each of our components in more detail. In Part 1, I’ll cover Model Selection and tackle the remaining components in Part 2.

For a more comprehensive overview of the components of recommender systems, I highly recommend this talk by Even Oldridge, the blog post by Karl Higley, as well as System Design for Recommendations and Search by Eugene Yan. However, keep in mind that most of the information on recommender systems comes from companies and experts facing challenges at the scale of billions of users and items. So, be skeptical about adding components, methods, and technologies to your system and thoroughly evaluate what you need to create a valuable experience for your users.

Model Selection

Use Interaction Data Over Explicit Data

For recommendation problems where you want to recommend items to users, you can get a good 60–80% solution by using some variant of a matrix factorization¹ model on interaction data.

Interaction data is data on the interactions between your users and items. In our case, this includes:

• User A cooked recipe Gazpacho.
• User B added recipe Asparagus and Mango Salad to his cooking list.
• User A viewed the recipe details page of the recipe Vegan Bean Burgers.

Simple cooking interactions might look like this in our data:

datetime	user	recipe
2020-07-14 09:59:28	User A	Recipe 6643
2020-04-30 07:42:56	User B	Recipe 6493
2020-06-15 14:51:44	User C	Recipe 7393
2020-04-13 14:30:24	User C	Recipe 8404
2020-06-15 18:27:12	User D	Recipe 8389

This interaction data is much more valuable than ratings, so if you have both, always start with the interaction data! We prioritized the cooking data and planned to experiment with additional interaction data soon.

From the beginning, a big concern for us was to keep user data private, secure, and, as a minimum requirement, GDPR compliant. Of course, good privacy practices can lead to higher efforts. However, building them right at the start is absolutely necessary and reduces efforts in the long term. For example, ask yourself:

• What user data do I need to store, and for how long? How can I minimize my need for data?
• How can I ensure that the users and their interactions stay anonymous?
• How can we delete data when users opt out from data sharing?

Training Data Selection – Global vs. Local

We had three main options for selecting interactions to train our model on:

1. Training one global model with all cooking interactions
2. Training one model per language
3. Training one model per language and geographic region (e.g., one model for interactions of English-speaking US users, another model for English-speaking users from the UK)

Cooking behavior is very different around the world, and localizing a recipe is not as easy as changing audio tracks or adding subtitles in the case of movies. Furthermore, on Cookidoo, most recipes only exist in one language. Even for familiar recipes like pancakes, ingredients and how ingredients are balanced might be highly specific to a language and geographic region. This rules out option 1.

On the downside, this implies that we cannot leverage the full potential of our data for training the model. After all, taste clusters like vegetarianism or a “taste for curry dishes” are stable across regions. Furthermore, limiting training data by slicing by language (and geographic region) makes getting good results for languages with small user bases incredibly challenging.

We chose option 2, training one model per language, to balance these two concerns. In addition, to respect each user’s regional preferences, we filter out recipes that don’t fit their region or measurement system (metric vs. imperial) in the candidate selection stage.

Choosing an Implementation – Start Simple

As we wanted to release the first version of our recommender to users as fast as possible, we evaluated two open-source packages, Implicit and LightFM. The Implicit package implements a very efficient version of the matrix factorization algorithm for interaction data (for algorithm details, see this paper: Collaborative Filtering for Implicit Feedback Datasets). The LightFM package also implements a matrix factorization algorithm but allows including side information (e.g., recipe categories or recipe tags).

In the end, we went with the LightFM package. It was vital for us to get a decent working solution as fast as possible and iteratively improve from there. Both Implicit and LightFM work pretty well out of the box and implement matrix factorization efficiently, so you don’t have to worry about performance too much. However, in contrast to the Implicit package, LightFM allows adding user and recipe metadata to quickly improve upon simple matrix factorization.

Deep Learning Doesn’t Automatically Equal Better Recommendations

Deep learning has lately gained lots of popularity in the recommender field. However, you should only expect significant recommendation improvements from deep learning when using massive interaction datasets, lots of context data, or unstructured metadata such as pictures or texts. Another problem that deep learning solves well is recommending the next item in a sequence (e.g., click-through prediction in online advertisement or next-item recommendations in browsing sessions).

Of course, there might be other reasons to use a deep learning framework such as PyTorch or TensorFlow, e.g., a fantastic ecosystem around model monitoring, versioning, and serving. However, we were convinced we would make faster progress with LightFM, and switching later to a more expressive and flexible DL framework is always possible.

On Spark ML

When starting our recommender journey, we were restricted to an on-premises Hadoop cluster and used the spark.ml package. I wouldn’t recommend going this way. Matrix factorization is pretty inefficient in Spark due to the data shuffling overhead, and you’ll probably train your model much faster on a single node with the packages above. Often, when you think that you have too much data for a single machine, you are okay after properly preparing your data and converting it to a sparse format. In my experience, distributed machine learning is seldom worth it and creates much unnecessary complexity.

Model Evaluation – Don’t Worry Too Much About Offline Evaluation

There are two evaluation modes for recommender systems: offline evaluation and online evaluation. Offline evaluation is analogous to evaluating other machine learning models (like classification), i.e., you hold out test data and evaluate the models’ predictions on this test data. You try to evaluate your recommendations without actually showing the recommendations to anyone. In contrast, online evaluation tests the model live on your users, usually through A/B testing.²

My key takeaway for offline recommendation success measurement is not to worry too much about it. We spent too much time deliberating on the best offline metric to use when we should have invested the time elsewhere.

Some guidance I can offer from our experience:

• Don’t rely too strongly on offline metrics. It is difficult to interpret these metrics and even more difficult to relate them to your business problem. Instead, use offline metrics only as a directional indication for comparing your different models.
• If you start with recommender systems, use either NDCG, Precision@k, or Mean Reciprocal Rank and worry later about offline metrics that might better fit your business case and recommendation design.
• Don’t just focus on hyperparameters and offline metrics but make qualitative judgments on the recommended items. For example, we found that recommendations tend to be heavily biased towards popular recipes. We counteracted this by subtracting the trained item biases from the final score. You can think of item biases capturing popularity.
• Invest early into a nice A/B testing setup and the definition of online success metrics! Unfortunately, we started too late with this and underestimated the technical and organizational difficulties we would face.

For more advice, I’ll leave you with some further readings that helped me a lot:

• Karl Higley in Top Model – How to Evaluate the Models You Train dives deeply into the topic. He has fantastic suggestions like segmenting your online and offline metrics by users and item attributes to see how performance is distributed (e.g., frequent users vs. occasional users, dramas vs. comedies) to identify underserved groups and make relevant improvements.
• In The Netflix Recommender System: Algorithms, Business Value, and Innovation, you’ll find an excellent description of how Netflix decided on online metrics that support training recommender models that positively impact their subscription-based business model.

Summary and Outlook

I hope you learned something new from our experience with pre-filtering training data and model selection for recommender systems. In Part 2, I’ll discuss the surprising amount of simple filtering rules used in modern recommendation systems and the engineering challenges in serving recommendations to users efficiently.

Part 2: Candidate Selection, Scoring & Ranking

Multiple themed recipe stripes on the For You page on Cookidoo; screenshot taken on March 18, 2022.

In Part 1, I introduced you to the challenge of personalized recipe recommendations on Cookidoo and covered the Model Selection phase. This part touches on the Candidate Selection and Scoring & Ranking components of our recommender system. I will walk you through some of the challenges we faced, the choices we made, and the lessons we learned from building the first iteration of our recipe recommender system. In the end, I will share a few of the steps we had planned for future improvements to our recipe recommendation system for Cookidoo.

Candidate Selection

Note the difference between the Training Data Selection and the Candidate Selection phase. In the former, we exclude items that we don’t want to influence our training (see Training Data Selection – Global vs. Local in Part 1). In the latter, we filter out items that we want to include in algorithm training but don’t want to recommend to a specific user or at a specific time. Another reason for including a candidate selection component could be that your item catalog is too big to be scored fully. In this case, you can train candidate selection models that are cheaper to score to pre-select items. You often find this two-stage system described in blogs and papers from large platform companies (e.g., Pinterest, Instagram, Netflix). In our case, simple filtering rules to narrow down the set of candidate items are sufficient; there is no need to maintain another machine learning model.

The extent to which simple filtering rules and business logic are employed in recommender systems is often surprising to people new to the field. Also, blogs and papers don’t often describe their application’s business logic (maybe because they require deep domain knowledge or because rule-based systems are not “cool enough”). However, to give you an idea, here are some of the crucial filters and heuristics we employ:

• Filter recipes for different recipe stripes
• Exclude recipes the user already cooked
• Exclude recipes that are out of season
• Exclude recipes that don’t fit the user’s preferred locale, e.g., to prevent a bad user experience caused by a different measurement system
• Exclude recipes that we find “too boring” to recommend, e.g., Boiled Eggs or Chopped Onions
• Introduce some randomness to results to increase the variety of recommended recipes over time

Let’s dive a bit deeper into the first three:

Filter recipes for different recipe stripes: We present recipes to the user in differently themed recipe stripes. In the screenshot above, you can see an exemplary “For You” page for a user with recipes grouped thematically into different recipe stripes (due to constant improvements, localization, and ongoing A/B tests, your Cookidoo might look different). For now, the selection of candidate recipes for these stripes follows filtering rules on recipe metadata like categories, ratings, and tags. These rules range from simple filters on categories, like on the desserts category to produce the stripe “Süßes für dich” (in English “Desserts for you”), to more complex business logic like for the stripe “Easy Alltagsrezepte für dich” (in English “Everyday recipes for you”). The latter requires a more extensive heuristics system to classify simple recipes for daily cooking sessions.

Exclude recipes the user already cooked: We filter out recipes the user has already cooked because we want our recipe recommendations to inspire users to try something new. Ideally, the user goes to our recommendations to discover recipes that are to their taste, fit their time budget, and at the same time are new to them – something they wouldn’t have searched for on their own.³

Exclude recipes that are out of season: We also filter out recipes based on the current time of the year. Cooking is highly seasonal due to user preferences and ingredient availability. For example, users in Germany only cook meals including asparagus between April and June (compare The Rhythm of Food – Asparagus), and recipes like homemade ice cream, cold soups, and cocktails are mostly cooked during the summer. Without excluding these recipes from the recommendation candidates, our model could recommend recipes that might fit the user’s taste but not the season and ingredient availability.

Scoring & Ranking

Performance Considerations

The scoring and ranking phase challenges are primarily on the engineering side. To get a ranked list of top recipes per user, we use the trained model to score candidate items from the previous phase and rank them accordingly. Scoring is done by multiplying the user embedding with the recipe embeddings of all candidate recipes and subsequently ranking them by the resulting score. When a user logs into Cookidoo, executing this step in real time can take prohibitively long when implemented naively. We considered multiple options to tackle this challenge:

• Using a database like Elasticsearch or an approximate nearest neighbors search index such as Annoy or Faiss to retrieve (approximate) scores fast enough for live scoring
• Further narrowing down the number of candidates to score in the Candidate Selection step by building a second, faster-to-score model to pre-select relevant items for live scoring
• Avoiding live scoring by pre-computing a set of recommendations per user and storing them in a database for quick access

The option of pre-computing recommendations doesn’t provide the highest flexibility but was the most reasonable to implement from a cost-benefit point of view. Since most of our users cook only once a day, a higher update frequency of our recommendations wouldn’t benefit users.⁴

It also comes with several other advantages:

• We don’t need to worry about speed at inference time at all because all we do is a simple database lookup
• We can easily pre-compute different sets of recommendations per user and run A/B tests on these

The implementation is straightforward; we have a daily job on AWS Batch that computes the users’ top recommendations and saves them in DynamoDB. Whenever a user logs into Cookidoo, we get an API request and look up the user’s recommendations.

Recency Matters

The way we create scores from the embeddings from our matrix factorization algorithm is a bit non-standard. We were faced with the challenge that recommendations wouldn’t vary much over time, i.e., even if users would cook new recipes, their recommendations wouldn’t change much based on these new interactions.

To tackle this problem, we decided to ignore the user embeddings learned by the model and build new user embeddings by averaging the recipe embeddings of the user’s last (randomly sampled) n cooking interactions. This way, user embeddings reflect the user’s latest behavior instead of including the entire history of their cooking interactions. At the same time, we can still train recipe embeddings on the complete set of historical data. Additionally, this approach requires a lower training frequency as the new user embeddings can be updated without retraining or partially training the model.

This change in the scoring logic was easy to implement, and we have seen exciting improvements in A/B tests and later in production for all our users.⁵

Where We Want to Go Next

Future steps for our team revolve around improving recommendations for Cookidoo users and increasing our online testing capacity. Building the first iteration of our production recommender system was a considerable engineering challenge. We put much effort into integrating our end-to-end system with the larger platform and building up an excellent CI/CD pipeline to enable fast iteration. With this firm basis, we can focus on improving our algorithm. We’d also like to increase iteration speed from idea to user testing constantly. We think fast iteration is key to building products that users love – Optimizing for iteration speed.

On the algorithm side, we have lots of challenges and ideas in our backlog; a few examples:

• Improve the diversity of recommendations to help the user enjoy the full range of available recipes on Cookidoo
• Test more themes for our recipe stripes and personalize site composition per user
• We started to experiment with deep learning architectures such as Two Tower Networks or Deep & Cross Networks, but it is too early to talk about improvements
• Calibrated Recommendations: A user’s niche tastes are essential and shouldn’t be crowded out by our recommender system. We are thinking about implementing something along the lines of the Calibrated Recommendations paper by Harald Steck (if you have experience with calibrated recommendations in production, I would love to hear your thoughts).
• Healthy Food Recommendations: How can Cookidoo support users with healthy eating goals? What role does the recommender system play in these goals? How can we make adjustments toward healthy eating without being too paternalistic about it?

Summary

I hope you learned something new from our experience with building a recipe recommender system for Cookidoo. Of course, there are many topics I haven’t touched on, especially more engineering-heavy issues around cloud architecture, deployment, and ensuring GDPR compliance. Maybe we’ll continue this series with more blog posts in the future and follow up with our learnings from architecting, deploying, and testing recommender systems.

If you have feedback or want to discuss any of the mentioned topics, don’t hesitate to reach out to me via Bluesky or LinkedIn.

Acknowledgments

I am the one putting this into writing, but all that I’ve written about was implemented with a fantastic team at Vorwerk Elektrowerke GmbH & Co. KG and Alexander Thamm GmbH: Bora Kiliclar, Stephan Geuter, Andreas Hausmann, Carsten Böhm, Michael Gehring, Felix Althammer, Stefan Schaub, Leon Luithlen.

Footnotes

1. For an excellent introduction to matrix factorization algorithms, I recommend the blog post Intro to Implicit Matrix Factorization by Ethan Rosenthal. ↩

2. I won’t go into more detail here; our A/B testing journey is worth another blog article in the future. ↩

3. In the recipe recommender literature, this last part is called serendipity. We don’t measure or explicitly optimize for serendipity, but filtering out already cooked recipes is an easy step in that direction. ↩

4. Also, real-time machine learning can be a hard problem to solve. Chip Huyen and Eugene Yan have great articles about the problem: Machine learning is going real-time, Real-time machine learning: challenges and solutions, Real-time Machine Learning for Recommendations. ↩

5. Of course, there are other ways to ensure recency, e.g., decaying interaction scores before training or training a separate sequence-based recommender model like GRU4Rec. ↩