tl;dr

Dry run track

- Solving the long run issue

- Preparing for the shadowing process

Product adaptations track

- Finish the UID adaptations

- Last check to get ready for the migration

Search track

- Deliver the client to use the new search

More details below 👇

Product adaptations

We are in the process of integrating a new database into the different systems of our product. The main idea is to make them able to communicate with the legacy and the new db simultaneously. We can monitor the new database behaviour using the legacy one as a source of truth. This is what we call the shadowing process. It will allow us to run the new system without taking the risk of a major incident involving data corruption in case of an issue.

Once done, we will switch to the new database.

We can't activate the shadowing process before having all sub-systems manage the new database. However, we can release progressively to ensure non-regression.

All product adaptations should be completed this week. We will also focus on preparing for

the shadowing process

We reported last week's goal for this week.

Integrating a new search

We defined a strategy to implement a new search engine a few weeks ago, going through intermediate steps. We have decided to make a bet and work directly towards the target search engine.

The team has started implementing the new search with OpenSearch. There will be three major steps:

- Implementing a lambda in charge of indexing documents ✅

- Implementing a lambda in charge of searching for documents ✅

- Creating a client to be used by the Page Builder team.

The first two steps are done. This week, we expect a JS client the Page Builder team can implement in the document list.

Dry run

We ended last week with 99.93% of successfully migrated repositories, including all major entities. We decided to move on to the next step, optimizing the migration speed.

Adding the remaining entities slows the process to more than three days for only 200 repositories. This week, we will focus on optimizing the migration time.