The Agency squad is currently working on creating a new Migration API. For that, they will rely on one of our internal Core APIs, the document API, that manages operations on the documents such as creating a document, publishing a document, etc.

Our first investigation shows that the main bottleneck is our database infrastructure (PrismicDB). It just doesn’t fit the current usage as well as expected future usages (e.g. Importing a large number of documents).

Kicking off a new project

Two weeks ago, we kicked off a new project with a few goals:

• - Scaling the Documents API to support the Migration API on time for their release
• - Set proper foundations for the next Architecture
• - Support future activity (10x active repositories) and features (e.g Autosave)

Timeline

We want to be ready by mid-December with the first milestone mid-November.

Main contributor to the project

• - Raphael
• - Mehdi
• - Sebastien
• - Arnaud

What did we do so far?

• - Evaluated and selected a new data architecture.
• - Defined the scope of the migration in terms of data entities and repositories.
• - Evaluated the product impacts and adaptations.
• - Evaluate the timeline.

This week plan

SRE:

• - Set up the database in staging and prod, add monitoring for perf tests (db resources, req. perf.).
• - Understand how one repository (mystique) is still able to go through the rate limiter.

Archi:

• - Define the DB performance baseline
• - Estimate the performance of the new database
• - We keep progressing on spikes to clarify the timeline
• - Start migration of two entities (we start this week but close in 2 two weeks)

Other:

• - Integration Field for Shopify that can't display more than 50 items
• - Code review for agency squad