Evolution of Search Engines Architecture-Algolia New Search Architecture Part 1(highscalability.com)

Evolution of Search Engines Architecture-Algolia New Search Architecture Part 1

http://highscalability.com/blog/2021/8/2/evolution-of-search-engines-architecture-algolia-new-search.html

1 comments

> Defining the number of shards is a critical element of your configuration, as it defines the ability to scale and your underlying performance...The number of shards is not big enough to use all the CPU resources on all machines...If you do not have enough CPU threads for all shards at the same time, it will negatively impact your response time.

This is a key point. There is rarely enough discussion around this in normal debates around sharding/parallelization.