The problem: from time to time i get slow (>10s) queries even though they are using based on indexed fields.

I'm using DocDB instance that has 128GB of memory. The total size of all of the indexes (across all collections) is 1.5TB, and data-size is 1TB. That obviously means that all of the indexes can't fit into memory together.

In addition, when I check the BufferCacheHitRatio metric I see values between 88% and 95%, which as far as I understand are considered very low, and mean that between 5% and 12% queries have to go to the disk, which is slower.

The way I thought of tackling it, is to split my system into logical applications, based on the way it queries the DB, i.e. the indexes it needs. Then, I can create read replica for every such application (all of the applications only perform "read" operations), and enforce that every application connects only to its corresponding read replica.

I can define the instance type of every read replica based on the size of all of the relevant indexes.

The reason I like this approach as oppose to simply scale the single instance I currently have so that the indexes can fit in memory is:

1. Not sure I can scale forever, i.e. is there a DocDB instance with 10TB of memory?
2. It enables me to have different QoS for different "applications", if I have one application that is client facing, and it's extremely important that the queries will always use the index, then I can allow it specifically for this application, where in other, i can still go to the disk 10% of time and it's ok. So it gives me this flexibility.

Is this the right approach for addressing such issues? Or should this be avoided and there's other pattern to get what I want with DocDB?

Thanks