What is the expected behavior?

Aer already has an optimization to not simulate idle qubits. That means even a 100 qubit circuit can be simulated if only a small subset of qubits actually contain gates. See: #1299

However this stops working in the Estimator, when an observable with lots of Is is specified. Instead, the optimization should be extended to not simulate/measure such qubits.

This is important due to the Qiskit transpiler often enlarging circuits/observables when it targets a large device. But the true non-trivial part of the circuit is still small, and should remain simulable.