The best way that I came up with is, according to the observer pattern, to notify all classes where the object was stored about its destruction and remove references to it from there.

That’s what I always do.

[System.Serializable] public class DestroyableEvent : UnityEvent<Destroyable> {}
public class Destroyable : MonoBehaviour {
    [SerializeField] private DestroyableEvent destroyedEvent = new DestroyableEvent();
    public DestroyableEvent DestroyedEvent => destroyedEvent;

    void OnDestroy() {
        destroyedEvent.Invoke(this);
    }
}

Then in some other class you have something like this:

[SerializeField] private Destroyable prefab;
private List<Destroyable> objects = new List<Destroyable>();

public void SpawnObject() {
    var newObject = Instantiate(prefab);
    newObject.DestroyedEvent.AddListener(objects.Remove);
    objects.Add(newObject);
}

If objects are created and destroyed frequently, you should use a pooling system instead, but you still need events to track when an object is returned to the pool so you can remove it from any relevant lists.

Another solution you can use is to periodically scan lists for null entries and remove them, or remove them when you try to access an entry in the list and determine that it’s null. I prefer to use an event-driven approach as shown above.