[Concurrent] Safely disposing uncommitted objects

[danielkcz]

How to safely keep a reference to uncommitted objects and dispose of them on unmount?

For a MobX world, we are trying to prepare for the Concurrent mode. In short, there is a Reaction object being created to track for observables and it is stored within useRef.

The major problem is, that we can't just useEffect to create it in a safe way later. We need it to start tracking the observables on a first render otherwise we might miss some updates and cause inconsistent behavior.

We do have a semi-working solution, basically, a custom made garbage collector based on setTimeout. However, it's unreliable as it can accidentally dispose of Reactions that are actually being used but weren't committed yet.

Would love to hear we are overlooking some obvious solution there.

[gaearon]

@mweststrate

[danielkcz]

No need to ping him, he knows about it.

Either way, I wanted to ask for a more generic solution, not only related to a MobX. I can imagine similar problems will be happening all over the place once people start preparing for Concurrent. It feels like that React is not giving enough tools to tackle the problem gracefully.

[gaearon]

I’m tagging him precisely because “he knows about it” and because we’ve had conversations in the past about it. :-)

Given that he knows constraints of both I thought his input could be valuable.

As for your question — the design tradeoff we’re going for is that we don’t have a cleanup phase for renders that never committed. Rendering isn’t supposed to “allocate” resources that have to be cleaned up later. Instead we rely on the garbage collector. Resources that have to be disposed are only safe to allocate after the first commit.

I don’t know for certain if tradeoffs chosen by MobX are compatible or not with this approach, as I don’t know enough about MobX.

Can you explain what “reactions” are and why GC isn’t sufficient for them? Or why creating them can’t wait until after the first commit?

[danielkcz]

Can you explain what “reactions” are and why GC isn’t sufficient for them? Or why creating them can’t wait until after the first commit?

Well, I don't know exact internals how it works, it's very entangled there, but my assumption is that Reaction kinda attaches itself to observable variables present in the callback function. So as long as those variables exist, the Reaction won't be GCed. Those variables can definitely exist outside the scope of the component, that's kinda point of the MobX.

I think we have managed to figure out some way around it. Custom GC :) Basically, we globally keep the list of created reactions (in WeakMap) along with a time of their creation. If Reaction gets committed, all is fine and it gets removed from the list. A global timer takes care of cleaning up those that weren't committed in a given time.

I am wondering if you have some idea for a good "timeout" period? I mean time from render to commit. For now, we are thinking like 1 second. I assume it should be a long enough period even on a really slow computer. Is it bad idea to rely on that?

[danielkcz]

Or why creating them can’t wait until after the first commit?

@RoystonS wrote a good explanation of that. We could end up in very ugly situations ...

[urugator]

Can you explain what “reactions”

Mobx automatically manages subscriptions based on usage.
You define a function (reaction), which can access observable objects.
When the function is invoked, the observables accessed inside the function are added as dependencies to the function (reaction).
When you modify an observable, the function is automatically invoked again, resubscribing and optionally running a side effect.
We wrap render inside such function, so that if you access an observable in render, it creates a subscription:

const disposer = reaction(
  () => {    
    // accesses observables, creating subscriptions
    render()
  }, 
  () => {
    // a side effect to invalidate the component
    setState() // or forceUpdate() 
  }
)

why creating them can’t wait until after the first commit

The subscriptions are determined by rendering logic, so we would have to re-run the rendering logic in effect (double render).

why GC isn’t sufficient for them

While reactions come and go (together with react's component instances), the observable objects possibly stay for the whole app lifetime and they hold references to reactions (preventing them from being GCed).

[RoystonS]

Basically: we're trying to avoid double-rendering all MobX-observer components. The only time we actually need to double-render is if StrictMode/ConcurrentMode began rendering our component but then never got to the commit phase with us, or if something we're observing changed between initial render and commit. Taking a double-render hit on every component to cope with those (hopefully edge) cases is something we'd like to avoid.

[mweststrate]

@gaearon @urugator's description is pretty accurate. Whether it should be fixed through a React life-cycleish thing or by doing some additional administration on MobX side (timer + custom GC) have both it's merits. So let's leave that aside for now, and just focus on the use case:

Basic concepts

But the reason why immediate subscription is important to MobX (as opposed to a lifecycle hook that happens after render, such as useEffect) is because observables can be in 'hot' and 'cold' states (in RxJS terms). Suppose we have:

1. A collection of todo items
2. An expression (we call this a computed) that computes the amount of unfinished computed items
3. A component that renders the amount of unfinished computed items.

Now, the expression computed can be read in two different modes:

1. If the computed is cold, that is, no one is subscribed to it's result; it evaluates as a normal expression, returning and forgetting its result
2. If the computed is hot, it will start observing its own dependencies (the todo items), and caching the response as long as the todo item set is unaffected and the output of the computed expression is unchanged.

MobX without concurrent mode

With that I mind, how observer was working so far is as follows:

1. During the first (and any subsequent) render, MobX creates a so called reaction object which keeps track of all the dependencies being read during the render and subscribes to them. This caused the computed to become hot immediately, and a subscription to the computed to be set up. The computed in turn subscribes to the todos it used (and any other observable used). So a dependency graph forms.
2. Since there is an immediate subscription, it means that the reaction will no longer be GC-ed as long as the computed exists. Until the subscription is disposed. This is done automatically done during componentWillUnmount. This cause the reaction to release its subscriptions. And in turn, if nobody else depends on the 'computed' value, that will also release its subscriptions to the todos. Reaction objects can now be GC-ed.

So what is the problem?

In concurrent mode, phase 2, disposing the reaction, might no longer happen, as an initial render can happen without matching "unmount" phase. (For just suspense promise throwing this seems avoidable by using finally)

Possible solutions

Solution 1: Having an explicit hook that is called for never committed objects (e.g. useOnCancelledComponent(() => reaction.dispose())

Solution 2: Have global state that records all created reaction objects, and remove reactions belonging to committed components from that collection. On a set interval dispose any non-committed reactions. This is the current (WIP) work around, that requires some magic numbers, such as what is a good grace period? Also needs special handling for reactions that are cleaned up, but actually get committed after the grace period!

Solution 3: Only subscribe after the component has been committed. This roughly looks like;

1. During the first render, read the computed value cold, meaning it will be untracked
2. After commit (using useEffect) render another time, this time with proper tracking. This serves two goals:
3. The dependencies can be established
4. Any updates to the computed between first rendering and commit will be picked up. (I can imagine this is a generic problem with subscribing to event emitters as part of useEffect, updates might have been missed?)

The downsides:

1. Every component will render twice!
2. The computed will always compute twice, as there is first a 'cold', which won't cache, and then a 'hot' read. Since in complicated applications there might be entire trees of depending computable values be needed, this problem might be worse than it sounds; as the dependencies of a computed value also go 'cold' if the computed value itself goes cold.

Hope that explains the problem succinctly! If not, some pictures will be added :).

[gaearon]

Does MobX have bigger problems in Concurrent Mode? How valuable is it to fix this in isolation without addressing e.g. rebasing?

[mweststrate]

I think the later problem can be largely avoided by making a clear distinction between component own state (use useState instead of MobX) and state that lives outside the component tree (external domain state). If I am not mistaken, my intuition is that rebasing only makes sense for 'volatile ui state' such as the state of pending requests, switching tabs / pages, charts etc. But state that is domain specific (e.g. a collection of todo's), typically doesn't need rebasing as that would lead to weird behavior for the end-user?

…

On Wed, Apr 10, 2019 at 4:36 PM Dan Abramov ***@***.***> wrote: Does MobX have bigger problems in Concurrent Mode? How valuable is it to fix this in isolation without addressing e.g. rebasing? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#15317 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABvGhOMREb3lbrEL4q9sccIXfNj5f5vfks5vffcHgaJpZM4ccUAP> .

[NE-SmallTown]

Any picture? Thanks~ :)

[radex]

We have a very similar issue to MobX's with 🍉 WatermelonDB, but even more tricky because of asynchronicity.

When connecting the data source to components, we subscribe to the Observable. This has to happen directly in render, because:

• if we subscribe in an effect/on did mount, we'll ALWAYS render twice, and the first render will be wasted, even if we have the data ready to render and we can skip a bunch of unnecessary work.
• it wouldn't be appropriate for an useObservable hook to return null on initial render, because it's not always clear what the component developer is supposed to do with it. For example, if an observable value is necessary to fetch another observable value, the developer would have to make a whole bunch of extra logic just to get to the return null (since we can't just bail early and render null with hooks - like we can today with a withObservables HOC).
• it would be appropriate for useObservable to throw a Promise and suspend, however, we risk showing stale data (which can lead to bugs or ugly glitches) if we treat the data source like an async promise, not a reactive Observable — it could be that between the time the initial data is fetched, and component re-rendered, the value of the observable has changed again. But it's not possible to know that on the second render if we don't stay subscribed to it. Which means we have to subscribe before we throw a Promise — and have to deal with cleaning up the subscription if the component never ends up being rendered