Ensure anchored components are properly stacked on top of Dialog components (#3111)

* ensure `Dialog` knows about `Modal`s via the `StackProvider`

When you render a `Listbox` in a `Dialog`, then clicking outside of the
`Listbox` will only close the `Listbox` and not the `Dialog`.

This is because the `Listbox` is rendered _inside_ the `Dialog`, and the
`useOutsideClick` hook will prevent the event from propagating to the
`Dialog` therefore it stays open.

Then, if you add the `anchor` prop to the `ListboxOptions` then a few
things will happen:

1. We will render the `ListboxOptions` in a `Modal`, which portals the
   component to the end of the `body` (aka, it won't be in the `Dialog`
   anymore).
2. The `anchor` prop, will use Floating UI to position the element
   correctly.

If you now click outside of the open `Listbox`, then the `Dialog` will
receive the click event (because it is rendered somewhere else in the
DOM) and therefore the `Listbox` **and** the `Dialog` will close.

The `Dialog` also uses a `StackProvider` to know if it is the top-level
`Dialog` or not. The problem is that the `Modal` doesn't use that
`StackProvider` to tell the `Dialog` that something is stacked on top of
the current `Dialog`.

That's what this commit fixes, the `Modal` will now use a
`StackProvider` to tell the `Dialog` that it's not the top-most element
anymore so it shouldn't enable the `useOutsideClick` behavior.

That said, this is one of the things that will be changed in the future
to make "parallel" dialogs possible. Essentially, we will track a global
stack and the top-most element (last one that was "opened") will win.

Then hooks such as `useOutsideClick` and `useScrollLock` will use that
information to know if they should undo scroll locking for example if
another element is still open.

* update CHANGELOG

packages/@headlessui-react/CHANGELOG.md

@@ -21,6 +21,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Render hidden form input fields for `Checkbox`, `Switch` and `RadioGroup` components ([#3095](https://github.com/tailwindlabs/headlessui/pull/3095))
 - Ensure the `multiple` prop is typed correctly when passing explicit types to the `Combobox` component ([#3099](https://github.com/tailwindlabs/headlessui/pull/3099))
 - Omit `nullable` prop from `Combobox` component ([#3100](https://github.com/tailwindlabs/headlessui/pull/3100))
+- Ensure anchored components are properly stacked on top of `Dialog` components ([#3111](https://github.com/tailwindlabs/headlessui/pull/3111))
 
 ### Changed
 

packages/@headlessui-react/src/components/dialog/dialog.tsx

@@ -390,7 +390,7 @@ function DialogFn<TTag extends ElementType = typeof DEFAULT_DIALOG_TAG>(
       enabled={dialogState === DialogStates.Open}
       element={internalDialogRef}
       onUpdate={useEvent((message, type) => {
-        if (type !== 'Dialog') return
+        if (type !== 'Dialog' && type !== 'Modal') return
 
         match(message, {
           [StackMessage.Add]: () => setNestedDialogCount((count) => count + 1),

packages/@headlessui-react/src/internal/modal.tsx

@@ -27,6 +27,7 @@ import {
   type RefProp,
 } from '../utils/render'
 import { ForcePortalRoot } from './portal-force-root'
+import { StackProvider } from './stack-context'
 
 function useScrollLock(
   ownerDocument: Document | null,
@@ -171,7 +172,7 @@ function ModalFn<TTag extends ElementType = typeof DEFAULT_MODAL_TAG>(
   }
 
   return (
-    <>
+    <StackProvider type="Modal" enabled={enabled} element={internalModalRef}>
       <ForcePortalRoot force={true}>
         <Portal>
           <FocusTrap
@@ -201,7 +202,7 @@ function ModalFn<TTag extends ElementType = typeof DEFAULT_MODAL_TAG>(
       <HoistFormFields>
         <MainTreeNode />
       </HoistFormFields>
-    </>
+    </StackProvider>
   )
 }