The accessible name computation spec has changed to explicitly mention this case:
Step 2A:
Hidden Not Referenced: If the current node is hidden and is:
- Not part of an aria-labelledby or aria-describedby traversal,
where the node directly referenced by that relation was hidden.
- Nor part of a native host language text alternative element
(e.g. label in HTML) or attribute traversal, where the root
of that traversal was hidden.
See https://w3c.github.io/accname/#computation-steps.