Solidity: await validation completion for diagnostics instead of a fixed wait

The Nomic solidity-language-server validates asynchronously: document
changes are debounced (~250ms), validation runs a forge/solc compile,
and completion is signalled with the custom notification
custom/validation-job-status -- emitted on every completion path
(success, validation failure, job errors; verified in the bundled 0.8.4
package, which publishes the document's diagnostics before the signal).
Serena ignored the signal ("Unhandled method
'custom/validation-job-status'") and waited a fixed 2.5s for a
non-empty publishDiagnostics, returning [] whenever validation finished
later. Warm environments fit the window; slow or cold ones (macOS and
Windows CI runners, which download solc 0.8.20 on every run -- the CI
cache only covers ~/.serena/language_servers/static, not
~/.local/share/svm) often do not. This is the root cause of the
empty-diagnostics failures of test_file_diagnostics[solidity]: xfailed
on macOS since its introduction (bd7e64e8) and failing on Windows on
Tests run #4236, job catch-all (windows-latest). Shrinking the wait
reproduces the exact failure signature (AssertionError: []) on Linux.

Changes:
- Handle custom/validation-job-status (threading.Event + last payload).
  The payload carries no document URI, so the event is global to the
  server instance.
- request_text_document_diagnostics keeps the 2.5s non-empty publish
  wait as the fast path. When it misses, it awaits the completion
  signal against a 60s deadline (aligned with the indexing wait) and
  takes the generation-advancing publication for the request URI,
  accepting an empty result as the validated answer; a completion that
  brings no publication for the URI (e.g. one belonging to another
  document) re-arms the wait until the deadline. If the signal never
  arrives, a warning is logged and the previous empty-result behavior
  applies, bounded by the deadline.
- Remove the darwin xfail from test_file_diagnostics; add regression
  tests for the fallback delivery (fast path forced to miss), for the
  never-signalled bound, and for re-arming on a spurious completion
  belonging to another document.
- CHANGELOG entry under Language Servers.

Design notes:
- Mirrors #1639 (TypeScript/VTS: event-based indexing wait replacing a
  fixed sleep).
- The simpler alternative -- accepting empty publications and raising
  the fixed wait -- was rejected: without the completion signal, an
  early clear-on-open publication would be taken as the answer, and
  every clean file would stall for the full wait.
- Supersedes the quarantine approach of draft PR #1674 (broadening the
  xfail to non-Linux); whichever lands second resolves the trivial
  conflict on the xfail block.

test/solidlsp/solidity/test_solidity_diagnostics.py

@@ -1,19 +1,42 @@
-import sys
-
 import pytest
 
 from solidlsp import SolidLanguageServer
+from solidlsp.language_servers.solidity_language_server import SolidityLanguageServer
 from solidlsp.ls_config import Language
 from test.solidlsp.util.diagnostics import assert_file_diagnostics
 
 
+class _NeverSetEvent:
+    """Stand-in for :class:`threading.Event` whose wait never succeeds."""
+
+    def wait(self, timeout: float | None = None) -> bool:
+        return False
+
+    def clear(self) -> None:
+        return None
+
+    def set(self) -> None:
+        return None
+
+
+class _AlwaysSignalledEvent:
+    """Stand-in for :class:`threading.Event` whose wait always reports a completion; counts clears."""
+
+    def __init__(self) -> None:
+        self.cleared = 0
+
+    def wait(self, timeout: float | None = None) -> bool:
+        return True
+
+    def clear(self) -> None:
+        self.cleared += 1
+
+    def set(self) -> None:
+        return None
+
+
 @pytest.mark.solidity
 class TestSolidityDiagnostics:
-    @pytest.mark.xfail(
-        sys.platform == "darwin",
-        reason="Unknown — passes on Ubuntu but consistently fails on macOS CI; root cause not yet identified.",
-        strict=False,
-    )
     @pytest.mark.parametrize("language_server", [Language.SOLIDITY], indirect=True)
     def test_file_diagnostics(self, language_server: SolidLanguageServer) -> None:
         assert_file_diagnostics(
@@ -22,3 +45,71 @@ class TestSolidityDiagnostics:
             (),
             min_count=1,
         )
+
+    @pytest.mark.parametrize("language_server", [Language.SOLIDITY], indirect=True)
+    def test_file_diagnostics_via_validation_completion(
+        self, language_server: SolidLanguageServer, monkeypatch: pytest.MonkeyPatch
+    ) -> None:
+        """Diagnostics must still arrive via the validation-completion fallback when the initial
+        publish wait does not observe them (the failure mode of cold/slow CI runners).
+        """
+        assert isinstance(language_server, SolidityLanguageServer)
+
+        # force the fast path to miss so that only the validation-completion fallback can deliver
+        monkeypatch.setattr(language_server, "_wait_for_relevant_published_diagnostics", lambda **kwargs: None)
+
+        assert_file_diagnostics(
+            language_server,
+            "contracts/DiagnosticsSample.sol",
+            (),
+            min_count=1,
+        )
+
+    @pytest.mark.parametrize("language_server", [Language.SOLIDITY], indirect=True)
+    def test_file_diagnostics_without_validation_signal(
+        self, language_server: SolidLanguageServer, monkeypatch: pytest.MonkeyPatch
+    ) -> None:
+        """A server that never signals validation completion must yield an empty result within the
+        bound instead of hanging or raising.
+        """
+        assert isinstance(language_server, SolidityLanguageServer)
+
+        # force the fast path to miss and make the completion signal unobtainable
+        monkeypatch.setattr(language_server, "_wait_for_relevant_published_diagnostics", lambda **kwargs: None)
+        monkeypatch.setattr(language_server, "_VALIDATION_COMPLETION_TIMEOUT", 0.2)
+        monkeypatch.setattr(language_server, "_validation_completed", _NeverSetEvent())
+
+        diagnostics = language_server.request_text_document_diagnostics("contracts/DiagnosticsSample.sol", min_severity=1)
+        assert diagnostics == []
+
+    @pytest.mark.parametrize("language_server", [Language.SOLIDITY], indirect=True)
+    def test_file_diagnostics_rearms_after_spurious_completion(
+        self, language_server: SolidLanguageServer, monkeypatch: pytest.MonkeyPatch
+    ) -> None:
+        """A completion signal belonging to another document (the payload carries no URI) must
+        re-arm the fallback wait instead of ending the request without diagnostics.
+        """
+        assert isinstance(language_server, SolidityLanguageServer)
+
+        expected_diagnostic = {
+            "uri": "file:///spurious-completion-test",
+            "severity": 1,
+            "message": "delivered after re-arm",
+            "range": {"start": {"line": 0, "character": 0}, "end": {"line": 0, "character": 1}},
+            "code": None,
+        }
+        # first wake is spurious (no publication for the request URI), second delivers
+        grace_reads: list[list | None] = [None, [expected_diagnostic]]
+        event = _AlwaysSignalledEvent()
+
+        # force the fast path to miss and script the fallback's wake/read sequence
+        monkeypatch.setattr(language_server, "_wait_for_relevant_published_diagnostics", lambda **kwargs: None)
+        monkeypatch.setattr(language_server, "_validation_completed", event)
+        monkeypatch.setattr(language_server, "_wait_for_published_diagnostics", lambda **kwargs: grace_reads.pop(0))
+
+        diagnostics = language_server.request_text_document_diagnostics("contracts/DiagnosticsSample.sol", min_severity=1)
+
+        # one clear at request start plus exactly one re-arm after the spurious wake
+        assert event.cleared == 2
+        assert not grace_reads  # both scripted reads consumed
+        assert [diagnostic["message"] for diagnostic in diagnostics] == ["delivered after re-arm"]