mirror of
https://github.com/tiennm99/DocsGPT.git
synced 2026-10-14 16:16:48 +00:00
fix(admin): second review pass
Correctness - /api/remote never recorded source.created, so URL, GitHub and connector sources had a source.deleted with no matching creation. All three creation paths now go through one _audit_source_created helper. - The prompt-cache rate divided cached tokens by a whole bucket's prompt tokens. A bucket is a day and mixes calls whose provider reports a cache breakdown with calls whose provider does not, so filtering buckets in the client could not separate them and the rate was understated by however much traffic ran on a non-reporting provider. The denominator is now computed in SQL over the reporting rows. - The outcome pill matched values nothing writes. Guardrails emit triggered / not_evaluated and the device feed emits dispatched; the map had blocked / denied / allowed, so a guardrail that fired rendered neutral grey -- the one signal the merged feed exists to surface. Fixtures were seeding the fictional values, so the tests passed on it too. - Stream duration_ms timed the consumer. stream_token_usage is a generator, so start-to-exhaustion includes the agent loop's tool handling and the SSE client's pace; a slow browser recorded ~30s for a sub-second call. It now accumulates only the time spent inside next(). Safety - Activity filters failed open: an unknown facet or unparseable timestamp was dropped, and no filter means every row, so a typo widened an audit view and on the export streamed the full history. Both are now a 400. - The search term was interpolated into an ILIKE pattern, so "100%" matched everything and "q1_report" matched more than it should. Escaped. - 0034 set actor_id NOT NULL with no default. A previous-release process inserting mid-rollout would raise, and in admin/routes.py that insert shares the request transaction, so a role grant beside it would roll back too. Noise and dead code - The per-user panel is a security panel: data-plane events file under the actor, so an active account's routine deletes pushed a denied login out of the 20-row window. It now excludes them; the Activity tab shows everything. - device_audit_log had no created_at-leading index, so the merged feed sequentially scanned that branch every page (migration 0036). - conversation.deleted_all no longer records when nothing was deleted, and agent.updated no longer records an empty field list. - Dropped by_model from /admin/usage (no consumer; an extra aggregate per page load), the duplicate filter surface on AuthEventsRepository that nothing called, and the unreachable FLOW_LABELS.schedule entry. - Type hints on record_event's conn and the remaining unannotated helpers.
This commit is contained in:
1 parent
2ea9d66789
commit
25c82003d7
33 files changed
+635
-206
No files matched your search
@@ -5,6 +5,7 @@ import os
|
||||
import tempfile
|
||||
import uuid
|
||||
import zipfile
|
||||
from typing import Optional
|
||||
|
||||
from flask import current_app, jsonify, make_response, request
|
||||
from flask_restx import fields, Namespace, Resource
|
||||
@@ -179,6 +180,40 @@ def _source_archive_limits() -> ZipExtractionLimits:
|
||||
)
|
||||
|
||||
|
||||
def _audit_source_created(
|
||||
*,
|
||||
source_id: str,
|
||||
user: Optional[str],
|
||||
name: Optional[str],
|
||||
source_type: str,
|
||||
task_id: Optional[str],
|
||||
) -> None:
|
||||
"""Record ``source.created`` for an accepted ingestion job.
|
||||
|
||||
Audited at the request, not in the worker: enqueuing is the user's action.
|
||||
The ingest may still fail, which the source's own status records.
|
||||
|
||||
Every path that mints a ``source_id`` and dispatches must call this --
|
||||
``source.deleted`` is recorded for remote sources too, and a trail showing
|
||||
a deletion with no matching creation is worse than no trail.
|
||||
"""
|
||||
try:
|
||||
with db_session() as conn:
|
||||
record_event(
|
||||
conn,
|
||||
"source.created",
|
||||
actor=user,
|
||||
source_id=source_id,
|
||||
name=name,
|
||||
type=source_type,
|
||||
task_id=task_id,
|
||||
)
|
||||
except Exception as err:
|
||||
current_app.logger.warning(
|
||||
"Could not audit source.created for %s: %s", source_id, err, exc_info=True
|
||||
)
|
||||
|
||||
|
||||
@sources_upload_ns.route("/upload")
|
||||
class UploadFile(Resource):
|
||||
@api.expect(
|
||||
@@ -397,26 +432,13 @@ class UploadFile(Resource):
|
||||
return make_response(jsonify({"success": False}), 400)
|
||||
# Predetermined id matches the dedup-claim row; loser GET sees same.
|
||||
response_task_id = predetermined_task_id or task.id
|
||||
# Audited here, not in the worker: this is the user action. The
|
||||
# ingest may still fail, which the source's own status records.
|
||||
try:
|
||||
with db_session() as conn:
|
||||
record_event(
|
||||
conn,
|
||||
"source.created",
|
||||
actor=user,
|
||||
source_id=str(source_uuid),
|
||||
name=job_name,
|
||||
type="local",
|
||||
task_id=response_task_id,
|
||||
)
|
||||
except Exception as err:
|
||||
current_app.logger.warning(
|
||||
"Could not audit upload for source %s: %s",
|
||||
source_uuid,
|
||||
err,
|
||||
exc_info=True,
|
||||
)
|
||||
_audit_source_created(
|
||||
source_id=str(source_uuid),
|
||||
user=user,
|
||||
name=job_name,
|
||||
source_type="local",
|
||||
task_id=response_task_id,
|
||||
)
|
||||
# ``source_uuid`` was minted above and passed to the worker as
|
||||
# ``source_id``; the worker uses it verbatim for every SSE event,
|
||||
# so the frontend can correlate inbound ``source.ingest.*`` to
|
||||
@@ -568,6 +590,13 @@ class UploadRemote(Resource):
|
||||
connector_kwargs["task_id"] = predetermined_task_id
|
||||
task = ingest_connector_task.apply_async(**connector_kwargs)
|
||||
response_task_id = predetermined_task_id or task.id
|
||||
_audit_source_created(
|
||||
source_id=str(source_uuid),
|
||||
user=user,
|
||||
name=data["name"],
|
||||
source_type=data["source"],
|
||||
task_id=response_task_id,
|
||||
)
|
||||
# ``source_uuid`` was minted above and passed to the
|
||||
# worker as ``source_id``; the worker uses it verbatim
|
||||
# for every SSE event, so the frontend can correlate
|
||||
@@ -601,6 +630,13 @@ class UploadRemote(Resource):
|
||||
_release_claim(scoped_key)
|
||||
return make_response(jsonify({"success": False}), 400)
|
||||
response_task_id = predetermined_task_id or task.id
|
||||
_audit_source_created(
|
||||
source_id=str(source_uuid),
|
||||
user=user,
|
||||
name=data["name"],
|
||||
source_type=data["source"],
|
||||
task_id=response_task_id,
|
||||
)
|
||||
response_payload = {
|
||||
"success": True,
|
||||
"task_id": response_task_id,
|
||||
|
||||
Reference in new issue
Block a user