mirror of
https://github.com/tiennm99/ccs.git
synced 2026-10-11 18:12:36 +00:00
feat(hardening): P1 maintainability metrics baseline + freshness gate (#1561)
Epic P1. Adds maintainability-metrics script (typed-error adoption, createLogger coverage, hotpath console.error, files>400 LOC), merges maintainability block into hardening-inventory report, adds 30-day freshness gate to ci-parity-gate.sh, re-baselines burndown 2026-06-18. Local validate + validate:ci-parity green.
This commit is contained in:
1 parent
27f4e89ad1
commit
95a2864ef3
7 files changed
+1290
-306
No files matched your search
@@ -1,7 +1,7 @@
|
||||
# Hardening Debt Burndown Tracker
|
||||
|
||||
Last Updated: 2026-02-12
|
||||
Owner: Stream D (`#542`)
|
||||
Last Updated: 2026-06-18
|
||||
Owner: Stream D (`#542`); maintainability epic owner TBD (open Q5)
|
||||
|
||||
## Scope
|
||||
|
||||
@@ -43,3 +43,35 @@ Baseline captured: `2026-02-12`.
|
||||
| Date | Area | Change | Safety Notes |
|
||||
|---|---|---|---|
|
||||
| 2026-02-12 | `src/web-server/jsonl-parser.ts` | Migrated `parseProjectDirectory()` directory listing from sync `readdirSync` to async `fs.promises.readdir` | Existing behavior kept (same filtering/fallback); covered by `tests/unit/jsonl-parser.test.ts` |
|
||||
|
||||
## Maintainability & Traceability Baseline (2026-06-18)
|
||||
|
||||
Baseline for the maintainability/traceability epic (`plans/260618-1346-maintainability-traceability-epic`). Sourced from `docs/reports/hardening-inventory.json` -> `maintainability` block after `bun run report:hardening`. Baseline captured: `2026-06-18`.
|
||||
|
||||
| Metric | Baseline | Epic target | Owner phase |
|
||||
|---|---:|---:|---|
|
||||
| typed-error adoption (typed / total throws) | 0.9% (4 / 431) | >40% in locked subdomains | P4 |
|
||||
| typed-error adoption (locked: cliproxy/quota, cliproxy/auth, web-server/routes, auth) | 0.0% (0 / 23) | >40% | P4 |
|
||||
| hotpath `console.error`/`warn` occurrences (non-exempt) | 931 (1091 total, 160 CLI-UX exempt) | < 10 | P3 |
|
||||
| hotpath `console.error`/`warn` files (non-exempt) | 134 | minimal | P3 |
|
||||
| files with `createLogger` | 35 / 685 (5.1%) | rise across all subdomains | P2/P3 |
|
||||
| subdomains with zero `createLogger` | 20 (incl. api, channels, config, delegation, dispatcher, docker, shared) | 0 in the named set | P2 |
|
||||
| files > 400 LOC | 95 | < 60 after P5+P6 | P5/P6 |
|
||||
| files > 600 LOC | 45 | drop | P5/P6 |
|
||||
| ESLint `no-new-throw-error` gate | not enforced | error + allowlist | P7 |
|
||||
| ESLint `max-lines` gate | not enforced | warn at 400 | P7 |
|
||||
| hardening report freshness | stale (2026-02-12) | < 30d gate in `validate:ci-parity` | P1 |
|
||||
|
||||
### Method
|
||||
|
||||
Metrics are grep-based and approximate (not a contract). Comments and string/template/regex literals are stripped before matching (`scripts/hardening-inventory.js#stripComments`). Subdomain granularity is 2-level under `src/cliproxy/` (`cliproxy/quota`, `cliproxy/auth`, ...) and 1-level elsewhere (`auth`, `config`, ...). The hotpath `console.error` count excludes CLI-UX print surfaces (`src/commands/`, `src/management/`, `src/utils/ui/`) which are legitimate user-facing terminal output. The typed-error denominator for the P4 target is LOCKED to the four named subdomains so the >40% goal cannot be gamed by narrowing scope. Re-baseline whenever the schema or method changes.
|
||||
|
||||
### Largest hotpath console.error offenders (2026-06-18)
|
||||
|
||||
| File | `console.error`/`warn` |
|
||||
|---|---:|
|
||||
| `src/utils/error-manager.ts` | 142 |
|
||||
| `src/cliproxy/accounts/account-safety.ts` | 56 |
|
||||
| `src/cliproxy/config/model-config.ts` | 32 |
|
||||
| `src/cliproxy/executor/arg-parser.ts` | 26 |
|
||||
| `src/dispatcher/flows/settings-flow.ts` | 26 |
|
||||
File diff suppressed because it is too large.
Load diff
@@ -6,43 +6,102 @@ Scope: `src/**/*.{ts,tsx,js,jsx,mjs,cjs}`
|
||||
|
||||
| Metric | Value |
|
||||
|---|---:|
|
||||
| Sync fs occurrences (all) | 835 |
|
||||
| Sync fs files affected (all) | 100 |
|
||||
| Sync fs occurrences (runtime hotpaths) | 724 |
|
||||
| Sync fs files affected (runtime hotpaths) | 89 |
|
||||
| Legacy shim markers | 131 |
|
||||
| Legacy shim files affected | 56 |
|
||||
| Sync fs occurrences (all) | 2304 |
|
||||
| Sync fs files affected (all) | 243 |
|
||||
| Sync fs occurrences (runtime hotpaths) | 1949 |
|
||||
| Sync fs files affected (runtime hotpaths) | 193 |
|
||||
| Legacy shim markers | 424 |
|
||||
| Legacy shim files affected | 163 |
|
||||
|
||||
## Top Runtime Hotpath Sync fs Files
|
||||
|
||||
| File | Sync Calls | API Names |
|
||||
|---|---:|---|
|
||||
| `src/management/shared-manager.ts` | 60 | copyFileSync, cpSync, existsSync, lstatSync, mkdirSync, readdirSync, readFileSync, readlinkSync, rmSync, statSync, symlinkSync, unlinkSync, writeFileSync |
|
||||
| `src/utils/claude-symlink-manager.ts` | 27 | copyFileSync, existsSync, lstatSync, mkdirSync, readdirSync, readlinkSync, renameSync, rmSync, statSync, symlinkSync, unlinkSync |
|
||||
| `src/utils/shell-completion.ts` | 23 | appendFileSync, copyFileSync, existsSync, mkdirSync, readFileSync, statSync |
|
||||
| `src/web-server/routes/settings-routes.ts` | 23 | copyFileSync, existsSync, mkdirSync, readFileSync, renameSync, statSync, writeFileSync |
|
||||
| `src/utils/claude-dir-installer.ts` | 21 | copyFileSync, cpSync, existsSync, lstatSync, mkdirSync, readdirSync, renameSync, rmSync, statSync, unlinkSync, writeFileSync |
|
||||
| `src/cliproxy/binary/version-cache.ts` | 20 | existsSync, mkdirSync, readFileSync, unlinkSync, writeFileSync |
|
||||
| `src/management/recovery-manager.ts` | 20 | copyFileSync, existsSync, mkdirSync, renameSync, writeFileSync |
|
||||
| `src/web-server/routes/cliproxy-stats-routes.ts` | 20 | closeSync, existsSync, fstatSync, mkdirSync, openSync, readdirSync, readFileSync, readSync, renameSync, statSync, writeFileSync |
|
||||
| `src/web-server/routes/misc-routes.ts` | 20 | copyFileSync, existsSync, mkdirSync, readdirSync, readFileSync, renameSync, statSync, writeFileSync |
|
||||
| `src/web-server/routes/persist-routes.ts` | 17 | closeSync, copyFileSync, existsSync, lstatSync, openSync, readdirSync, readSync, renameSync, unlinkSync, writeFileSync |
|
||||
| `src/cliproxy/__tests__/pool-routing-phase3.test.ts` | 96 | existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync |
|
||||
| `src/cliproxy/config/__tests__/config-generator.test.js` | 88 | existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync |
|
||||
| `src/management/shared-manager.ts` | 86 | copyFileSync, cpSync, existsSync, lstatSync, mkdirSync, readdirSync, readFileSync, readlinkSync, rmSync, statSync, symlinkSync, unlinkSync, writeFileSync |
|
||||
| `src/cliproxy/config/__tests__/claude-model-neutral.test.ts` | 63 | existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, statSync, writeFileSync |
|
||||
| `src/cliproxy/accounts/__tests__/account-safety-quota-exhaustion.test.ts` | 48 | existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync |
|
||||
| `src/cliproxy/accounts/__tests__/account-registry-integrity.test.ts` | 45 | existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync |
|
||||
| `src/cliproxy/executor/__tests__/variant-port-integration.test.js` | 36 | existsSync, mkdirSync, readdirSync, readFileSync, rmSync, unlinkSync, writeFileSync |
|
||||
| `src/cliproxy/executor/__tests__/composite-variant-service.test.ts` | 33 | existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync |
|
||||
| `src/cliproxy/executor/__tests__/variant-port-edge-cases.test.js` | 33 | existsSync, mkdirSync, readdirSync, rmSync, unlinkSync, writeFileSync |
|
||||
| `src/utils/browser/mcp-installer.ts` | 32 | chmodSync, copyFileSync, existsSync, mkdirSync, readFileSync, renameSync, statSync, unlinkSync, writeFileSync |
|
||||
|
||||
## Top Legacy Shim Marker Files
|
||||
|
||||
| File | Marker Count |
|
||||
|---|---:|
|
||||
| `src/auth/profile-detector.ts` | 18 |
|
||||
| `src/utils/config-manager.ts` | 13 |
|
||||
| `src/auth/profile-detector.ts` | 11 |
|
||||
| `src/config/unified-config-loader.ts` | 9 |
|
||||
| `src/commands/setup-command.ts` | 7 |
|
||||
| `src/management/checks/config-check.ts` | 6 |
|
||||
| `src/web-server/routes/account-routes.ts` | 6 |
|
||||
| `src/config/migration-manager.ts` | 5 |
|
||||
| `src/api/services/profile-writer.ts` | 4 |
|
||||
| `src/cliproxy/quota-fetcher-gemini-cli.ts` | 4 |
|
||||
| `src/auth/profile-registry.ts` | 3 |
|
||||
| `src/cliproxy/__tests__/pool-onboarding-phase5.test.ts` | 12 |
|
||||
| `src/cliproxy/executor/__tests__/variant-port-allocation.test.js` | 12 |
|
||||
| `src/config/schemas/websearch.ts` | 10 |
|
||||
| `src/commands/cursor-command-display.ts` | 9 |
|
||||
| `src/config/migration-manager.ts` | 9 |
|
||||
| `src/cliproxy/config/__tests__/env-builder-provider-url.test.ts` | 8 |
|
||||
| `src/cliproxy/executor/__tests__/variant-port-edge-cases.test.js` | 8 |
|
||||
| `src/cliproxy/config/__tests__/config-generator.test.js` | 7 |
|
||||
|
||||
## Explicit Shim/Re-export Files
|
||||
|
||||
- `src/cliproxy/openai-compat-manager.ts`
|
||||
- `src/cliproxy/__tests__/model-catalog-compat.test.ts`
|
||||
- `src/cliproxy/ai-providers/__tests__/codex-plan-compatibility.test.ts`
|
||||
- `src/cliproxy/ai-providers/__tests__/openai-compat-manager.test.js`
|
||||
- `src/cliproxy/ai-providers/openai-compat-manager.ts`
|
||||
- `src/cliproxy/types/__tests__/types-backward-compat.test.ts`
|
||||
- `src/utils/profile-compat.ts`
|
||||
- `src/web-server/services/compatible-cli-docs-registry.ts`
|
||||
## Maintainability Metrics
|
||||
|
||||
| Metric | Value |
|
||||
|---|---:|
|
||||
| typed-error adoption (typed/total throws) | 0.9% (4/431) |
|
||||
| typed-error adoption (P4 locked subdomains) | 0.0% (0/23), target 40% |
|
||||
| hotpath console.error/warn occurrences | 931 (1091 total, 160 CLI-UX exempt) |
|
||||
| hotpath console.error/warn files | 134 |
|
||||
| files with createLogger | 35/685 |
|
||||
| subdomains with zero createLogger | 20 (api, bin, channels, cliproxy, cliproxy/accounts, cliproxy/ai-providers, cliproxy/binary, cliproxy/config, cliproxy/executor, cliproxy/management, cliproxy/quota, cliproxy/routing, cliproxy/sync, cliproxy/types, config, delegation, dispatcher, docker, shared, types) |
|
||||
| files > 400 LOC | 95 |
|
||||
| files > 600 LOC | 45 |
|
||||
|
||||
### Top Hotpath console.error/warn Files
|
||||
|
||||
| File | console.error/warn |
|
||||
|---|---:|
|
||||
| `src/utils/error-manager.ts` | 142 |
|
||||
| `src/cliproxy/accounts/account-safety.ts` | 56 |
|
||||
| `src/cliproxy/config/model-config.ts` | 32 |
|
||||
| `src/cliproxy/executor/arg-parser.ts` | 26 |
|
||||
| `src/dispatcher/flows/settings-flow.ts` | 26 |
|
||||
| `src/copilot/copilot-executor.ts` | 24 |
|
||||
| `src/delegation/delegation-handler.ts` | 23 |
|
||||
| `src/dispatcher/cli-argument-parser.ts` | 22 |
|
||||
| `src/web-server/routes/cliproxy-stats-routes.ts` | 22 |
|
||||
| `src/cliproxy/executor/lifecycle-manager.ts` | 16 |
|
||||
| `src/cursor/cursor-profile-executor.ts` | 16 |
|
||||
| `src/dispatcher/profile-resolver.ts` | 16 |
|
||||
| `src/cliproxy/auth/antigravity-responsibility.ts` | 15 |
|
||||
| `src/cliproxy/executor/auth-coordinator.ts` | 14 |
|
||||
| `src/cliproxy/executor/model-warnings.ts` | 13 |
|
||||
|
||||
### Files > 400 LOC (top 15)
|
||||
|
||||
| File | LOC |
|
||||
|---|---:|
|
||||
| `src/management/shared-manager.ts` | 1631 |
|
||||
| `src/web-server/routes/cliproxy-auth-routes.ts` | 1502 |
|
||||
| `src/cliproxy/auth/oauth-handler.ts` | 1453 |
|
||||
| `src/cursor/cursor-executor.ts` | 1234 |
|
||||
| `src/cliproxy/quota/quota-fetcher-gemini-cli.ts` | 1130 |
|
||||
| `src/commands/cliproxy/quota-subcommand.ts` | 1130 |
|
||||
| `src/web-server/routes/cliproxy-stats-routes.ts` | 1103 |
|
||||
| `src/cliproxy/quota/quota-fetcher.ts` | 1087 |
|
||||
| `src/commands/persist-command.ts` | 1071 |
|
||||
| `src/web-server/model-pricing.ts` | 1070 |
|
||||
| `src/web-server/routes/settings-routes.ts` | 1040 |
|
||||
| `src/cliproxy/proxy/tool-sanitization-proxy.ts` | 1039 |
|
||||
| `src/cliproxy/config/env-builder.ts` | 1037 |
|
||||
| `src/cliproxy/auth/oauth-process.ts` | 1018 |
|
||||
| `src/cliproxy/config/generator.ts` | 1012 |
|
||||
|
||||
@@ -57,6 +57,37 @@ if git show-ref --verify --quiet "refs/remotes/origin/$BASE_BRANCH"; then
|
||||
fi
|
||||
fi
|
||||
|
||||
# Hardening inventory freshness: the maintainability metrics artifact must be
|
||||
# regenerated within 30 days so the burndown stays current. Runs only after the
|
||||
# skip conditions above (CCS_SKIP_PREPUSH_GATE, detached HEAD, behind origin).
|
||||
HARDENING_JSON="docs/reports/hardening-inventory.json"
|
||||
if [[ ! -f "$HARDENING_JSON" ]]; then
|
||||
echo "[X] Missing $HARDENING_JSON."
|
||||
echo " Regenerate with: bun run report:hardening"
|
||||
exit 1
|
||||
fi
|
||||
HARDENING_TS=""
|
||||
# If the working-tree copy differs from HEAD (contributor regenerated but not
|
||||
# yet committed), use filesystem mtime; otherwise use the last commit time,
|
||||
# which is stable across CI clones (checkout resets mtimes) and so correctly
|
||||
# flags a stale committed artifact.
|
||||
if git diff --quiet -- "$HARDENING_JSON" 2>/dev/null && git diff --cached --quiet -- "$HARDENING_JSON" 2>/dev/null; then
|
||||
HARDENING_TS=$(git log -1 --format=%ct -- "$HARDENING_JSON" 2>/dev/null)
|
||||
else
|
||||
HARDENING_TS=$(stat -f %m "$HARDENING_JSON" 2>/dev/null || stat -c %Y "$HARDENING_JSON" 2>/dev/null)
|
||||
fi
|
||||
if [[ -n "$HARDENING_TS" ]]; then
|
||||
NOW_TS=$(date +%s)
|
||||
AGE_DAYS=$(( (NOW_TS - HARDENING_TS) / 86400 ))
|
||||
if (( AGE_DAYS > 30 )); then
|
||||
echo "[X] Hardening inventory is stale (${AGE_DAYS}d old; max 30d)."
|
||||
echo " Regenerate with: bun run report:hardening"
|
||||
echo " Then commit docs/reports/hardening-inventory.{json,md}."
|
||||
exit 1
|
||||
fi
|
||||
echo "[i] Hardening inventory fresh (${AGE_DAYS}d old; max 30d)."
|
||||
fi
|
||||
|
||||
echo "[i] Running CI-parity local checks..."
|
||||
# `set -euo pipefail` above makes every step fail fast. Keep these commands
|
||||
# explicit so parity drift is visible when CI changes.
|
||||
|
||||
@@ -3,6 +3,8 @@
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
|
||||
const { collectMaintainabilityMetrics } = require('./maintainability-metrics.js');
|
||||
|
||||
const ROOT_DIR = path.resolve(__dirname, '..');
|
||||
const SRC_DIR = path.join(ROOT_DIR, 'src');
|
||||
const REPORT_DIR = path.join(ROOT_DIR, 'docs', 'reports');
|
||||
@@ -430,6 +432,7 @@ function buildReport() {
|
||||
.filter((file) => /shim|re-export|compat/i.test(path.basename(file)))
|
||||
),
|
||||
},
|
||||
maintainability: collectMaintainabilityMetrics(ROOT_DIR),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -489,6 +492,55 @@ function renderMarkdown(report) {
|
||||
lines.push('- _none_');
|
||||
}
|
||||
|
||||
const m = report.maintainability;
|
||||
if (m) {
|
||||
lines.push('## Maintainability Metrics');
|
||||
lines.push('');
|
||||
lines.push('| Metric | Value |');
|
||||
lines.push('|---|---:|');
|
||||
lines.push(
|
||||
`| typed-error adoption (typed/total throws) | ${(m.typedErrors.adoptionRatio * 100).toFixed(1)}% (${m.typedErrors.typedThrows}/${m.typedErrors.totalThrows}) |`
|
||||
);
|
||||
lines.push(
|
||||
`| typed-error adoption (P4 locked subdomains) | ${(m.typedErrorAdoption.ratio * 100).toFixed(1)}% (${m.typedErrorAdoption.numerator}/${m.typedErrorAdoption.denominator}), target 40% |`
|
||||
);
|
||||
lines.push(
|
||||
`| hotpath console.error/warn occurrences | ${m.hotpathConsoleErrors.hotpathOccurrences} (${m.hotpathConsoleErrors.totalOccurrences} total, ${m.hotpathConsoleErrors.exemptOccurrences} CLI-UX exempt) |`
|
||||
);
|
||||
lines.push(`| hotpath console.error/warn files | ${m.hotpathConsoleErrors.filesAffected} |`);
|
||||
lines.push(
|
||||
`| files with createLogger | ${m.loggerCoverage.filesWithCreateLogger}/${m.loggerCoverage.totalSourceFiles} |`
|
||||
);
|
||||
lines.push(
|
||||
`| subdomains with zero createLogger | ${m.loggerCoverage.subdomainsWithZeroCreateLogger.length} (${m.loggerCoverage.subdomainsWithZeroCreateLogger.join(', ') || 'none'}) |`
|
||||
);
|
||||
lines.push(`| files > 400 LOC | ${m.largeFiles.countOver400} |`);
|
||||
lines.push(`| files > 600 LOC | ${m.largeFiles.countOver600} |`);
|
||||
lines.push('');
|
||||
lines.push('### Top Hotpath console.error/warn Files');
|
||||
lines.push('');
|
||||
lines.push('| File | console.error/warn |');
|
||||
lines.push('|---|---:|');
|
||||
for (const item of m.hotpathConsoleErrors.topFiles) {
|
||||
lines.push(`| \`${item.file}\` | ${item.count} |`);
|
||||
}
|
||||
if (m.hotpathConsoleErrors.topFiles.length === 0) {
|
||||
lines.push('| _none_ | 0 |');
|
||||
}
|
||||
lines.push('');
|
||||
lines.push('### Files > 400 LOC (top 15)');
|
||||
lines.push('');
|
||||
lines.push('| File | LOC |');
|
||||
lines.push('|---|---:|');
|
||||
for (const item of m.largeFiles.topOver400) {
|
||||
lines.push(`| \`${item.file}\` | ${item.loc} |`);
|
||||
}
|
||||
if (m.largeFiles.topOver400.length === 0) {
|
||||
lines.push('| _none_ | 0 |');
|
||||
}
|
||||
lines.push('');
|
||||
}
|
||||
|
||||
lines.push('');
|
||||
return lines.join('\n');
|
||||
}
|
||||
@@ -510,17 +562,23 @@ function main() {
|
||||
console.log(
|
||||
`[hardening-inventory] legacy markers total=${report.legacyShim.totalMarkers}, files=${report.legacyShim.filesAffected}`
|
||||
);
|
||||
if (report.maintainability) {
|
||||
const mt = report.maintainability;
|
||||
console.log(
|
||||
`[hardening-inventory] maintainability: typed-adoption=${(mt.typedErrors.adoptionRatio * 100).toFixed(1)}% (${mt.typedErrors.typedThrows}/${mt.typedErrors.totalThrows}), locked=${(mt.typedErrorAdoption.ratio * 100).toFixed(1)}% (${mt.typedErrorAdoption.numerator}/${mt.typedErrorAdoption.denominator}), console.error=${mt.hotpathConsoleErrors.hotpathOccurrences}, zero-logger-subdomains=${mt.loggerCoverage.subdomainsWithZeroCreateLogger.length}, >400LOC=${mt.largeFiles.countOver400}`
|
||||
);
|
||||
}
|
||||
console.log(`[hardening-inventory] wrote ${relJson}`);
|
||||
console.log(`[hardening-inventory] wrote ${relMd}`);
|
||||
}
|
||||
|
||||
if (require.main === module) {
|
||||
main();
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
buildReport,
|
||||
collectSyncCallSites,
|
||||
renderMarkdown,
|
||||
stripComments,
|
||||
};
|
||||
|
||||
if (require.main === module) {
|
||||
main();
|
||||
}
|
||||
@@ -0,0 +1,349 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Maintainability metrics collector for the CCS CLI maintainability epic.
|
||||
*
|
||||
* Computes the metrics the epic tracks across phases:
|
||||
* - typed-error adoption (throw new <TypedError> vs throw new Error vs other)
|
||||
* - createLogger coverage by subdomain (which subdomains have zero)
|
||||
* - hotpath console.error / console.warn call-site count (CLI-UX exempt)
|
||||
* - files > 400 / 600 LOC
|
||||
* - typed-error adoption in the P4 LOCKED denominator subdomains
|
||||
*
|
||||
* Accuracy relies on stripping comments/strings before regex matching. We
|
||||
* reuse hardening-inventory.js#stripComments for that. The require is lazy
|
||||
* (inside sanitize()) because hardening-inventory.js requires THIS module at
|
||||
* its top level; a top-level require back would capture hardening-inventory's
|
||||
* partial module.exports during the load cycle (it reassigns module.exports at
|
||||
* the bottom). A function-scope require resolves against the fully-loaded
|
||||
* module at call time, so the cycle is harmless.
|
||||
*
|
||||
* Approximate by design (grep-based). Method documented in
|
||||
* docs/hardening-debt-burndown.md. Re-baseline when the schema changes.
|
||||
*/
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
|
||||
const SOURCE_EXTENSIONS = new Set(['.ts', '.tsx', '.js', '.jsx', '.mjs', '.cjs']);
|
||||
|
||||
// CCSError subclasses (src/errors/error-types.ts). A `throw new <OneOfThese>`
|
||||
// counts as TYPED. `throw new Error(...)` is plain. Any other `throw new X(`
|
||||
// is "other" (error subclass outside the canonical taxonomy).
|
||||
const TYPED_ERROR_CLASSES = new Set([
|
||||
'CCSError',
|
||||
'ConfigError',
|
||||
'NetworkError',
|
||||
'AuthError',
|
||||
'BinaryError',
|
||||
'ProviderError',
|
||||
'ProfileError',
|
||||
'ProxyError',
|
||||
'MigrationError',
|
||||
'UserAbortError',
|
||||
'ValidationError',
|
||||
'RetryableError',
|
||||
]);
|
||||
|
||||
// P4 LOCKED denominator: typed-error adoption is measured ONLY over these
|
||||
// subdomains so the >40% goal cannot be gamed by narrowing scope. Emits both
|
||||
// numerator and denominator counts alongside the ratio.
|
||||
const TYPED_ADOPTION_SUBDOMAINS = ['cliproxy/quota', 'cliproxy/auth', 'web-server/routes', 'auth'];
|
||||
|
||||
// CLI-UX print surfaces exempt from the hotpath console.error sweep (P3).
|
||||
// Diagnostics here are legitimate user-facing terminal output, not loggable
|
||||
// errors, and stay on stdout/stderr via utils/ui.
|
||||
const CLI_UX_EXEMPT_PREFIXES = ['src/commands/', 'src/management/', 'src/utils/ui/'];
|
||||
|
||||
const THROW_NEW_REGEX = /\bthrow\s+new\s+([A-Za-z_$][A-Za-z0-9_$]*)\s*\(/g;
|
||||
const CONSOLE_ERR_WARN_REGEX = /\bconsole\s*\.\s*(?:error|warn)\s*\(/g;
|
||||
const CREATE_LOGGER_REGEX = /\bcreateLogger\s*\(/;
|
||||
|
||||
function sanitize(sourceText) {
|
||||
// Lazy require; see module header for the circular-dependency rationale.
|
||||
return require('./hardening-inventory.js').stripComments(sourceText);
|
||||
}
|
||||
|
||||
function toPosixPath(filePath) {
|
||||
return filePath.split(path.sep).join('/');
|
||||
}
|
||||
|
||||
function isSourceFile(filePath) {
|
||||
return SOURCE_EXTENSIONS.has(path.extname(filePath));
|
||||
}
|
||||
|
||||
function isTestFile(relPath) {
|
||||
return (
|
||||
/(?:^|\/)(?:__tests__|tests?)\//.test(relPath) ||
|
||||
/\.test\./.test(relPath) ||
|
||||
/\.spec\./.test(relPath)
|
||||
);
|
||||
}
|
||||
|
||||
function isCliUxExempt(relPath) {
|
||||
return CLI_UX_EXEMPT_PREFIXES.some(function (prefix) {
|
||||
return relPath.startsWith(prefix);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Subdomain key for a src-relative path.
|
||||
* src/cliproxy/quota/x.ts -> "cliproxy/quota"
|
||||
* src/auth/x.ts -> "auth"
|
||||
* src/x.ts -> "<root>"
|
||||
*/
|
||||
function subdomainOf(relPath) {
|
||||
const rest = relPath.replace(/^src\//, '');
|
||||
const parts = rest.split('/');
|
||||
if (parts[0] === 'cliproxy' && parts.length > 2) {
|
||||
return parts[0] + '/' + parts[1];
|
||||
}
|
||||
return parts[0] || '<root>';
|
||||
}
|
||||
|
||||
function round4(n) {
|
||||
return Math.round(n * 10000) / 10000;
|
||||
}
|
||||
|
||||
// Items may be file-shaped ({file,count}) or subdomain-shaped ({subdomain,count});
|
||||
// the tiebreaker key is whichever label field is present.
|
||||
function labelOf(item) {
|
||||
return item.file || item.subdomain || '';
|
||||
}
|
||||
|
||||
function topByCount(items, limit) {
|
||||
return items
|
||||
.slice()
|
||||
.sort(function (a, b) {
|
||||
return b.count - a.count || labelOf(a).localeCompare(labelOf(b));
|
||||
})
|
||||
.slice(0, limit);
|
||||
}
|
||||
|
||||
/** Classify `throw new X(` occurrences in sanitized source. */
|
||||
function classifyThrows(sourceText) {
|
||||
const sanitized = sanitize(sourceText);
|
||||
const counts = { typed: 0, plain: 0, other: 0, total: 0 };
|
||||
const re = new RegExp(THROW_NEW_REGEX.source, 'g');
|
||||
let match;
|
||||
while ((match = re.exec(sanitized)) !== null) {
|
||||
const identifier = match[1];
|
||||
counts.total += 1;
|
||||
if (identifier === 'Error') {
|
||||
counts.plain += 1;
|
||||
} else if (TYPED_ERROR_CLASSES.has(identifier)) {
|
||||
counts.typed += 1;
|
||||
} else {
|
||||
counts.other += 1;
|
||||
}
|
||||
}
|
||||
return counts;
|
||||
}
|
||||
|
||||
/** Count console.error / console.warn call sites in sanitized source. */
|
||||
function countConsoleErrors(sourceText) {
|
||||
const sanitized = sanitize(sourceText);
|
||||
const re = new RegExp(CONSOLE_ERR_WARN_REGEX.source, 'g');
|
||||
return (sanitized.match(re) || []).length;
|
||||
}
|
||||
|
||||
/** True if the file directly creates a logger via createLogger(...). */
|
||||
function hasCreateLogger(sourceText) {
|
||||
return CREATE_LOGGER_REGEX.test(sanitize(sourceText));
|
||||
}
|
||||
|
||||
/** Raw line count of a source file (matches `wc -l` content semantics). */
|
||||
function countLoc(sourceText) {
|
||||
if (!sourceText) return 0;
|
||||
const parts = sourceText.split(/\r?\n/);
|
||||
return sourceText.endsWith('\n') ? parts.length - 1 : parts.length;
|
||||
}
|
||||
|
||||
function walkFiles(dirPath) {
|
||||
const output = [];
|
||||
let entries;
|
||||
try {
|
||||
entries = fs.readdirSync(dirPath, { withFileTypes: true });
|
||||
} catch (_err) {
|
||||
return output;
|
||||
}
|
||||
for (const entry of entries) {
|
||||
if (entry.name === 'node_modules' || entry.name === 'dist') continue;
|
||||
const fullPath = path.join(dirPath, entry.name);
|
||||
if (entry.isDirectory()) {
|
||||
output.push.apply(output, walkFiles(fullPath));
|
||||
continue;
|
||||
}
|
||||
if (entry.isFile() && isSourceFile(fullPath)) output.push(fullPath);
|
||||
}
|
||||
return output;
|
||||
}
|
||||
|
||||
function emptyAggregates() {
|
||||
return {
|
||||
typedBySubdomain: {},
|
||||
loggerBySubdomain: {},
|
||||
hotpathByFile: [],
|
||||
hotpathTotal: 0,
|
||||
hotpathExempt: 0,
|
||||
hotpathNonExempt: 0,
|
||||
hotpathFilesAffected: 0,
|
||||
filesWithLogger: 0,
|
||||
totalSourceFiles: 0,
|
||||
typedTotal: 0,
|
||||
typedTyped: 0,
|
||||
typedPlain: 0,
|
||||
typedOther: 0,
|
||||
over400: [],
|
||||
over600: [],
|
||||
};
|
||||
}
|
||||
|
||||
function ensureBucket(obj, key, factory) {
|
||||
if (!obj[key]) obj[key] = factory();
|
||||
return obj[key];
|
||||
}
|
||||
|
||||
/**
|
||||
* Walk <rootDir>/src and aggregate maintainability metrics.
|
||||
* Returns the `maintainability` block merged into the hardening inventory.
|
||||
*/
|
||||
function collectMaintainabilityMetrics(rootDir) {
|
||||
const srcDir = path.join(rootDir, 'src');
|
||||
const files = walkFiles(srcDir);
|
||||
const agg = emptyAggregates();
|
||||
|
||||
for (const fullPath of files) {
|
||||
const relPath = toPosixPath(path.relative(rootDir, fullPath));
|
||||
if (isTestFile(relPath)) continue;
|
||||
const sourceText = fs.readFileSync(fullPath, 'utf8');
|
||||
const sub = subdomainOf(relPath);
|
||||
|
||||
const throws = classifyThrows(sourceText);
|
||||
agg.typedTotal += throws.total;
|
||||
agg.typedTyped += throws.typed;
|
||||
agg.typedPlain += throws.plain;
|
||||
agg.typedOther += throws.other;
|
||||
if (throws.total > 0) {
|
||||
const bucket = ensureBucket(agg.typedBySubdomain, sub, function () {
|
||||
return { typed: 0, plain: 0, other: 0, total: 0 };
|
||||
});
|
||||
bucket.typed += throws.typed;
|
||||
bucket.plain += throws.plain;
|
||||
bucket.other += throws.other;
|
||||
bucket.total += throws.total;
|
||||
}
|
||||
|
||||
agg.totalSourceFiles += 1;
|
||||
const hasLogger = hasCreateLogger(sourceText);
|
||||
if (hasLogger) agg.filesWithLogger += 1;
|
||||
const lc = ensureBucket(agg.loggerBySubdomain, sub, function () {
|
||||
return { files: 0, withLogger: 0 };
|
||||
});
|
||||
lc.files += 1;
|
||||
if (hasLogger) lc.withLogger += 1;
|
||||
|
||||
const ce = countConsoleErrors(sourceText);
|
||||
if (ce > 0) {
|
||||
agg.hotpathTotal += ce;
|
||||
if (isCliUxExempt(relPath)) {
|
||||
agg.hotpathExempt += ce;
|
||||
} else {
|
||||
agg.hotpathNonExempt += ce;
|
||||
agg.hotpathFilesAffected += 1;
|
||||
agg.hotpathByFile.push({ file: relPath, count: ce });
|
||||
}
|
||||
}
|
||||
|
||||
const loc = countLoc(sourceText);
|
||||
if (loc > 600) agg.over600.push({ file: relPath, loc: loc });
|
||||
if (loc > 400) agg.over400.push({ file: relPath, loc: loc });
|
||||
}
|
||||
|
||||
const adoptionRatio = agg.typedTotal > 0 ? agg.typedTyped / agg.typedTotal : 0;
|
||||
|
||||
let numerator = 0;
|
||||
let denominator = 0;
|
||||
for (const sub of TYPED_ADOPTION_SUBDOMAINS) {
|
||||
const bucket = agg.typedBySubdomain[sub];
|
||||
if (bucket) {
|
||||
numerator += bucket.typed;
|
||||
denominator += bucket.total;
|
||||
}
|
||||
}
|
||||
const typedAdoptionRatio = denominator > 0 ? numerator / denominator : 0;
|
||||
|
||||
const subdomainsWithZeroCreateLogger = Object.keys(agg.loggerBySubdomain)
|
||||
.filter(function (k) {
|
||||
return agg.loggerBySubdomain[k].files > 0 && agg.loggerBySubdomain[k].withLogger === 0;
|
||||
})
|
||||
.sort();
|
||||
|
||||
const topOver400 = agg.over400
|
||||
.slice()
|
||||
.sort(function (a, b) {
|
||||
return b.loc - a.loc || a.file.localeCompare(b.file);
|
||||
})
|
||||
.slice(0, 15);
|
||||
|
||||
return {
|
||||
typedErrors: {
|
||||
totalThrows: agg.typedTotal,
|
||||
typedThrows: agg.typedTyped,
|
||||
plainThrows: agg.typedPlain,
|
||||
otherThrows: agg.typedOther,
|
||||
adoptionRatio: round4(adoptionRatio),
|
||||
topSubdomainsByThrows: topByCount(
|
||||
Object.keys(agg.typedBySubdomain).map(function (k) {
|
||||
const v = agg.typedBySubdomain[k];
|
||||
return { subdomain: k, count: v.total, typed: v.typed, plain: v.plain };
|
||||
}),
|
||||
10
|
||||
),
|
||||
},
|
||||
typedErrorAdoption: {
|
||||
subdomains: TYPED_ADOPTION_SUBDOMAINS.slice(),
|
||||
numerator: numerator,
|
||||
denominator: denominator,
|
||||
ratio: round4(typedAdoptionRatio),
|
||||
targetRatio: 0.4,
|
||||
},
|
||||
loggerCoverage: {
|
||||
filesWithCreateLogger: agg.filesWithLogger,
|
||||
totalSourceFiles: agg.totalSourceFiles,
|
||||
coverageRatio: round4(
|
||||
agg.totalSourceFiles > 0 ? agg.filesWithLogger / agg.totalSourceFiles : 0
|
||||
),
|
||||
subdomainsWithZeroCreateLogger: subdomainsWithZeroCreateLogger,
|
||||
topSubdomainsByFiles: topByCount(
|
||||
Object.keys(agg.loggerBySubdomain).map(function (k) {
|
||||
const v = agg.loggerBySubdomain[k];
|
||||
return { subdomain: k, count: v.files, withLogger: v.withLogger };
|
||||
}),
|
||||
10
|
||||
),
|
||||
},
|
||||
hotpathConsoleErrors: {
|
||||
totalOccurrences: agg.hotpathTotal,
|
||||
exemptOccurrences: agg.hotpathExempt,
|
||||
hotpathOccurrences: agg.hotpathNonExempt,
|
||||
filesAffected: agg.hotpathFilesAffected,
|
||||
topFiles: topByCount(agg.hotpathByFile, 15),
|
||||
},
|
||||
largeFiles: {
|
||||
countOver400: agg.over400.length,
|
||||
countOver600: agg.over600.length,
|
||||
topOver400: topOver400,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
collectMaintainabilityMetrics: collectMaintainabilityMetrics,
|
||||
classifyThrows: classifyThrows,
|
||||
countConsoleErrors: countConsoleErrors,
|
||||
hasCreateLogger: hasCreateLogger,
|
||||
countLoc: countLoc,
|
||||
TYPED_ERROR_CLASSES: TYPED_ERROR_CLASSES,
|
||||
TYPED_ADOPTION_SUBDOMAINS: TYPED_ADOPTION_SUBDOMAINS,
|
||||
};
|
||||
@@ -0,0 +1,148 @@
|
||||
import { describe, expect, test } from 'bun:test';
|
||||
|
||||
const fs = require('fs');
|
||||
const os = require('os');
|
||||
const path = require('path');
|
||||
|
||||
const {
|
||||
classifyThrows,
|
||||
countConsoleErrors,
|
||||
hasCreateLogger,
|
||||
countLoc,
|
||||
collectMaintainabilityMetrics,
|
||||
} = require('../../../scripts/maintainability-metrics.js');
|
||||
|
||||
describe('maintainability-metrics.classifyThrows', () => {
|
||||
test('counts plain Error, typed, and other throws; ignores re-throws', () => {
|
||||
const source = [
|
||||
'throw new Error("plain");',
|
||||
'throw new ConfigError("typed-a");',
|
||||
'throw new AuthError("typed-b");',
|
||||
'throw new FoobarError("other");',
|
||||
'throw rethrownErr;', // not `throw new`, ignored
|
||||
].join('\n');
|
||||
|
||||
const result = classifyThrows(source);
|
||||
expect(result.total).toBe(4);
|
||||
expect(result.plain).toBe(1);
|
||||
expect(result.typed).toBe(2);
|
||||
expect(result.other).toBe(1);
|
||||
});
|
||||
|
||||
test('ignores throws inside comments and string/template literals', () => {
|
||||
const source = [
|
||||
'// throw new Error("commented")',
|
||||
'const s = "throw new Error(\\"in string\\")";',
|
||||
'const t = `throw new Error("in template")`;',
|
||||
'/* throw new Error("block") */',
|
||||
'throw new NetworkError("real");',
|
||||
].join('\n');
|
||||
|
||||
const result = classifyThrows(source);
|
||||
expect(result.total).toBe(1);
|
||||
expect(result.typed).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('maintainability-metrics.countConsoleErrors', () => {
|
||||
test('counts console.error and console.warn, ignores log/info/debug', () => {
|
||||
const source = [
|
||||
'console.error("a");',
|
||||
'console . warn("b");', // tolerate whitespace
|
||||
'console.log("c");',
|
||||
'console.info("d");',
|
||||
'console.debug("e");',
|
||||
'// console.error("commented")',
|
||||
].join('\n');
|
||||
|
||||
expect(countConsoleErrors(source)).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('maintainability-metrics.hasCreateLogger', () => {
|
||||
test('detects createLogger adoption and ignores comments', () => {
|
||||
expect(hasCreateLogger("const logger = createLogger('foo:bar');")).toBe(true);
|
||||
expect(hasCreateLogger('console.log("no logger here");')).toBe(false);
|
||||
expect(hasCreateLogger('// const logger = createLogger("commented");')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('maintainability-metrics.countLoc', () => {
|
||||
test('counts raw lines with wc -l semantics', () => {
|
||||
expect(countLoc('a\nb\nc')).toBe(3);
|
||||
expect(countLoc('a\nb\nc\n')).toBe(3); // trailing newline -> still 3
|
||||
expect(countLoc('')).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('maintainability-metrics.collectMaintainabilityMetrics (fixtures tree)', () => {
|
||||
test('aggregates throws, console errors, logger coverage, and LOC over a tree', () => {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'mtep-fix-'));
|
||||
|
||||
// src/auth/a.ts: 1 typed throw, 1 hotpath console.error, no createLogger
|
||||
fs.mkdirSync(path.join(root, 'src', 'auth'), { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(root, 'src', 'auth', 'a.ts'),
|
||||
[
|
||||
"import { AuthError } from '../../errors/error-types';",
|
||||
"throw new AuthError('typed');",
|
||||
"console.error('hotpath diagnostic');",
|
||||
].join('\n')
|
||||
);
|
||||
|
||||
// src/commands/b.ts: 1 plain throw, 1 console.error (CLI-UX exempt), no createLogger
|
||||
fs.mkdirSync(path.join(root, 'src', 'commands'), { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(root, 'src', 'commands', 'b.ts'),
|
||||
['console.error(\'cli print\');', "throw new Error('plain');"].join('\n')
|
||||
);
|
||||
|
||||
// src/cliproxy/quota/q.ts: createLogger present, 1 plain throw, 0 console.error
|
||||
fs.mkdirSync(path.join(root, 'src', 'cliproxy', 'quota'), { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(root, 'src', 'cliproxy', 'quota', 'q.ts'),
|
||||
["import { createLogger } from '../../../services/logging';", 'const logger = createLogger();', "throw new Error('quota plain');"].join('\n')
|
||||
);
|
||||
|
||||
// non-source file is ignored
|
||||
fs.writeFileSync(path.join(root, 'src', 'auth', 'README.md'), '# not source\n');
|
||||
|
||||
// test file is ignored (would otherwise inflate console.error)
|
||||
fs.mkdirSync(path.join(root, 'src', 'auth', '__tests__'), { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(root, 'src', 'auth', '__tests__', 'a.test.ts'),
|
||||
"console.error('test noise');\n"
|
||||
);
|
||||
|
||||
const metrics = collectMaintainabilityMetrics(root);
|
||||
|
||||
// throws: typed 1 (AuthError), plain 2 (Error x2), other 0
|
||||
expect(metrics.typedErrors.totalThrows).toBe(3);
|
||||
expect(metrics.typedErrors.typedThrows).toBe(1);
|
||||
expect(metrics.typedErrors.plainThrows).toBe(2);
|
||||
|
||||
// console errors: total 2 non-test (auth + commands); exempt 1 (commands); hotpath 1 (auth)
|
||||
expect(metrics.hotpathConsoleErrors.totalOccurrences).toBe(2);
|
||||
expect(metrics.hotpathConsoleErrors.exemptOccurrences).toBe(1);
|
||||
expect(metrics.hotpathConsoleErrors.hotpathOccurrences).toBe(1);
|
||||
expect(metrics.hotpathConsoleErrors.filesAffected).toBe(1);
|
||||
|
||||
// logger coverage: 3 source files, 1 with createLogger (cliproxy/quota)
|
||||
expect(metrics.loggerCoverage.totalSourceFiles).toBe(3);
|
||||
expect(metrics.loggerCoverage.filesWithCreateLogger).toBe(1);
|
||||
expect(metrics.loggerCoverage.subdomainsWithZeroCreateLogger).toContain('auth');
|
||||
|
||||
// P4 LOCKED denominator: auth (1 typed / 1 total) + cliproxy/quota (0 typed / 1 total)
|
||||
// -> numerator 1, denominator 2
|
||||
expect(metrics.typedErrorAdoption.subdomains).toEqual([
|
||||
'cliproxy/quota',
|
||||
'cliproxy/auth',
|
||||
'web-server/routes',
|
||||
'auth',
|
||||
]);
|
||||
expect(metrics.typedErrorAdoption.numerator).toBe(1);
|
||||
expect(metrics.typedErrorAdoption.denominator).toBe(2);
|
||||
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user