feat: harden support, localize knowledge and add operator availability [skip ci]

Source-only publication of reviewed commit 64e96a7f423673dc29ad56485be9fc79bce8af1c.

Passed in isolation: focused frontend/backend tests, race checks, TypeScript typecheck, lint, Go compilation, Flowgram build, and deterministic SDK build/tests.

Not accepted as a production release: Next.js export/native assembly remains blocked by environment memory limits; browser interaction and MySQL staging acceptance remain unverified. No production server, database/schema, runtime configuration, deployment, release tag, or workflow execution was performed.

Expected source tree: 0a4a7528e9.
This commit is contained in:
2026-10-16 15:21:42 +00:00
parent 299ea50fb5
commit 2f7a491666
176 changed files with 11869 additions and 2716 deletions

View File

@@ -0,0 +1,109 @@
# Customer identity and live-access upgrade
These are operator instructions for a reviewed staging rollout, not commands
executed by this change. Back up data and verify the exact schema/engine before
any production upgrade. Existing customer and conversation records are retained.
## Additive schema and issuer boundaries
`CustomerIdentity` adds:
- `identity_scope`: indexed varchar(80), default empty for legacy rows.
- `scoped_key`: nullable varchar(64) with a unique index. Legacy rows remain NULL.
The normal GORM startup migration adds these definitions. No destructive index
replacement, automatic legacy merge, or data deletion is supplied. Test this
additive change on a backup with the deployment's actual SQLite/MySQL version.
A public identity scope is `v1:<jwt|hmac|guest>:SHA256(trimmed channel code)`.
The server derives it; clients cannot select an issuer or inject this scope.
The unique claim key is SHA-256 of the JSON array containing scope, external
source and trimmed external ID. Use the implementation's
`ScopedCustomerIdentityKey` helper rather than a hand-built delimiter format.
New public registrations insert the claim in the same customer transaction.
A competing claim rolls back rather than creating two authorized customers.
WebSocket routing uses canonical customer ID and channel record ID. Incoming
subscriptions and outgoing recipients are checked against current ownership.
Raw external IDs are not cross-channel or cross-trust routing keys.
## Existing sessions and history
Old customer-session JWTs without the new scope are rejected. The updated client
must exchange a fresh token. Until legacy mappings are explicitly reconciled,
a newly authenticated scoped identity can receive a separate customer record,
not the old account's history. The original records remain available to
appropriately authorized operators.
Before enabling traffic:
1. Back up data; stop exchanges/writes for the affected scope during reconciliation.
2. Run the read-only `scripts/audit-customer-identity-scopes.sql` reports against an
approved staging copy after the additive columns exist. Keep report output
private: it contains customer identifiers.
3. Establish channel and issuer ownership using trustworthy host-side evidence.
Legacy `user` rows alone cannot distinguish JWT from HMAC provenance.
4. For a proven unique mapping, review a transaction setting both its scope and
computed claim key while retaining its customer ID and conversation history.
Check all associated conversation channels before making a change.
5. Stop on duplicate, inactive, multi-channel or mixed-trust mappings. Never take
the first row or merge customers solely because external IDs match. If a new
scoped record already exists, reconcile both histories explicitly.
6. Reissue customer tokens and verify authorized history, isolation, and rollback
behavior on the staging copy before resuming traffic.
A channel reassigned to a different identity issuer needs a new channel code.
Ordinary signing-key rotation intentionally retains the same issuer scope.
Use a distinct host JWT signing secret for each independently trusted channel.
The existing host JWT contract validates the channel's secret and expiry, but
does not require a channel audience claim. Reusing a signing key across unrelated
channels lets the holder assert an identity to either channel; scoped storage
is not a defense against a shared issuer key. HMAC proofs bind the channel code
inside the signed payload. No mandatory JWT audience rollout is introduced here.
Legacy duplicate repair remains an operator-reviewed migration, not an automatic
startup side effect. The nullable unique claim allows dirty legacy rows to
remain isolated without blocking new correctly scoped registrations.
## Anonymous and authenticated-only channels
Unsigned exchange issues a new random server-owned anonymous identity. Supplying
an old guest external ID is not proof of ownership. Normal resume and refresh
use the signed customer-session token. Near-expiry, still-valid credentials are
renewed through the rate-limited `POST /api/customer/session_refresh` endpoint,
which authenticates the existing session and never creates an anonymous identity.
A refresh failure is not silently replaced with a fresh guest. Losing or expiring
that credential does not authorize reclaiming history by an external ID.
An optional boolean `requireSignedIdentity` in a web channel's `configJson`
rejects unsigned exchanges and existing guest sessions server-side. Default
false retains anonymous channels. The SDK's `requireUserToken` is not a substitute
for this policy.
Use the existing authenticated channel GET and
`POST /api/dashboard/channel/update` flow with `channel.update` permission,
preserving all unrelated configuration and required fields. No dedicated new UI
toggle is provided. Older editors which omit this property preserve a configured
true value; an explicit JSON false is needed to disable it. Null and nonboolean
values are rejected. This source change does not configure a live channel.
## Staff and customer live-session revocation
Staff sockets bind their original login-session ID and absolute expiry, without
serializing that internal metadata into API responses. Fresh session, account,
role and permission state is checked for incoming operations, event fanout,
actual queued writes and idle checks. Scope changes terminate the connection;
reconnection must obtain fresh authority. Idle checks run every five seconds;
data operations revalidate immediately rather than using a five-second cache.
Notification sockets cannot subscribe to conversation topics. Dashboard
conversation sockets require conversation-view permission. Customer connections
recheck token expiry, channel policy, customer state and exact identity mapping.
This implementation deliberately makes database reads at authorization
boundaries. Measure the cost under realistic fanout; any later caching must have
complete revocation invalidation rather than silently reintroducing stale access.
Channel configuration updates and token-secret resets compare the previously read
configuration bytes and update timestamp before writing. A stale writer receives
a conflict instead of silently overwriting a newly enabled signed-identity policy.
Reload the channel and reapply the intended change after reviewing that conflict.

71
docs/hardening/README.md Normal file
View File

@@ -0,0 +1,71 @@
# Security and reliability hardening
This independent fork starts from the current source tree of
[MonoVerseNS/agent-desk-ng](https://github.com/MonoVerseNS/agent-desk-ng), revision
`b06954b0a495389cbb5194e0d388342eabe60636`. It does not import the old Git history.
The upstream Apache-2.0 license and notices remain applicable. This document
records changes made by this fork; it is not a claim of production certification.
## Integration baseline
The isolated integration reconciles public Gitea main `299ea50f` with the reviewed
hardening branch `63e04578`. It preserves main's bounded connection attempts,
abortable ongoing HTTP recovery and System setup navigation alongside the
identity/live-access/draft/setup safeguards. Only the clean public repository
history is used. See `VALIDATION.md` for executed checks and unresolved gates.
## Release gate
The hardening changes must be reviewed and validated before deployment. Source
inspection, a compiler check, a passing isolated test, and live acceptance are
separate kinds of evidence. See `VALIDATION.md` for the actual checks performed.
No production service, database, credentials, or external deployment is changed
by committing this source code.
## Changes being integrated
- Separate guest, signed-user, issuer and channel identities. Use canonical
customer/channel WebSocket topics and recheck recipients instead of treating
a topic subscription as authorization.
- Apply conversation-view permission to staff WebSockets; keep notifications
separate. Revalidate session expiry, revocation, account state and permissions
on long-lived connections. Disabled roles must no longer grant permissions.
- Keep drafts and asynchronous responses bound to their identity and dialogue.
Preserve drafts on failed sends, protect newly typed revisions, and prevent
duplicate concurrent sends. Profile refresh and delayed API errors must not
restore or expire the wrong signed-in account.
- Turn setup repair into an explicit preview/apply operation. Never resurrect
deleted configuration or silently introduce continuous working hours. Apply
reviewed writes transactionally and dispatch only after commit.
- Make empty setup, initial retry and standalone chat dismissal truthful and
recoverable; remove bearer-token-bearing WebSocket URLs from browser logs.
## Upgrade precautions
Customer identity scoping is an additive schema and token-contract change, not
only a frontend patch. Existing unscoped customer tokens will be rejected.
Legacy identities and conversations must be inventoried and reconciled with
trusted issuer/channel provenance before expecting their old history in new
customer sessions. No automatic customer merge or destructive cleanup is
appropriate. See `CUSTOMER_IDENTITY_UPGRADE.md` before any deployment.
New anonymous sessions are server-issued. Knowing an old anonymous external ID
is not sufficient to reclaim its history. Preserve the signed customer-session
credential for normal resume; loss or expiry can require a new anonymous session.
The setup quick-fix API now requires preview and explicit apply. Older callers
must not treat a successful preview response as proof that configuration changed.
See [the setup contract](SETUP_PREVIEW.md) before integrating custom callers.
## Verification priorities
1. Guest/user/channel collisions, unauthorized staff topics, revoked/expired
sockets, disabled roles and concurrent identity registration.
2. Wrong-recipient drafts, failed and duplicate sends, delayed responses after
switching conversations or accounts, and logout during profile refresh.
3. Preview/apply tampering, permission denial, rollback, deleted configuration,
off-hours schedules, empty-channel simulation and dispatch failure reporting.
4. Desktop and narrow-screen interaction checks in Russian, English and Chinese.
Use temporary, isolated data for checks. Do not run migration/repair commands
against a live database merely to validate the code.

View File

@@ -0,0 +1,62 @@
# Reviewed setup repairs
The existing authenticated `POST /api/dashboard/setup/quick_fix` endpoint now uses a two-phase contract. It does not write configuration unless the caller explicitly applies a current preview.
## Contract
1. Request a preview with `action` and, for an existing agent/channel/team, its `entity_id`. Omit `apply` or set it to `false`.
2. Review `changes`. Each entry names the entity, operation, and exact operational fields before and after. New linked entities are identified by their proposed names until IDs are assigned. Audit timestamps and acting-user metadata are maintained automatically.
3. Submit the same action and target with `apply: true` and the returned `preview_token`.
The response includes:
- `applied`: whether an explicit apply request completed. An empty `applied_actions` array means no configuration writes were needed.
- `changes`: the reviewed plan; an empty array is a valid no-op, not a claim that setup is complete.
- `preview_token`: a fingerprint of the acting user, action, target, proposed changes, and current setup configuration.
- `applied_actions`: committed operation/entity identifiers, empty on preview.
- `warning_codes`: manual follow-up requirements or post-commit failures.
- `readiness`: the refreshed diagnostic, omitted if refreshing failed.
Old callers that omit `apply` receive a read-only preview. Callers must not interpret any HTTP success as a successful mutation. There is deliberately no compatibility path that silently applies an unreviewed repair.
## Authorization and freshness
The handler authenticates every request. Setup checks the principal's resolved permission list, matching the normal authorization service. An `admin` or `super_admin` role name alone does not bypass missing permissions. Every planned create or field update also checks its specific permission.
Apply rebuilds the plan inside one transaction and verifies the fingerprint before writing. A stale preview is rejected and must be reviewed again. Transactional reads use separate GORM statements, preserving row locks without leaking a prior query's table or filters. Updates also compare the target's previous update timestamp. The fingerprint is a freshness/intent check, not an authentication credential or signed authorization grant.
## Deliberately narrow behavior
- Broad setup only creates missing defaults. It never rewrites existing configuration, moves existing operators to another team, expands an agent's team pool, or rebinds existing channel routes.
- Deleted records are not update candidates and are never restored. Disabled records and zero-rollout settings are preserved. A targeted “enable my profile” action can enable the authenticated user's non-deleted profile only when the preview explicitly shows the change and the required permissions are present.
- A targeted handoff change affects only the named eligible agent. Missing teams, route choices, disabled configurations, and rollout changes must be chosen in their existing editors.
- Quick fixes never create or modify shifts. Missing schedules and off-hours schedules are distinct diagnostics. Working hours must be entered explicitly in the existing schedule editor.
- A newly created default agent is unpublished. Setup does not publish agents. A default channel can be created only after an eligible agent has a published revision, matching the normal channel editor's publication guard. Readiness reports unpublished agents as incomplete.
Therefore a broad repair is not a promise of a production-ready routing path. Publishing, deliberate routing choices, and explicit schedules may still be required.
## Transactions and dispatch
All configuration writes in an apply operation share one transaction. A failed write rolls back the entire repair. Pending dispatch begins only after the configuration transaction commits.
A dispatch failure produces `dispatch_failed`; it does not undo committed changes or return an error implying that the configuration write should be retried. A diagnostic refresh failure similarly produces `readiness_refresh_failed`. The UI displays these warnings and requires a fresh preview for a subsequent repair.
The pending-queue query now reports database read errors and applies its existing batch limit in SQL. Its existing newest-first priority and periodic retry semantics are unchanged. This repair does not redesign queue fairness.
## Simulation
The simulator is read-only and sends no customer message. `candidate_operators` is always an array, including missing-channel responses; the client also normalizes legacy null values.
`bypass_ai: true` is explicitly rejected because the service does not implement AI bypass. AI-only paths remain blocked. AI-first/unknown service modes and partial channel rollout are reported as unverified, never as successful operator delivery. Only an eligible, published, full-channel-rollout, human-only route can produce a positive current-eligibility result. That result is a point-in-time diagnostic, not a delivery guarantee or an end-to-end AI execution test.
## Validation status
Focused regression tests cover preview-only behavior, explicit apply, stale previews, atomic rollback, deleted/disabled/zero-rollout preservation, named-target scope, off-hours versus missing schedules, role-name permission bypass, status-only permissions, empty arrays, simulation limitations, publication requirements, and post-commit failures.
The focused setup regressions and their race-enabled run passed in the isolated
2026-10-16 integration baseline. Go formatting/compilation, declared-dependency
frontend typecheck and Node regression tests were also executed. The remaining
combined-tree validation, frontend release build, actual MySQL upgrade and browser
acceptance gates are recorded in [VALIDATION.md](VALIDATION.md). No production
configuration or data was changed. Passing isolated tests does not establish
production readiness or authorize an upgrade.

View File

@@ -0,0 +1,185 @@
# Validation status: isolated checks, not a production release
Validated on 2026-10-16 against the integrated public-source baseline: Gitea main
`299ea50fb5bb6874c3fca96277fce187205ac20a` and hardening-review
`63e045788623e88f7fd7b44102e649e08f4c1f1c`, merged as `3868181`, with the follow-up
corrections listed below. This baseline does not include later knowledge-base
localization or operator-availability feature work. Its historical results are
retained below; the final combined-tree checks are recorded separately at the end
of this document.
## Isolation and toolchain
- Go 1.26.9, Node 24.19.0, pnpm 10.30.2, locked TypeScript 6.0.3/Next 16.2.11.
- Official Go archive checksum and downloaded Go module checksums verified.
Frontend dependencies installed with the frozen lockfile and lifecycle scripts
disabled. Dependency manifests/lockfiles were not changed.
- Tests ran with a cleared application environment, dedicated temporary/cache
directories, temporary or in-memory SQLite fixtures, mocked frontend APIs and
loopback WebSocket fixtures. No production configuration, credentials,
databases, application server, or external support messages were used.
- The service test harness rejects non-loopback HTTP requests before the
underlying transport/DNS, including redirects. Concrete Slack, Telegram,
Discord, Zalo and HTTP email clients were inspected to confirm their default
transport usage. The only test WebSocket dialer targets its own loopback server.
- Service fixtures explicitly replace the transitive AI-runtime hook with a
test-only no-op. Dedicated hook tests still install their own spy; AI-runtime
behavior is not claimed as covered by the service suite.
- `TestHandleRequest*` email/SMTP cases were excluded from the expanded service
run: raw SMTP does not use the HTTP guard. No unrestricted `go test ./...` was
run. This exclusion is an isolation limit, not a passing result.
## Historical baseline completed checks
- `gofmt` on all changed Go files and `git diff --check`.
- `go test ./internal/pkg/openidentity`: passed, including scoped issuer/channel,
signed-policy and signature-boundary tests.
- `GOMAXPROCS=2 go test -p 2 -race -tags dev ./internal/pkg/eventbus ./internal/pkg/openidentity`:
passed.
- Focused service security/setup/realtime suite with `-race -tags dev`: 46
top-level tests, 102 successful test/subtest events, no failures or skips.
Covers identity collisions/concurrent claims, live session revocation,
permission boundaries, actual WebSocket pumps, channel CAS, preview/apply,
rollback and dispatch outcomes, plus the transport guard and redirect check.
- `GOMAXPROCS=2 go test -p 2 -tags dev ./internal/services -skip '^TestHandleRequest' -count=1`:
passed, 217 top-level tests and 295 successful test/subtest events.
- `GOMAXPROCS=2 go build -p 2 -tags dev -buildvcs=false ./...`: passed. The `dev`
embedding mode is the repository CI test mode. VCS stamping was unavailable in
the isolated worktree and was disabled only for this compile check.
- `cd web && pnpm typecheck`: passed with declared locked dependencies before
the later six-file lint cleanup. The post-cleanup retry was killed with exit
137 without a TypeScript diagnostic; the final combined tree must rerun it.
- All frontend Node test files under app/components/hooks/i18n/lib:
226 tests passed, no failures or skips. This includes the SDK, draft revisions,
account/customer-session isolation, setup contract, abortable realtime recovery,
hidden/unmounted lifecycle and navigation checks.
- Four additional offline component-lifecycle regressions passed after the
six-file lint cleanup (bulk-move selection/stale responses, palette snapshots,
notification cleanup, editor/origin initialization). The earlier 226-test full
run predates these four additions.
- Full `pnpm lint`: exit 0 after localized lifecycle corrections, with zero
errors and 45 existing warnings. No lint rules were disabled.
## Corrections found by execution
- The old realtime test fixture lacked required conversation counters. It now
matches the DTO; the history/realtime merge assertion was retained unchanged.
- Async assignment notifications could outlive their test database. The eventbus
now exposes a drain for already-published handlers, and the fixture waits before
restoring global state. Nested publishes/error callbacks have a regression
test. No sleep or disabled race check is used.
- Setup background responses are fenced by effect lifetime/request generation.
The staff realtime hook captures its user ID in the effect rather than writing
a ref during render. VM test harness variable names satisfy Next ESLint rules.
- Four missing generic setup locale keys were supplied in English, Russian and
Chinese. Existing locale-integrity assertions are unchanged.
- Three stale source-test harnesses were aligned to current route groups and
component ownership while retaining realtime placement, accessible label,
trigger-style and collapsed-navigation checks.
- Seven existing lint errors across six components were corrected without rule
suppressions: browser-origin/palette hydration now uses external snapshots;
bulk-move targets reset on dialog/base changes and obsolete directory results
are ignored; initial notification requests cannot update after cleanup;
fullscreen initialization and toolbar construction no longer need the flagged
mount-state effect or mutable ref-bearing action list.
## Historical baseline remaining release gates
- Full-project lint has 45 existing warnings. The final feature integration must
rerun lint, typecheck and the complete Node suite after all source changes.
- The canonical frontend export (`pnpm build`) did not complete. An initial
cross-worktree node_modules symlink was replaced by a local offline frozen-lock
installation. The subsequent default Turbopack build reached optimized
compilation, then was killed with exit 137 and no source diagnostic while the
shared environment had little free memory. This is not a successful build; a
low-concurrency build on the final combined tree remains required. No build,
typecheck or security checks were disabled to bypass this result.
- Actual MySQL CAS/nullable-unique-index behavior and additive schema upgrade/
identity reconciliation on an approved staging backup have not been run.
- Browser acceptance at desktop/narrow widths in Russian, English and Chinese
has not been performed for this integrated baseline.
- Production migration, service startup and deployment have not been performed.
SQLite results do not establish MySQL behavior. Automated tests do not establish
visual acceptance or a safe production upgrade. Preserve the identity/token
upgrade precautions in `CUSTOMER_IDENTITY_UPGRADE.md` and preview/apply contract
in `SETUP_PREVIEW.md` before any rollout.
## Final combined source validation (2026-10-16)
The combined canonical checkout at `1ec72508cdda2dfe3b3eed3158f8ba930d76d260`
includes security/realtime integration, knowledge localization, operator
availability/calendar/presence, final locale changes and the two fixture-only
corrections described below. These results are distinct from the historical
baseline above and from the separately integrated CoverIt vendor tree.
The same sanitized toolchain, temporary SQLite fixtures and loopback-only test
transport were used. No application server, production database, credentials,
migration or external support message was used. Raw SMTP `TestHandleRequest*`
cases remain excluded, and the service-suite AI hook remains a test-only no-op;
neither scope is claimed as covered. No unrestricted `go test ./...` was run.
### Completed combined checks
- All changed Go files are `gofmt`-clean; `git diff --check` passed.
- `GOMAXPROCS=2 go test -p 2 -tags dev ./internal/services -skip '^TestHandleRequest' -count=1 -timeout=5m -json`:
passed, 240 top-level tests / 333 test-and-subtest events, no failures or skips.
The same audited scope with `-race` passed with identical counts and no race
report (181.272 seconds).
- Full race suites for `internal/pkg/eventbus`, `internal/pkg/openidentity` and
`internal/pkg/i18nx`: 48 top-level tests / 78 successful events.
- Focused dashboard availability permission/ACL race tests: 4 tests passed.
- Focused knowledge handler, builder and RAG localization race tests: 12
top-level tests / 18 successful events, with mocked provider callbacks and no
external application calls.
- `GOMAXPROCS=2 go build -p 2 -tags dev -buildvcs=false ./...`: passed. This is a
complete source compile in the repository's test embedding mode, not a binary
containing the production web export. VCS stamping was disabled only for the
local compile environment.
- `NODE_OPTIONS=--max-old-space-size=1024 pnpm typecheck`: passed.
- Every frontend `*.test.mjs` under app/components/hooks/i18n/lib, run with
`node --test --test-concurrency=1`: 262 tests in 14 suites passed; zero failures,
cancellations or skips.
- Full `pnpm lint`: exit 0, zero errors and 42 warnings. No rules were disabled.
The first combined service run identified two obsolete fixture assumptions,
not production-code failures. The realtime operator fixture now reuses the
existing `conversation.view` permission without changing its state. Only the
readiness success scenario that expects an eligible human now seeds an employee,
current permission, login session and bounded live-presence lease. Negative
permission scenarios and all original behavior assertions were retained. The
full normal and race results above were obtained after these fixture repairs.
### Asset assembly and packaging limit
- Both frontend projects used unchanged frozen lockfiles with dependency
lifecycle scripts disabled. Web dependencies totalled 978 locked packages;
Flowgram dependencies totalled 953. Official Go/module integrity checks passed.
- Flowgram's production `pnpm build` passed and generated its complete public
assets. Third-party parser warnings were reported, without build errors.
- The SDK's official `pnpm build:sdk` ran twice; both outputs were deterministic
and byte-identical to the tracked bundle. No generated SDK change is required.
- The default Next 16.2.11 Turbopack production build did not complete: optimized
compilation ended with exit 137 (`Killed`), without a source diagnostic.
- A sequential supported `next build --webpack` attempt, with a 1 GiB Node heap
and bounded workers, ended with a build-worker `SIGKILL`, also without a source
diagnostic. This was not a successful alternate export.
- One distinct experiment used the [documented memory optimization](https://nextjs.org/docs/app/guides/memory-usage)
`experimental.webpackMemoryOptimizations` option and a 640 MiB Node heap. It
ended with an explicit JavaScript heap-out-of-memory error and worker
`SIGABRT`. The temporary config was restored byte-for-byte afterward. No build
check, typecheck, security setting, font or application asset was bypassed or
mocked. Official Google font CSS and a real font asset were reachable through
the existing trusted proxy configuration.
- Dependency manifest and lockfile hashes, and the tracked SDK hash, remained
unchanged by assembly; no production source/config edit was retained. No native production-embedded binary was
assembled because a complete Next export was not available. Flowgram/SDK
success and the `-tags dev` Go compile do not satisfy that packaging gate.
The default frontend export and complete native assembly therefore still need
a suitably provisioned isolated build environment. These failures do not prove
a source defect, and the passed tests do not establish a deployable release.
Actual MySQL behavior, approved staging upgrade/identity reconciliation, real
browser desktop/narrow-width acceptance in Russian/English/Chinese and production
migration/startup/deployment remain unverified. Preserve all upgrade precautions
above. No cross-platform or LanceDB release assembly was attempted.