(llm-first): context budget, validation, and eval harness, orchestration general-prompt
This commit is contained in:
MaKarin
@@ -1,98 +1,101 @@
|
||||
# Validation Rules
|
||||
|
||||
Validation is now a lightweight automated gate instead of a prose-only checklist.
|
||||
<!-- prompt-version: 2.0 -->
|
||||
<!-- applies-to: tools/validate-generation.mjs and npm run eval:generation -->
|
||||
<!-- validated-by: self -->
|
||||
|
||||
## Commands
|
||||
Use this document during the **Verification / Success Gate** stage defined in `prompts/general-prompt.md`.
|
||||
|
||||
- `npm run generate:domain-summary`
|
||||
- `npm run validate:generation`
|
||||
- `npm run validate:generation:runtime`
|
||||
## Purpose
|
||||
|
||||
Define the repository gates that convert a plausible generation run into a verified one.
|
||||
|
||||
## Primary Gates
|
||||
|
||||
- `node tools/validate-generation.mjs --artifacts-only`
|
||||
- `npm run eval:generation`
|
||||
|
||||
## Auxiliary Freshness Prep
|
||||
|
||||
- `npm run generate:api-summary`
|
||||
|
||||
Run the freshness prep when the repository validator or supporting tooling expects `api-summary.json` to exist and match the current DSL. This artifact is auxiliary to validation and inventory, not the generation source of truth.
|
||||
|
||||
## Prompt-Gate Alignment Rule
|
||||
|
||||
- Every invariant described as required in the active prompt corpus must either be enforced by this gate or be called out explicitly as a manual/runtime-only check.
|
||||
- Validation must not stay silent about a violation that the prompts describe as forbidden.
|
||||
- Validation must not report green buildability when build verification was skipped.
|
||||
- every invariant marked required in the active prompt corpus must either be enforced by a gate or called out as manual/runtime-only
|
||||
- validation must not silently ignore a forbidden pattern
|
||||
- build verification must not be reported as green when it was skipped
|
||||
|
||||
## Gate groups
|
||||
## Gate Groups
|
||||
|
||||
### Build checks
|
||||
### Build Checks
|
||||
|
||||
- at least one `domain/*.dsl` file exists
|
||||
- required artifacts exist
|
||||
- Prisma schema exists
|
||||
- frontend/backend env contracts exist
|
||||
- frontend/backend framework workspace files exist
|
||||
- `domain-summary.json` matches the current DSL
|
||||
- project `.env.example` files keep the working domain-based Keycloak examples unless explicitly overridden
|
||||
- at least one `domain/*.api.dsl` file exists
|
||||
- required artifacts exist:
|
||||
- `server/prisma/schema.prisma`
|
||||
- env examples
|
||||
- required scaffold files
|
||||
- auth/runtime/realm artifacts
|
||||
- if the current validator policy checks `api-summary.json`, it exists and is fresh relative to the DSL
|
||||
- `server/` remains a valid Nest workspace
|
||||
- `client/` remains a valid Vite workspace
|
||||
- generation must not pass validation if framework scaffolding files were deleted and replaced by a hand-written minimal skeleton
|
||||
- if dependencies are installed, build verification runs for `server/` and `client/`
|
||||
- if dependencies are installed, backend and frontend build verification runs
|
||||
- if dependencies are missing, build verification is reported as skipped with reason instead of green
|
||||
|
||||
### Auth checks
|
||||
### Auth Checks
|
||||
|
||||
- frontend auth seam files exist
|
||||
- backend auth seam files exist
|
||||
- `401` and `403` semantics stay split
|
||||
- `401` and `403` semantics remain split
|
||||
- auth code keeps the required Keycloak/JWT contracts
|
||||
- JWKS resolution chain matches the contract:
|
||||
- JWKS resolution order remains:
|
||||
1. explicit `KEYCLOAK_JWKS_URL`
|
||||
2. OIDC discovery
|
||||
3. certs fallback
|
||||
|
||||
### Filter checks
|
||||
### Filter And UI Checks
|
||||
|
||||
- list resources expose filter UI (including `FilterButton`)
|
||||
- list resources expose filter UI including `FilterButton`
|
||||
- reference filters use `ReferenceInput` + `AutocompleteInput` with `filterToQuery`
|
||||
- data provider preserves repeated query params for array filters
|
||||
- backend FK filters keep exact-match semantics
|
||||
- enum repeated params are mapped to Prisma `in`
|
||||
- typed form mapping is preserved:
|
||||
- `integer` / `decimal` -> `NumberInput`
|
||||
- `date` -> `DateInput`
|
||||
- reference fields intended for navigation keep `ReferenceField link="show"`
|
||||
- `dataProvider` preserves repeated query params for array filters
|
||||
- backend FK filters remain exact-match
|
||||
- repeated enum params map to Prisma `in`
|
||||
- Create/Edit forms keep type-correct inputs
|
||||
- navigable references keep `ReferenceField link="show"`
|
||||
- resources keep `show={...}` registration in `App.tsx`
|
||||
|
||||
### Natural-key checks
|
||||
### Natural-Key Checks
|
||||
|
||||
- response records expose `id`
|
||||
- route/update contracts use the real primary key
|
||||
- natural-key sort/update paths do not regress to a fake physical `id`
|
||||
|
||||
### Realm checks
|
||||
### Realm Checks
|
||||
|
||||
- a root `*-realm.json` artifact exists
|
||||
- realm roles exist
|
||||
- audience delivery exists
|
||||
- required claims are explicit
|
||||
- SPA/backend client structure is explicit
|
||||
- required roles, audience delivery, and claims remain explicit
|
||||
- SPA and backend client structure remains explicit
|
||||
|
||||
### Runtime checks
|
||||
### Runtime Checks
|
||||
|
||||
- compose topology stays PostgreSQL-only
|
||||
- Prisma lifecycle scripts remain in place
|
||||
- `/health` stays public
|
||||
- backend can execute `npm run build` inside `server/`
|
||||
- frontend can execute `npm run build` inside `client/` after dependencies are installed
|
||||
- client/server `.env.example` stay aligned with the working runtime defaults:
|
||||
- `https://sso.greact.ru`
|
||||
- `toir`
|
||||
- `toir-frontend`
|
||||
- `toir-backend`
|
||||
- `https://toir-frontend.greact.ru`
|
||||
- optional runtime execution mode runs:
|
||||
- `npx prisma generate`
|
||||
- `npx prisma migrate dev`
|
||||
- `npx prisma db seed`
|
||||
- Docker topology remains PostgreSQL-only
|
||||
- Prisma lifecycle commands remain available where required
|
||||
- `/health` remains public
|
||||
- backend build runs inside `server/`
|
||||
- frontend build runs inside `client/`
|
||||
- client/server `.env.example` stay aligned with repository defaults
|
||||
|
||||
### Scaffold checks
|
||||
### Output Contract Checks
|
||||
|
||||
- backend initialization starts from official Nest CLI scaffolding
|
||||
- frontend initialization starts from official Vite React TypeScript scaffolding
|
||||
- feature generation happens after scaffold creation, not instead of scaffold creation
|
||||
- repair happens before generation when workspace is degraded
|
||||
- required framework configs and entry files must survive subsequent LLM edits
|
||||
- every generated Create/Update DTO imports from `'class-validator'`
|
||||
- DTO fields have type-correct decorators
|
||||
- optional/nullable fields carry `@IsOptional()` before the type decorator
|
||||
- controllers carry the required guards and roles
|
||||
- React Admin components use correct input/field types
|
||||
|
||||
The automated gate is intentionally small. It enforces the critical reproducibility contract without turning the repository into a test platform or a generator engine.
|
||||
### Eval Harness
|
||||
|
||||
- `npm run eval:generation` runs fixture-based semantic checks
|
||||
- eval failures block completion
|
||||
- prompt changes that break evals are regressions, not acceptable simplifications
|
||||
|
||||
Reference in New Issue
Block a user