PRISMCreate structured JSON prompts for automated transcript quality analysis. v23 — What's new
agentcustomerMap each output value to Positive, Negative, or Neutral (excluded from confusion matrix). Defaults: Yes → Positive, No/NA → Negative.
Ordinal questions (e.g. Excellent / Meets / Needs Improvement) are auto-detected and use MAE instead of F1/Variance — their Rating Config mappings are ignored during metric computation. You can set them to Neutral here for clarity, or leave them mapped; either way the Mode column in the Per Question table will show Ordinal.
| Output Value | Designation |
|---|---|
| Click "Refresh from cards" to populate. | |
Name this run to track it in ⑤ Run Comparison.
⚠ No human scores loaded — run will proceed but cannot be saved for comparison.
#f97316).program_scorecard_yymmdd if it is empty.Tx 13/30 · 43% · ETA 5m 5s. ETA appears after the second call completes and uses elapsed time per call × remaining calls.requestAnimationFrame wrapper ensures the browser has painted the injected HTML before window.print() is called.IP3) rather than positional placeholders (Q0.1), eliminating model confusion that caused it to return QIP3 keys the parser did not recognise._resolveQid now maps both bare IDs (IP3) and Q-prefixed variants (QIP3) to the canonical question ID, making the parser resilient to model formatting variations.event_type: bench_run). The dashboard shows a dedicated Bench Runs KPI card, an amber series on the activity chart, bench token usage in the Tokens by Model chart, a Bench Runs column in the By User table, and a Bench filter + amber badge in the Event Log.Question # column) now populates scorecard data for every question. The Develop panel pre-fills with the question text, rating options, rating guidelines, instructions to evaluators, verbatim examples, and notes.disqualifying_operator and reflected in the generated override_rule.source_conditions[].operator fields.output_rule hardened — Source results are now described as final and binding with an explicit instruction not to override them with independent transcript reasoning. The justification format no longer asks for a verbatim transcript quote, removing the foothold that allowed models to rationalise overriding a source result.getDepCompoundSources now captures sourceQid alongside the card ID, and the restore path (and dep-map modal resolver) now prefers QID-based lookup over positional card-ID lookup, which is stable across reimports.Speaker A:, Speaker B:, etc., ready for tag substitution.rebuildClassificationCues ran after tag restoration. Fixed with a guard that preserves incoming prevCues values and only fills gaps from the live DOM.taxonomy_mode always multi_pass — The field was hardcoded; it now derives from the Passes selector (single_pass when set to 1, multi_pass otherwise).taxResults is now unconditionally cleared on import before any saved results are restored.source_results from prior phases injected, preserving correct dependency resolution.{"Q{id}": {"output", "justification", "confidence"}}. A partial-extraction fallback handles truncated responses, recovering any fully-formed entries.max_tokens is scaled by question count and capped per provider (Groq: 8 192; others: 131 072) to prevent 400 errors.option field — The positional option: N field (never read by the evaluator) has been removed from all option objects across phrase, behavior, dependency, and metadata card types.required_behaviors — required_behaviors is now only emitted when non-empty, matching the existing pattern for optional_behaviors. Applies to behavior, dependency, and metadata party conditions.exact_wording_required from phrase cards — Always the exact inverse of semantic_match / verbatim_match, so it was redundant. Removed from both matching rule branches.compound_operator — Only emitted on dependency cards when there are 2+ source questions. Has no combinatory effect with a single source.operator on source entries — operator is now omitted on the first entry of both dependencies[] and source_conditions[]. The first entry has no sibling to combine with, making it noise.filter_key duplication in metadata cards — filter_key was emitted three times (card level, applies_when, and FILTER_MISSING_OR_INVALID condition). Now only at the authoritative card-level filter field.{ logic: 'ANY', rubric: [] } shape. Now standardised to {} and { source_conditions: [] } respectively.required_behaviors are now described as global to the option (not per-source), resolving a structural ambiguity.FILTER_RULE rewritten as two numbered directives; justification instruction moved to the end; ~45% shorter.< 0.6.DEPENDENCY_RULE constant — Dead code superseded by buildDependencyRule(); deleted.⚠ badge in the header, matching the Prompts to Run indicator.↗ link next to each dep card QID navigates directly to that card in the main UI (closes modal, scrolls, flashes). Source QID labels in the expanded body are similarly clickable to jump to the source card.{ProgramName}-YYYYMMDD_HHmmss filename convention and are greyed out until results are available.Justification requirement: line directly into the system message, overriding the default justification format for computational questions./v1/chat/completions endpoint. No RunPod API key required; model name is sent in the payload.Authorization: Bearer auth. Ping check uses /v1/models instead of /health.POD_CONFIGS map with url, model, and optional apiKey fields. Adding future models is a one-line change.interaction_id, filter_key_value) and the Test Runner automatically injects the correct filter key value into each prompt at run time, so the model knows which filter rule to apply. The map persists with Save/Load Project.filter_rules[].parties in the generated JSON. Fixed: a party is now included if it has a name or any behaviors defined.← Q1.2, Q1.3); a ⚠ badge appears whenever a dep question is selected but one or more of its source questions is not, warning of a potential missing-context issue.z-index to position:sticky header cells).MyProgram-qa-test-results-2026-03-06.csv).question_id / task, flexible header aliases) and creates blank behavior cards pre-populated with Q ID and task text; replaces existing cards after confirmation⚠ N badge appears on the Test Prompts button when any cards are dirty_qaBuilderProject file via the File System Access API automatically links it for auto-save (subsequent saves overwrite in place)35% (−5%)); requires human data to be loaded in the Compare tabv1, v2, …); filenames include version and date for easy identification.json file; fully round-trips on importrefreshPreview was firing during session restore before card fields were populated, causing a blank re-render that wiped visible content; a restore guard flag now prevents preview refresh until all card state is fully applied<th> element for reliable browser tooltip behaviour)output_rule automatically appends a directive instructing the model to retrieve and use those source question results directly, with multi-source matching instructionsDEPENDENCY_RULE constant to explicitly instruct the model that source question results are already computed and must be treated as ground truthsource_conditions directly; only evaluate the transcript if required_behaviors are explicitly specified alongside source conditionsoutput_rule in the JSON output — no new field, just enriches the existing rule string› OUTPUT RULE to expanddraggable="true" baked in, which the browser intercepts before any JS event can stop itmousedown temporarily enables dragging on the parent row, and dragend clears it — text selection now works normally</div> (from a --- separator) that closed the scroll container before any content entered itv13) now shown in the page subtitle and next to the header badgefilter_values: ["A", "B"] array instead of single filter_value string<input> to <textarea rows="2"> across all card types so long descriptions wrap naturallygetFvLogic(null) was throwing TypeError because .logic-select elements were removed from fv-block and party-block headers in an earlier pass; getFvLogic is now null-safeinp.value to '' before calling addTag/addAwTag, so any duplicate event fire is rejected at the top of the handlerselectFvPill function added back after being dropped in a failed patch run)/health poll — eliminates the queue-blocking chicken-and-egg problem that was causing 20-minute waits.xlsx, .xls, .csv support to the transcript dropzone using SheetJSshowSaveFilePicker) so you can choose the save folder; graceful fallback for unsupported browsersswitchType now copies the current task value across all type panels before switchingtrShouldStop flag with AbortController so Stop immediately cancels in-flight fetchestrRun in try/finally so UI cleanup always fires even on errortrDelay is abort-aware and doesn't wait out the full sleep after Stop— instead of Error: Stoppedbrow-tier-cell styles that were causing unstyled/unclickable tier buttonsmin_optional/health endpoint; shows queue depth and worker counts in run log{ condition: "else" } in JSON/v1/chat/completions is appended automatically.