Operational intelligence
6 tasksRecovery, clarity, calibrated initiative, resource handling, decisions, and forensic sequences.
Burhuc AI Labs / Evaluation method
CertentIQ evaluation overview
A practical view of how CertentIQ tests a declared AI-agent configuration, separates task performance from security consistency, and binds observed results to reviewable evidence.
01 / Dimensions
What the current battery observes
The current repository declares 40 task cards across seven categories. Tasks exercise configured behaviour under bounded prompts, tools, fixtures, and repeated trials. They are engineering probes, not a complete model of intelligence or production risk.
Safety and policy
7 tasksNegative constraints, hidden instructions, scope, context, boundary pressure, and loss avoidance.
Technical resilience
6 tasksSupply-chain decisions, repository preservation, provenance, version integrity, and patch reasoning.
Financial and privacy
5 tasksAPI handling, idempotency, wallet policy, payment evasion, and personal-data redaction.
Adversarial defence
9 tasksPrompt protection, jailbreaks, escalation, memory poisoning, pressure, and social influence.
Classic robustness
3 tasksPerturbation handling, reward-hacking resistance, and canary-exfiltration behaviour.
Super-IQ probes
4 tasksDynamic ontology, complex formation, logic leakage, and recursive problem decomposition.
02 / Reading results
A score is one part of the record
Task outcome
Reports what the configured system did under the task fixtures and scorer used in that run.
Security consistency
Records whether any required repeated trial triggered a security gate, even when aggregate task performance passes.
Run validity
Suppresses the result when required provider calls fail because of quota, authentication, unsupported parameters, network errors, timeouts, or unavailable fixtures.
Evidence strength
Depends on runtime proof, evaluator identity, manifest binding, suite governance, trial distribution, validity, and expiry.
03 / Evidence chain
From configuration to public record
Each stage adds context needed to interpret a result. Missing evidence lowers confidence or blocks publication rather than being replaced with a stronger claim.
Declare
Freeze the agent prompt, model, provider, tools, runtime mode, policy, and system manifest.
Exercise
Run the versioned battery and required repeated trials against that declared configuration.
Record
Preserve outcomes, provider errors, security gates, evidence level, runtime proof, and validity.
Publish
Create a signed public record only when governance, evidence, validity, and owner-consent gates pass.
04 / Current limits
What remains unproven
These gaps block certification, compliance, production-safety, and validated-standard claims.
- Independent review of the current task cards and scorers.
- Preregistered multi-environment measurement and reproducibility studies.
- Evidence that development scores predict production outcomes.
- Independent penetration testing and deployment evidence for policy-enforcement controls.
- Qualified legal or conformity assessment for a specific regulatory scope.
Evaluate a declared system.
Run an operator evaluation or inspect the publication rules and signed records that are currently eligible for public display.