Capability

Bidirectional Health-Check Scenarios

Separate policy for entering failover and returning to normal — symmetry is a choice, not a default.

Most GSLB platforms treat health-check scenarios as one-way state machines: a condition becomes true, traffic shifts. The same condition becomes false, traffic shifts back. That symmetry is convenient — and frequently wrong. The real-world path into a failover is rarely the same as the path back. TR7 GTM scenarios define the activation direction and the deactivation direction independently. Each direction carries its own condition expression (combined AND/OR groups across health-check results), its own set of triggers to fire, and its own gating check. Operators decide: do we return to normal as soon as the primary recovers, or do we wait for an extended stability window? Do we re-run a synthetic transaction before returning traffic? Does an activation event email the SRE team, but a deactivation event also kick a deployment hook? This design lets enterprises express the asymmetric policies they actually want — fast in, slow out for safety; conservative in, fast out for SLA pressure; manual-confirm on the return path while keeping failover automated. The scenario itself is reusable: bound to multiple DNS records, attached to disaster-recovery flows, referenced across data-center pairs. Operators describe the state machine once; the platform applies it everywhere it is referenced.

2-way
Independent activation and deactivation paths
AND / OR
Combined-condition expression with negation
Auto + user
Auto-generated DC-pair scenarios + custom scenarios
Reusable
Bind one scenario to many records, DR flows and DCs

One-way scenarios force symmetric policy where business reality is asymmetric.

The standard GSLB pattern looks like this: monitor an endpoint, when it goes unhealthy shift DNS responses to the backup, when it goes healthy shift back. Implementations from major GSLB vendors treat both transitions as the inverse of the same condition.

Production reality is not symmetric. Failing over to a backup is a defensive move executed under pressure; returning to the primary is an offensive move executed under confidence. The conditions, the wait windows, the operator approvals, and the triggers that should fire are different in each direction.

Examples: fast-in / slow-out — failover on first detected error to protect the user, but require 15 minutes of clean health before returning to avoid a flap. Conservative-in / fast-out — require three consecutive failures before initiating failover, but return immediately when primary recovers to meet RPO/RTO commitments. Auto-in / manual-out — failover is automated, but the return path requires SRE acknowledgment after a runbook review.

None of these are expressible in a one-way scenario. The operator either picks one direction and lives with the wrong policy in the other, or wires together brittle custom scripts that drift from the GSLB's source of truth.

TR7 GTM bidirectional scenarios let activation and deactivation carry independent conditions, independent gating checks and independent trigger actions — the policy structure your incident response runbook already assumes.

Our approach

A scenario is a named, reusable state machine with two directions. Each direction is defined by a combined condition expression and a trigger set; the two directions do not need to be inverses.

Activation direction — when the scenario turns on

A combined condition expression evaluates the underlying health checks. When it returns true, the scenario activates. Optional triggers fire actions (HTTP/HTTPS webhooks, Oracle queries) and an optional gating check confirms the trigger should proceed.

Deactivation direction — when the scenario turns off

A separate combined condition expression evaluates a separate set of health checks. The deactivation path can be the inverse of activation, or it can require additional stability, additional probes, or different triggers entirely.

Combined conditions with AND/OR groups

Conditions are not single booleans — they are groups of health-check results joined with AND inside a group and OR across groups, with optional negation. The same DSL that drives traffic-rule logic on the ADC also drives scenario evaluation here.

Reusable across records, domains and data-center pairs

A scenario is defined once and referenced by name from DNS records, disaster-recovery configurations and cross-DC failover policy. Operators do not re-author the same logic in multiple places.

Capabilities

Bidirectional scenarios are the foundation of TR7 GTM's failover and recovery policy.

Combined-condition activation expression

Activation condition is built from health-check results across one or more health-check profiles. Groups join checks with AND; multiple groups join with OR. Each individual check can be negated. Operators express conditions like "(API is up AND database is up) OR (failover path A is up AND failover path B is up)" without scripting.

Combined-condition deactivation expression

Deactivation condition is independent of activation. Operators express conditions like "primary has been healthy for 15 minutes AND latency is below threshold" while activation may have been simply "primary is down."

Independent trigger sets per direction

Activation triggers and deactivation triggers are separate selectable sets. An activation event may notify the on-call SRE; a deactivation event may notify the SRE plus run a synthetic transaction plus emit a webhook to the deployment system.

Gating check before triggers fire

An optional gating condition runs before each direction's triggers execute. If the gating check returns false, the state transition still happens but the triggers do not fire. Use case: state transitions automatically, but external notifications only fire during business hours.

Three turn modes per direction: auto / on / off

Each direction supports three operator-selectable modes. Auto follows the condition expression. On forces the direction to activate regardless of conditions (manual override). Off disables the direction entirely (e.g., disable failback during a maintenance window).

Auto-generated scenarios for DC pairs

When two data centers are defined, TR7 GTM auto-generates four scenarios per pair: from-active, to-active, from-backup, to-backup — each with appropriate condition logic based on WAN access, LAN access and internet reachability checks. Operators can use the auto-generated scenarios as-is, customize them, or create their own from scratch.

Scenario-driven DNS record health state

DNS records can have their healthy/unhealthy state driven by a scenario rather than a static boolean or single health check. Per-record `cond` field accepts a scenario reference: when the scenario activates, the record is excluded from responses; when it deactivates, the record returns.

Scenario-driven disaster recovery

Disaster-recovery records can specify `drCond` — a scenario that determines when the DR record set replaces the primary record set in responses. The DR scenario evaluation is bidirectional, supporting controlled failover and controlled failback.

HTTP, HTTPS and Oracle triggers

Triggers fire as HTTP/HTTPS calls (custom URI, method, headers, body, expected status codes, content-match query) or Oracle database calls (configured SQL). Operators wire scenario activations into existing incident management, deployment, or audit pipelines.

Audit trail per state transition

Every scenario state change is recorded: which direction fired, which conditions evaluated true/false, which triggers ran, which gating check passed. Post-incident review reconstructs the exact sequence of automated decisions without manual log archaeology.

Operational depth

Scenarios are operated together with health-check definitions, trigger configurations, DNS record bindings, and disaster-recovery configurations.

01

Condition group semantics

Within a condition group, all listed checks must evaluate true (AND). Across groups, any one group evaluating true is sufficient (OR). The `!` suffix on a check ID negates it. The grouping structure is symmetric for activation and deactivation; each direction has its own group set.

02

Combined check ID space

Conditions reference health checks by ID. User-defined health-check profiles and auto-generated DC-pair checks share the same ID space. Operators mix manual and auto checks in the same condition group.

03

Turn mode interaction with deactivation

When activation is forced to on (manual override), deactivation evaluation typically continues — operators can manually activate, then let the deactivation condition decide when to restore. Forcing both directions to on creates a stuck state and is logged as a configuration warning.

04

Trigger payload and retry behaviour

Triggers fire with a structured payload carrying scenario ID, direction, evaluation timestamp, and the configuration snapshot at trigger time. Trigger failure (HTTP non-2xx, Oracle error) is logged and optionally retried per trigger profile.

05

Scenario evaluation cadence

Scenarios are evaluated on every health-check state change, not on a polling timer. The first state change that crosses an activation or deactivation threshold triggers the transition. Evaluation cost stays low because conditions reference precomputed health states.

06

Visibility into the current scenario state

Operators see the current state of every scenario (activated / deactivated), the time of the last transition, the last evaluation result for each condition group, and the trigger outcomes. The dashboard surfaces stuck transitions and conflicting overrides.

When to use it

Fast-in / slow-out failover

Activate failover on the first detected error to protect the user. Deactivate only after 15 minutes of clean primary health to avoid a flap. Different conditions, different timing — same scenario object.

Auto failover, manual failback

Failover is automated; the return path requires SRE acknowledgement. Deactivation direction is set to off; an operator manually flips it to on after runbook review. Activation continues to evaluate automatically.

Asymmetric inter-DC failover policy

DC A → DC B failover triggers an HTTP webhook to the incident management system. DC B → DC A failback triggers the same webhook plus a deployment-system call to re-warm caches. Triggers in each direction are independent.

Database-aware scenarios

Use the Oracle trigger to query a database before failover — for example, confirm the backup database has caught up via log shipping. The trigger result gates the actual state transition.

Frequently asked questions

Can I make activation and deactivation conditions identical?
Yes — that gives you traditional one-way behaviour. The bidirectional structure is opt-in; if both directions use the same condition expression, the scenario behaves like a classic on/off scenario. The architecture lets you express asymmetric policy when you want it without forcing it.
What happens if both activation and deactivation conditions evaluate true at the same time?
Activation wins. The scenario transitions to active and remains active until the deactivation condition is true and the activation condition is false. This avoids oscillation in edge cases where conditions overlap.
How are triggers different from the underlying health checks?
Health checks evaluate endpoint state. Triggers are actions that fire when the scenario transitions. Triggers can be HTTP/HTTPS calls (webhooks) or Oracle database calls. Health checks decide the scenario state; triggers communicate that state change to external systems.
Do bidirectional scenarios add latency to failover?
No. State transitions are evaluated on every health-check change, not on a polling timer. The first health-check result that crosses the threshold flips the scenario. The bidirectional structure adds policy expressiveness, not latency.
Can a scenario reference another scenario?
Conditions reference health checks (manual or auto). Scenario-of-scenarios composition is not directly expressible; instead, complex policies are built by binding scenarios to records and disaster-recovery configurations that themselves participate in higher-level decision logic.

Define the path in and the path back, independently.

Walk through a bidirectional scenario built on your own runbook: fast-in / slow-out, manual failback, asymmetric trigger sets — your policy, not the platform's default.