← All drills
Plexivo reliability report
Whole site lost: the Plexivo Brain detected it and rebuilt it automatically
Objective.
Lose every machine of a hosting site and let Plexivo rebuild it and move every resource, with no operator.
Scenario.
Every machine of a dedicated site was stopped at the provider.
Expected behaviour.
- Declare the site lost from independent signals.
- Build a replacement site.
- Move every application and database to it, with its data.
Measured recovery.
- Detection time
- 2 min 47 s
- Recovery time RTO
- 19 min 3 s
- How long recovery took.
- Recovery point RPO
- 0 s
- Every committed write kept.
- Application health
- Pass
- Database consistency
- Pass
Customer impact.
- Application availability
- App answering 933 s after the loss
- Management operations
- Fully automatic, no operator
- Data integrity
- Verified
- Committed data
- Fully preserved
What happened.
- Every machine stopped.
- Site declared lost; replacement ordered.
- Replacement site active.
- Every resource moved.
- App answering.
- PostgreSQL and MySQL answering; every row present.
Correctness checks.
- Committed data
- Fully preserved
Every measurement of this drill.
| Date | Measured recovery | Report |
|---|---|---|
| 7 min 4 s | View report | |
| 19 min 3 s | This report |
What we learned.
- A whole site is rebuilt and every resource moved in about 19 minutes, with no operator.
What we changed.
- A replacement always takes a free name.
- Projects follow their resources to the new site.
- A recovery that cannot finish waits for an operator instead of retrying.
Scope of this drill.
Measured with one App, one PostgreSQL and one MySQL database on a dedicated site.