From 7582a523b5413016f54cd47e4c4511f12d22faf0 Mon Sep 17 00:00:00 2001 From: Patrick Birch <48594400+patrickbirch@users.noreply.github.com> Date: Thu, 5 Feb 2026 05:14:40 -0600 Subject: [PATCH] PXB-3666 [DOCS] - Add pages from 8.0 into 8.4 new file: docs/backup-strategy.md new file: docs/differences-logical-backup.md new file: docs/example-full-incr.md modified: mkdocs-base.yml --- docs/backup-strategy.md | 135 ++++++++++++++++++++++++++++ docs/differences-logical-backup.md | 55 ++++++++++++ docs/example-full-incr.md | 138 +++++++++++++++++++++++++++++ mkdocs-base.yml | 5 +- 4 files changed, 332 insertions(+), 1 deletion(-) create mode 100644 docs/backup-strategy.md create mode 100644 docs/differences-logical-backup.md create mode 100644 docs/example-full-incr.md diff --git a/docs/backup-strategy.md b/docs/backup-strategy.md new file mode 100644 index 00000000..f6dbf60c --- /dev/null +++ b/docs/backup-strategy.md @@ -0,0 +1,135 @@ +# Backup strategy and backup policy + +In conversation, "backup strategy," "backup policy," and "backup plan" are used interchangeably, but the terms have distinct meanings. + +A backup strategy is an overall approach or methodology for backing up the data in a database system. The strategy includes the decisions and considerations for what data to back up, how frequently backups should be taken, what backup methods and technologies to use, where to store the backup files, and what steps should be taken to ensure data integrity and the ability to recover. + +A backup strategy typically involves business requirements, the Recovery Point Objective (RPO), the maximum acceptable data loss in case of system failure or disaster, and the Recovery Time Objective (RTO), the acceptable downtime for restoring the system after a failure, and compliance regulations. + +A backup policy defines the guidelines, rules, or standards for specific procedures and practices for implementing the backup strategy. The policy outlines the following: + +* Detailed steps and parameters for performing backups +* Retention periods +* Backup types +* Backup verification methods +* Error handling procedures +* Security measures + +A backup plan, based on the backup strategy and the backup policy, is a detailed operation document that specifies how to execute the backup strategy in actionable steps and defines the timelines for backup activities. A backup plan can include the following details: + +* Who performs the backup +* When backups are scheduled +* How backups are monitored and audited +* What actions to take in case of backup failures or emergencies + +## Types of backup + +A [full backup](create-full-backup.md) copies all data to a storage location. This type of backup creates a complete copy of the data set. The benefit of a full backup is a quick recovery since the data is stored in one place. However, this type of backup is also the most time-consuming and storage-intensive; it duplicates all data regardless of whether it has changed since the last backup. + +An [incremental backup](create-incremental-backup.md) only copies data that has changed since the last backup, whether that was a full or another incremental backup. This method is faster and more storage-efficient than a full backup. However, the recovery can be slower because you must restore each set of changes in sequence. + +For more information, see the [example of using a full backup with incremental backups](example-full-incr.md). + +## Backup policy defined + +The backup policy sets the guidelines and objectives for data protection, while the backup strategy provides the roadmap for achieving those objectives. + +A backup policy may state that all customer data must be backed up daily, with a full backup at the start of each week and incremental backups taken each business day. These definitions are based on business and compliance needs. The policy also defines how long backups should be retained based on legal or regulatory requirements. + +The difference between a backup policy and a backup strategy lies in their scope and detail. A backup policy is more about the following: + +* What — What should be backed up and may include information on compliance and governance + +* When — When should you do backups + +The backup strategy contains the how — how the backups are scheduled and includes information on the backup technologies and the restoration process. The backup strategy details the use of specific backup software, the configuration of backup schedules, the process for monitoring backup success, and the necessary steps for restoring data. + +## Backup policy best practices + +An effective backup policy is crucial for safeguarding an organization's data integrity and availability. Here are some best practices to consider: + +| Best Practice | Description | +|---------------|-------------| +| Regularly Update and Review | Ensure the backup policy is a living document, updated to reflect changes in technology, business processes, and regulations. | +| Clearly Define Roles and Responsibilities | Assign specific individuals or teams for executing backups, monitoring success, and managing restoration. | +| Adopt the 3-2-1 Backup Rule | Have 3 data copies, store them on 2 different media types, and keep 1 copy offsite. | +| Ensure Comprehensive Coverage | The policy should cover all critical data, applications, and systems to avoid data loss gaps. | +| Encrypt Backup Data | Protect backups with encryption, especially for offsite or cloud storage. | +| Test Backups Regularly | Regularly test backup processes to ensure data can be effectively restored. | +| Implement a Disaster Recovery Plan | Integrate the backup policy with a broader disaster recovery plan for data loss scenarios. | +| Use Reliable Backup Software and Hardware | Invest in proven technologies with robust backup and restoration capabilities. | +| Monitor Backup Procedures | Continuously monitor procedures to ensure they function as intended and identify issues promptly. | +| Train Staff | Educate relevant staff on the importance of backups and their role in the process. | +| Document the Backup Process | Maintain thorough documentation of the backup process, including schedules, technologies, and restoration procedures. | +| Define Service Level Agreements (SLAs) | Establish clear SLAs for backup and recovery times to set expectations and ensure accountability. | +| Comply with Legal and Regulatory Requirements | Ensure the policy complies with relevant laws and regulations regarding data retention and protection. | +| Plan for Scalability | Design the policy to accommodate future data volume growth and infrastructure changes. | +| Conduct Periodic Risk Assessments | Regularly assess data risks and adjust the backup policy to mitigate those risks. | + +By adhering to these best practices, organizations can create a robust backup policy that protects their data and supports their business continuity and disaster recovery strategies. The effectiveness of a backup policy depends not only on its creation but on its implementation and ongoing management. Regular reviews, updates, and staff training ensure the backup policy remains relevant and effective. + +## Backup policy example + +An example of a backup policy might include the following components: + +| Aspect | Details | +|--------|---------| +| Purpose | To define the procedures and responsibilities for backing up organizational data to ensure its availability in case of data loss. | +| Scope | This policy applies to all data, hardware, and software in the organization's information systems. | +| Policy Statement | The organization shall maintain a regular backup schedule for all critical data. Full backups are to be performed weekly, with incremental backups occurring daily. All backups must be encrypted and stored in a secure, off-site location. | +| Roles and Responsibilities | The IT department is responsible for executing backups as scheduled. The department head will ensure that backup procedures are followed and reviewed regularly. | +| Backup Procedures | The IT department must use approved software and hardware. Data to be backed up includes, but is not limited to, customer information, financial records, and operational data. | +| Data Retention | Retain the backup data for five years or as required by law or industry regulations. | +| Disaster Recovery | In the event of data loss, the IT department shall follow the disaster recovery plan to restore data from the most recent backup. | +| Review and Testing | Review the backup policy and procedures annually. Test the backup and restoration procedures quarterly to ensure they are effective. | + +## Test the backups + +You must test the effectiveness of your backups and demonstrate that you can recover your data in the event of loss or corruption. Here are several methods to assess the reliability of your backup system: + +| Aspect | Details | +|--------|---------| +| Perform Regular Test Restores | The most reliable way to test backups is to periodically restore the data to a separate system. This verifies the backup's integrity and the restoration process's functionality. | +| Monitor Backup Reports | Regularly review backup reports and logs for any errors or failures during the backup process. This can help identify issues early before they become critical problems. | +| Check for Data Consistency | Compare the original data with the backed-up data to ensure consistency. This can be done by checking file sizes, using checksums, or employing other data validation techniques. You can use [pt-table-checksum](https://docs.percona.com/percona-toolkit/latest/pt-table-checksum.html) from Percona Toolkit. | +| Simulate Disaster Scenarios | Conducting a simulated disaster recovery exercise can help determine how well your backup system will perform under stress and identify any gaps in your recovery plan. | +| Audit Backup Procedures | Regular audits of the backup procedures can ensure that the backups are being performed in accordance with the established policies and that the backup system is functioning as expected. | +| Validate Backup Configuration | Ensure that the backup configuration is correct and that all necessary data is being backed up. This includes checking backup schedules, retention policies, and the scope of the data backed up. | +| Verify Security Measures | Confirm that the backup data is secure and that encryption and other security measures are functioning correctly to protect against unauthorized access. | +| Evaluate Recovery Time Objectives | Measure how long it takes to restore data from backups and compare this with your recovery time objectives to ensure they align with your business requirements. | +| Assess Backup Media Reliability | Regularly check the physical and logical condition of the backup media, whether it's tape, disk, or cloud storage, to ensure it's in good working order. | +| Conduct Incremental Backup Tests | Test the restoration of incremental backups to ensure that they work in conjunction with full backups and that the data can be restored to a specific point in time. | +| Review Backup Software Updates | Keep the backup software updated and test after updates to ensure new versions or patches do not introduce issues with the backup or restoration processes. | +| Check for Compatibility Issues | Ensure that the backup system is compatible with all the software and data formats used in your organization to avoid restoration issues. | +| Document Test Results | Keep detailed records of all backup tests, including the scope, date, issues encountered, and resolutions. This documentation is invaluable for troubleshooting and improving the backup process. | + +By implementing these methods, you can be confident that your backup system protects your data and that you can recover your data when necessary. Regular testing and monitoring are key to maintaining an effective backup strategy and ensuring business continuity. + +### Common challenges + +Common backup challenges often include issues such as damaged backups, missing or failed backups, maintaining legacy systems, adhering to strict privacy and security regulations, and managing the costs associated with data backup. To overcome these challenges, organizations can adopt several strategies. + +Implementing a modern, reliable backup solution that includes automated data consistency checking can help protect against damaged backups. Regular recovery tests can ensure that data is recoverable and not corrupt. For missing or failed backups, it's essential to have a robust monitoring system that alerts IT staff to any issues with the backup process, allowing for immediate action to rectify problems before they escalate. + +Maintaining legacy systems can be addressed by gradually phasing out older technology and replacing it with more current, supported solutions. Adhering to privacy and security regulations requires a thorough understanding of the laws and standards that apply to an organization's industry and geography. Implementing encryption and access controls can help ensure that backups are secure and compliant. Finally, managing the costs of data backup involves careful planning and budgeting, including the total cost of ownership of backup solutions. + +In addition to these strategies, follow best practices such as the 3-2-1 backup rule (three copies of data on two different media, with one copy stored offsite), regular testing and validation of backups, clear documentation of backup procedures, and continuous training for staff involved in the backup process. + +## Disaster recovery plan + +A disaster recovery plan (DRP) is a structured approach with detailed instructions for responding to unplanned incidents such as natural disasters, power outages, cyber-attacks, and other disruptive events. + +| Section | Description | +|---------|-------------| +| Introduction | This section outlines the purpose of the plan, its scope, and the types of disasters it covers. It also includes the plan's objectives, such as minimizing downtime and data loss. | +| Roles and Responsibilities | This part defines the disaster recovery team's specific roles. It lists the contact information for each team member and outlines the chain of command. | +| Incident Response | This section provides the steps to detect, report, and assess damage caused by a disaster. It includes procedures for immediate response to limit the impact on operations. | +| Plan Activation | Here, the criteria for launching the disaster recovery plan are detailed. It specifies who has the authority to activate the plan and under what circumstances. | +| Communication Plan | This part details how communication should be handled during a disaster, including internal communication with employees and external communication with customers, suppliers, and the media. | +| Recovery Strategies | This section outlines the strategies for restoring hardware, software, and data that are critical to resuming business operations. It includes prioritized actions for recovery and the resources required. | +| Data Backup Procedures | Detailed instructions for restoring data from backups are provided here. It includes the locations of backups, the types of backups (full, incremental), and the backup schedule. | +| Site Relocation | If the primary business location is compromised, this part of the plan details the process for moving operations to an alternate site. | +| Plan Testing and Maintenance | This section describes the schedule for regular testing of the DRP to ensure its effectiveness. It also outlines the process for updating the plan as the business and technology environment changes. | +| Appendices | The appendices may include site maps, network diagrams, floor plans, and any other documentation that supports disaster recovery efforts. | + +A DRP should be tailored to the specific needs of the organization and should be reviewed and tested regularly to ensure it remains effective and up-to-date. diff --git a/docs/differences-logical-backup.md b/docs/differences-logical-backup.md new file mode 100644 index 00000000..4eaf720d --- /dev/null +++ b/docs/differences-logical-backup.md @@ -0,0 +1,55 @@ +# Differences between a logical backup and Percona XtraBackup + +You'll notice several key differences when transitioning from a logical database backup to Percona XtraBackup. A logical backup contains the database contents as SQL or similar logical data (tables, schemas, procedures); see the [Glossary](glossary.md#logical-backup). A physical backup copies the database’s raw data files from disk; see the [Glossary](glossary.md#physical-backup). The following table summarizes how they differ. + +| Type | Logical backup | Physical backup | +|------|----------------|-----------------| +| Definition | Contains logical data (tables, schemas, procedures). | Copies entire database directories and files. | +| Method | Exported as binary files using EXPORT/IMPORT tools. | Copies raw data files directly from the database. | +| Data contents | Includes data structure (schema, tables, procedures). | Contains complete data (structure, tables, transactions). | +| Instance Independence | Can be restored on any other machine or environment. | Tied to the specific database instance and environment. | +| OS-level restoration | Not suitable for OS-level restoration. | Suitable for full disaster recovery scenarios. | +| Granularity | Granular recovery of specific components (for example, one table or schema). | Typically full restore; XtraBackup also supports [partial backup](create-partial-backup.md) and [restore individual tables](restore-individual-tables.md). | +| Performance | For full backup/restore, slower (SQL parsing and execution); partial restore of one table can be faster. | For full backup/restore, faster (direct file copying); bottleneck is often disk or network I/O. | +| Use cases | Data migration, platform-independent copies, version or engine change. | Full database restoration, disaster recovery. | +| Security | Unencrypted dumps are human-readable; security depends on encryption and access control. | Binary files are less directly readable when unencrypted; security depends on encryption and access control for both. | + +Physical backups are suitable for OS-level restoration (copying back the database files and directory structure) and full disaster recovery (recovering systems and data after a major failure). Percona XtraBackup is designed to perform hot backups (backups taken while the database is running and in use) of MySQL, Percona Server for MySQL, and other MySQL-compatible databases without interrupting database services. XtraBackup performs physical backups by copying the data files; impact on server performance is often relatively low compared to blocking backup methods, but it depends on I/O load and workload. XtraBackup includes backup compression and encryption, and a [streaming backup](take-streaming-backup.md) option for large databases or transferring backups to another server. Many logical backup tools (for example, a plain `mysqldump` run) do not build in compression, encryption, or streaming; you can add them via external pipelines, while XtraBackup provides these in one tool. XtraBackup also integrates with [Percona Toolkit](https://docs.percona.com/percona-toolkit/) and [Percona Monitoring and Management (PMM)](https://docs.percona.com/pmm/) for monitoring and management. + +Another difference is the support for [incremental backups](create-incremental-backup.md) with XtraBackup. You can back up only the data that has changed since the last full backup, significantly reducing the time and storage requirements compared to a full logical backup. + +## Logical backups in more detail + +A logical backup contains copies of information about a database, such as tables, schemas (the structure of the database: tables, columns, indexes), and procedures (reusable SQL code stored in the database). Commonly, these backups are exported as binary or text files using export/import tools (for example, `mysqldump`). + +Logical backups focus on the data structure and SQL statements. You can back up a selected set of data or exclude specific data during the export, saving time and storage. They contain only structural information about the database, not instance-specific details (that is, details tied to one running copy of the database server). + +For full backup and full restore, logical backups are usually slower than physical backups because of SQL parsing and execution; for restoring a single table, a logical dump of that table can be faster than preparing and extracting from a physical backup. A logical backup does not preserve the file system-level security settings, user accounts, privileges, or authentication settings. During restoration, you must manually reconfigure these settings. + +You can restore a logical backup on any other machine and in different environments. Logical backups provide granular recovery (restoring only selected parts, such as individual tables or schemas) instead of the whole database. The recovery process can be more involved because it may require recreating the data structure. + +Use logical backups when restoring or moving a database copy to another environment; they are well suited to platform-independent data migration (restoring on different operating systems or hardware). When the goal is to change MySQL version, storage engine, or move to another database product, logical backup or logical export is often the only practical option—physical backups are tied to the same engine and often the same minor version. + +When unencrypted, logical backup files (for example, SQL from `mysqldump`) are human-readable, so anyone with access can read sensitive data. Both logical and physical backups need access control and encryption for strong security. + +## Physical backups and Percona XtraBackup in more detail + +Percona XtraBackup is a physical backup tool. It copies the database’s data files and log files from disk while the server is running (hot backup), then uses a prepare step to apply redo and undo so the copy is consistent. Because it works at the file level, full backup and full restore are usually faster than with logical backups, and the result is suitable for same-version, same-engine restore and for disaster recovery. + +XtraBackup supports [full](create-full-backup.md), [incremental](create-incremental-backup.md), and [compressed](create-compressed-backup.md) backups, plus [encryption](encrypt-backups.md) and [streaming](take-streaming-backup.md) to remote storage. You can also take [partial backups](create-partial-backup.md) or back up [individual partitions](create-individual-partition-backup.md), and [restore individual tables](restore-individual-tables.md) when needed. Backup impact on the server is usually moderate; it depends on I/O load and is often lower than with blocking methods such as `FLUSH TABLES WITH READ LOCK`. + +Use physical backups with XtraBackup when you need fast, non-blocking backups and restores on the same MySQL-compatible server (same or compatible version and engine). For migrating to a different version, engine, or database product, use logical backup or export instead. + +!!! admonition "See also" + + [How Percona XtraBackup works](how-xtrabackup-works.md) + + [Backup process overview](backup-overview.md) + + [Backup strategy and backup policy](backup-strategy.md) + + [Create a full backup](create-full-backup.md) + + [Restore a backup](restore-a-backup.md) + + [Glossary](glossary.md) (logical backup, physical backup) diff --git a/docs/example-full-incr.md b/docs/example-full-incr.md new file mode 100644 index 00000000..ba3934a3 --- /dev/null +++ b/docs/example-full-incr.md @@ -0,0 +1,138 @@ +# Backup example + +This example demonstrates how to set up daily full backups and hourly incremental backups using Percona XtraBackup {{vers}} with a MySQL {{vers}}-compatible database server. + +## Assumptions + +* Percona XtraBackup {{vers}} is installed on your system. +* A MySQL {{vers}}-compatible database server (for example, Percona Server for MySQL {{vers}}) is running. +* You have created dedicated directories for storing backups and the backup user can write to them: + * `/backups/mysql/full` — for daily full backups + * `/backups/mysql/incr` — for hourly incremental backups + +!!! warning "Credentials and security" + Using `--password` on the command line exposes the password in process lists (for example, `ps`). For production, use a [configuration file](configure-xtrabackup.md) or a login path (for example, `--login-path`) instead of passing the password on the command line. + +## Daily full backup script + +The following script defines the server details such as hostname, username, password, data directory, and backup directory. Replace these with your actual values. The script: + +* Defines the server credentials +* Generates a timestamp with the `date` command for the full backup filename +* Uses `xtrabackup` with the `--backup` option for a full backup +* Specifies the directory to store the backup with `--target-dir` +* Uses `--datadir` to point to the location of your MySQL data files +* Provides the credentials with `--user` and `--password` to access the database server +* Creates the target directory if it does not exist +* Runs the backup and reports success or failure based on the exit code + +The script does not verify that the backup completed successfully; consider checking the exit code of `xtrabackup` and only then echoing success or writing to logs. + +```bash +#!/bin/bash +# Set server details (replace with your actual values) +SERVER_HOST="your_server_hostname" +SERVER_USER="your_db_username" +SERVER_PASSWORD="your_db_password" +DATA_DIR="/var/lib/mysql" # Replace with your data directory path +BACKUP_DIR="/backups/mysql/full" + +# Get current date for filename +TODAY=$(date +%Y-%m-%d) +FULL_BACKUP_FILENAME="${BACKUP_DIR}/${TODAY}" + +# Ensure target directory exists +mkdir -p "${FULL_BACKUP_FILENAME}" + +# Run xtrabackup with full backup command +if xtrabackup --backup \ + --target-dir="${FULL_BACKUP_FILENAME}" \ + --datadir="${DATA_DIR}" \ + --user="${SERVER_USER}" \ + --password="${SERVER_PASSWORD}"; then + echo "Daily full backup completed at $(date)" +else + echo "Daily full backup FAILED at $(date)" >&2 + exit 1 +fi +``` + +## Running the daily script + +You can schedule this script to run automatically using cron. For example, to run it every day at midnight (00:00), add this line to your crontab: + +```bash +0 0 * * * /path/to/daily_full_backup.sh +``` + +## Hourly incremental backup script + +This script uses the same server details as the daily script. It retrieves the current date and hour for the incremental backup filename. The core command uses `xtrabackup` with the `--backup` and `--incremental` options for an incremental backup. `--target-dir` specifies the directory to store the incremental backup. `--incremental-basedir` must point to the base directory of a completed full or incremental backup. This example uses the **most recent** full backup as the base: `ls -t` lists full backups by modification time (newest first), and `head -n 1` selects that newest directory. The script checks that the base directory exists before running and reports success or failure based on the exit code. + +```bash +#!/bin/bash +# Set server details (same as daily script) +SERVER_HOST="your_server_hostname" +SERVER_USER="your_db_username" +SERVER_PASSWORD="your_db_password" +DATA_DIR="/var/lib/mysql" # Replace with your data directory path +BACKUP_DIR="/backups/mysql/incr" + +# Get current date and hour for filename +TODAY=$(date +%Y-%m-%d) +HOUR=$(date +%H) # Get current hour (00-23) +INCR_BACKUP_FILENAME="${BACKUP_DIR}/${TODAY}_${HOUR}" + +# Set the base directory to the most recent full backup (newest first: ls -t, then head -n 1) +LATEST_FULL=$(ls -t /backups/mysql/full 2>/dev/null | head -n 1) +if [ -z "${LATEST_FULL}" ]; then + echo "No full backup found; cannot take incremental backup" >&2 + exit 1 +fi +BASE_DIR="/backups/mysql/full/${LATEST_FULL}" + +# Ensure target directory exists +mkdir -p "${INCR_BACKUP_FILENAME}" + +# Run xtrabackup with incremental backup command +if xtrabackup --backup \ + --incremental \ + --target-dir="${INCR_BACKUP_FILENAME}" \ + --incremental-basedir="${BASE_DIR}" \ + --user="${SERVER_USER}" \ + --password="${SERVER_PASSWORD}"; then + echo "Hourly incremental backup completed at $(date)" +else + echo "Hourly incremental backup FAILED at $(date)" >&2 + exit 1 +fi +``` + +!!! note "Incremental base directory" + For a chain of incremental backups, you can set the base to the most recent incremental backup directory instead of the full backup, so each incremental is based on the previous one. The example above bases each hourly backup on the latest full backup for simplicity. + +!!! important "Schedule so the full backup completes before incrementals" + Each incremental must be based on a **completed** full or incremental backup. If the daily full runs at midnight and can take more than an hour, the 01:00 incremental might still see "latest full" as yesterday's backup, and the 02:00 incremental might see today's. That produces two different chains. Ensure the full backup finishes before the first incremental that depends on it: for example, run the full at 00:00 and start incrementals at 02:00, or use a wrapper that runs the first incremental only after the full has completed. + +## Running the hourly script + +Schedule this script to run every hour using cron: + +```bash +0 * * * * /path/to/hourly_incremental_backup.sh +``` + +## Restore chain + +To restore from a full backup and its incrementals you must: + +1. **Prepare the full backup** once (apply redo only, no incremental yet). +2. **Apply each incremental in order** (oldest to newest) to the same target directory. +3. **Prepare again** (apply redo and undo) so the backup is consistent. + +All incrementals that you apply must form a single chain from one full backup: each incremental is based on the previous one or on the full. Do not mix incrementals from different full backups in one restore. For step-by-step commands, see [Prepare a full backup](prepare-full-backup.md), [Prepare an incremental backup](prepare-incremental-backup.md), and [Restore full and incremental backups](restore-a-backup.md). + +## Next steps + +* [Prepare a full backup](prepare-full-backup.md) and [prepare an incremental backup](prepare-incremental-backup.md) before restore. +* [Restore full and incremental backups](restore-a-backup.md) when you need to recover data. diff --git a/mkdocs-base.yml b/mkdocs-base.yml index 67df67c0..22338f53 100644 --- a/mkdocs-base.yml +++ b/mkdocs-base.yml @@ -185,8 +185,11 @@ nav: - about-xtrabackup.md - features.md - how-xtrabackup-works.md + - differences-logical-backup.md + - backup-overview.md + - backup-strategy.md + - example-full-incr.md - generated-files.md - - Features: - Backup features: