diff --git a/docs/details/authentication.md b/docs/details/authentication.md index 83e7a6b3..6891ee22 100644 --- a/docs/details/authentication.md +++ b/docs/details/authentication.md @@ -17,13 +17,13 @@ etc. as the `mongo` shell or `mongodump` command does. The `pbm-agent` processes should connect to their localhost `mongod` with a standalone type of connection. - ```sh + ```{.bash data-prompt="$"} pbm-agent --mongodb-uri "mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin" ``` Alternatively: - ```sh + ```{.bash data-prompt="$"} export PBM_MONGODB_URI="mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin" pbm-agent ``` @@ -32,13 +32,13 @@ etc. as the `mongo` shell or `mongodump` command does. === "The `pbm` CLI connection string" - ```sh + ```{.bash data-prompt="$"} pbm list --mongodb-uri "mongodb://pbmuser:secretpwd@mongocsvr1:27017,mongocsvr2:27017,mongocsvr3:27017/?replicaSet=configrs&authSource=admin" ``` Alternatively: - ```sh + ```{.bash data-prompt="$"} export PBM_MONGODB_URI="mongodb://pbmuser:secretpwd@mongocsvr1:27017,mongocsvr2:27017,mongocsvr3:27017/?replicaSet=configrs&authSource=admin" $ pbm list ``` diff --git a/docs/index.md b/docs/index.md index cc0869ee..30347b6d 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,6 +1,5 @@ # Percona Backup for MongoDB Documentation - Percona Backup for MongoDB (PBM) is an open source and distributed solution for consistent backups and restore of [MongoDB sharded clusters and replica sets](details/deployments.md). There is no notable performance nor operating degradation associated with PBM. You can make backups on a running server and restore your database to a specific point in time using the PBM command line interface. To do these tasks from a user interface, [use PBM with Percona Monitoring and Management](https://docs.percona.com/percona-monitoring-and-management/get-started/backup/index.html). Read more [how PBM works :material-arrow-top-right:](intro.md). @@ -22,7 +21,7 @@ You can make backups on a running server and restore your database to a specific Ready to try out PBM? Get started quickly with the step-by-step installation instructions. -[Quickstart Guides :material-arrow-right:](#){ .md-button } +[Quickstart Guides :material-arrow-right:](installation.md){ .md-button }
diff --git a/docs/install/backup-storage.md b/docs/install/backup-storage.md new file mode 100644 index 00000000..f593c57d --- /dev/null +++ b/docs/install/backup-storage.md @@ -0,0 +1,64 @@ +# Configure remote backup storage + +The easiest way to provide remote backup storage configuration is to specify it in a YAML config file and upload this file to Percona Backup for MongoDB using `pbm` CLI. + +The storage configuration itself is out of scope of the present document. We assume that you have configured one of the supported remote backup storages. +{.power-number} + +1. Create a config file (e.g. `pbm_config.yaml`). + +2. Specify the storage information within. + + The following is the sample configuration for Amazon AWS: + + ```yaml + storage: + type: s3 + s3: + region: us-west-2 + bucket: pbm-test-bucket + prefix: data/pbm/backup + credentials: + access-key-id: + secret-access-key: + serverSideEncryption: + sseAlgorithm: aws:kms + kmsKeyID: + ``` + + This is the sample configuration for Microsoft Azure Blob storage: + + ```yaml + storage: + type: azure + azure: + account: + container: + prefix: pbm + credentials: + key: + ``` + + This is the sample configuration for filesystem storage: + + ```yaml + storage: + type: filesystem + filesystem: + path: /data/local_backups + ``` + + See more examples in [Configuration file examples](../details/storage-config-example.md). + + +3. Insert the config file + +```{.bash data-prompt="$"} +$ pbm config --file pbm_config.yaml +``` + +To learn more about Percona Backup for MongoDB configuration, see [Percona Backup for MongoDB configuration in a cluster (or non-sharded replica set)](../reference/config.md). + +## Next steps + +[Start `pbm-agent` :material-arrow-right:](start-pbm-agent.md){.md-button} \ No newline at end of file diff --git a/docs/install/cloud.md b/docs/install/cloud.md deleted file mode 100644 index c5cadffd..00000000 --- a/docs/install/cloud.md +++ /dev/null @@ -1,7 +0,0 @@ -# Deploy Percona Backup for MongoDB on Kubernetes - -Deploy and run Percona Backup for MongoDB on Kubernetes with Percona Operator for MongoDB. See [Quickstart guides](https://docs.percona.com/percona-operator-for-mongodb/index.html#quickstart-guides) for the installation instructions on your favorite Kubernetes flavor. - -## Next steps - -[Set up and configure](initial-setup.md) \ No newline at end of file diff --git a/docs/install/configure-authentication.md b/docs/install/configure-authentication.md new file mode 100644 index 00000000..2f2cc5c9 --- /dev/null +++ b/docs/install/configure-authentication.md @@ -0,0 +1,185 @@ +# Configure authentication in MongoDB + +Percona Backup for MongoDB uses the authentication and authorization subsystem of MongoDB. This means that to authenticate Percona Backup for MongoDB, you need to: + +* [Create a corresponding `pbm` user](#create-the-pbm-user) in the `admin` database +* [Set a valid MongoDB connection URI string for **pbm-agent**](#set-the-mongodb-connection-uri-for-pbm-agent) +* [Set a valid MongoDB connection URI string for `pbm` CLI](#set-the-mongodb-connection-uri-for-pbm-cli) + +## Create the `pbm` user + +:material-information: Info: Execute this step on a primary node of each replica set. In a sharded cluster, this means on every shard replica set and the config server replica set. +{.power-number} + +1. Create the role that allows any action on any resource. + + ```javascript + db.getSiblingDB("admin").createRole({ "role": "pbmAnyAction", + "privileges": [ + { "resource": { "anyResource": true }, + "actions": [ "anyAction" ] + } + ], + "roles": [] + }); + ``` + +2. Create the user and assign the role you created to it. + + ```javascript + db.getSiblingDB("admin").createUser({user: "pbmuser", + "pwd": "secretpwd", + "roles" : [ + { "db" : "admin", "role" : "readWrite", "collection": "" }, + { "db" : "admin", "role" : "backup" }, + { "db" : "admin", "role" : "clusterMonitor" }, + { "db" : "admin", "role" : "restore" }, + { "db" : "admin", "role" : "pbmAnyAction" } + ] + }); + ``` + +You can specify the `username` and `password` values and other options of the `createUser` command as you require so long as the roles shown above are granted. + + +!!! tip + + To list all the host+port lists for the shard replica sets in a cluster, run the following command: + + ```javascript + db.getSiblingDB(“config”).shards.find({}, {“host”: true, “_id”: false}) + ``` + + The replica set name at the *front* of these “host” strings will have to be placed as a “/?replicaSet=xxxx” argument in the parameters part of the connection URI (see below). + +## Set the MongoDB connection URI for `pbm-agent` + +:material-information: Info: Execute this step needs on each node where `pbm-agent` is installed. + +A **pbm-agent** process connects to its localhost `mongod` node with a standalone type of connection. + +To set the MongoDB URI connection string means to configure a service init script (`pbm-agent.service` systemd unit file) that runs a **pbm-agent**. + +The `pbm-agent.service` systemd unit file includes the environment file. You set the MongoDB URI connection string for the `PBM_MONGODB_URI` variable within the environment file for every **pbm-agent**. + +??? tip "How to find the environment file" + + The path to the environment file is specified in the `pbm-agent.service` systemd unit file. + + In Ubuntu and Debian, the pbm-agent.service systemd unit file is at the path `/lib/systemd/system/pbm-agent.service`. + + In Red Hat and CentOS, the path to this file is `/usr/lib/systemd/system/pbm-agent.service`. + + **Example of pbm-agent.service systemd unit file** + + ```init + [Unit] + Description=pbm-agent + After=time-sync.target network.target + + [Service] + EnvironmentFile=-/etc/default/pbm-agent + Type=simple + User=pbm + Group=pbm + PermissionsStartOnly=true + ExecStart=/usr/bin/pbm-agent + + [Install] + WantedBy=multi-user.target + ``` + +=== "On Debian and Ubuntu Linux" + + Edit the environment file `/etc/default/pbm-agent` and specify the MongoDB connection URI string for the `pbm` user to the local `mongod` node. + + For example, if `mongod` node listens on port 27017, the MongoDB connection URI string will be the following: + + ``` + PBM_MONGODB_URI="mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin" + ``` + +=== "On Red Hat Enterprise Linux and derivatives" + + Edit the environment file `/etc/sysconfig/pbm-agent` and specify the MongoDB connection URI string for the `pbm` user to the local `mongod` node. + + For example, if `mongod` node listens on port 27017, the MongoDB connection URI string will be the following: + + ``` + PBM_MONGODB_URI="mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin" + ``` + +### Passwords with special characters + +If the password includes special characters like `#`, `@`, `/` and so on, you must convert these characters using the [percent-encoding mechanism](https://datatracker.ietf.org/doc/html/rfc3986#section-2.1) when passing them to Percona Backup for MongoDB. For example, the password `secret#pwd` should be passed as follows in `PBM_MONGODB_URI`: + +``` +PBM_MONGODB_URI="mongodb://pbmuser:secret%23pwd@localhost:27017/?authSource=admin" +``` + +## Set the MongoDB connection URI for `pbm CLI` + +:material-information: Info: Execute this step only on a host at which you will use `pbm` CLI. + +Set the MongoDB URI connection string for `pbm` CLI in your shell. This allows you to call `pbm` commands without the `--mongodb-uri` flag. + +Use the following command: + +``` +export PBM_MONGODB_URI="mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin&replSetName=xxxx" +``` + +For more information about what connection string to specify, refer to the [pbm connection string](../details/authentication.md#mongodb-connection-strings-a-reminder-or-primer) section. + +## External authentication support in Percona Backup for MongoDB + +In addition to SCRAM, Percona Backup for MongoDB supports other [authentication methods](https://docs.percona.com/percona-server-for-mongodb/latest/authentication.html) that you use in MongoDB or Percona Server for MongoDB. + +For external authentication, you create the `pbm` user in the format used by the authentication system and set the MongoDB connection URI string to include both the authentication method and authentication source. + +For example, for [Kerberos authentication](https://docs.percona.com/percona-server-for-mongodb/6.0/authentication.html#kerberos-authentication), create the `pbm` user in the `$external` database in the format `` (e.g. [pbm@PERCONATEST.COM](mailto:pbm@PERCONATEST.COM)). + +Specify the following string for MongoDB connection URI: + +``` +PBM_MONGODB_URI="mongodb://%40@:27018/?authMechanism=GSSAPI&authSource=%24external&replSetName=xxxx" +``` + +Note that you must first obtain the ticket for the `pbm` user with the `kinit` command before you start the **pbm-agent**: + +```{.bash data-prompt="$"} +$ sudo -u {USER} kinit pbm +``` + +Note that the `{USER}` is the user that you will run the `pbm-agent` process. + +For [authentication and authorization via Native LDAP](https://docs.percona.com/percona-server-for-mongodb/6.0/authorization.html#authentication-and-authorization-with-direct-binding-to-ldap), you only create roles for LDAP groups in MongoDB as the users are stored and managed on the LDAP server. However, you still define the `$external` database as your authentication source: + +``` +PBM_MONGODB_URI="mongodb://:@:27017/?authMechanism=PLAIN&authSource=%24external&replSetName=xxxx" +``` + +When using [AWS IAM authentication](), create the `pbm` user in the `$external` database with the username that contains the ARN of the IAM user/role. + + +=== "User authentication" + + ``` + arn:aws:iam:::user/ + ``` + +=== "Role authentication" + + ``` + arn:aws:iam:::role/ + ``` + +The MongoDB connection URI string then looks like the following: + +``` +PBM_MONGODB_URI="mongodb://:@:27017/?authMechanism=MONGODB-AWS&authSource=%24external&replSetName=xxxx" +``` + +## Next steps + +[Configure backup storage :material-arrow-right:](backup-storage.md){.md-button} \ No newline at end of file diff --git a/docs/install/docker.md b/docs/install/docker.md index dd68efe8..ef20801c 100644 --- a/docs/install/docker.md +++ b/docs/install/docker.md @@ -78,4 +78,4 @@ In the same way you can run other pbm commands. Find the full list of available ## Next steps -[List backups](../usage/list-backup.md) \ No newline at end of file +[List backups :material-arrow-right:](../usage/list-backup.md){.md-button} \ No newline at end of file diff --git a/docs/install/initial-setup.md b/docs/install/initial-setup.md index e9d6259c..25e9e56f 100644 --- a/docs/install/initial-setup.md +++ b/docs/install/initial-setup.md @@ -1,309 +1,17 @@ # Initial setup -After you [install Percona Backup for MongoDB](../installation.md) on every server with the `mongod` node that is not an arbiter node, the setup steps are the following: - -1. [Configure authentication in MongoDB](#configure-authentication-in-mongodb). - -2. [Configure the remote backup storage](#configure-remote-backup-storage). - -3. [Start `pbm-agent` process](#start-the-pbm-agent-process). - The following diagram outlines the installation and setup steps: ![image](../_images/setup.png) -## Configure authentication in MongoDB - -Percona Backup for MongoDB uses the authentication and authorization subsystem of MongoDB. This means that to authenticate Percona Backup for MongoDB, you need to: - -* [Create a corresponding `pbm` user](#create-the-pbm-user) in the `admin` database -* [Set a valid MongoDB connection URI string for **pbm-agent**](#set-the-mongodb-connection-uri-for-pbm-agent) -* [Set a valid MongoDB connection URI string for `pbm` CLI](#set-the-mongodb-connection-uri-for-pbm-cli) - -### Create the `pbm` user - -!!! note "" - - This step needs to be executed on a primary node of each replica set. In a sharded cluster, this means on every shard replica set and the config server replica set. - -1. Create the role that allows any action on any resource. - - ```javascript - db.getSiblingDB("admin").createRole({ "role": "pbmAnyAction", - "privileges": [ - { "resource": { "anyResource": true }, - "actions": [ "anyAction" ] - } - ], - "roles": [] - }); - ``` - -2. Create the user and assign the role you created to it. - - ```javascript - db.getSiblingDB("admin").createUser({user: "pbmuser", - "pwd": "secretpwd", - "roles" : [ - { "db" : "admin", "role" : "readWrite", "collection": "" }, - { "db" : "admin", "role" : "backup" }, - { "db" : "admin", "role" : "clusterMonitor" }, - { "db" : "admin", "role" : "restore" }, - { "db" : "admin", "role" : "pbmAnyAction" } - ] - }); - ``` - -You can specify the `username` and `password` values and other options of the `createUser` command as you require so long as the roles shown above are granted. - - -!!! tip - - To list all the host+port lists for the shard replica sets in a cluster, run the following command: - - ```javascript - db.getSiblingDB(“config”).shards.find({}, {“host”: true, “_id”: false}) - ``` - - The replica set name at the *front* of these “host” strings will have to be placed as a “/?replicaSet=xxxx” argument in the parameters part of the connection URI (see below). - -### Set the MongoDB connection URI for `pbm-agent` - -!!! note "" - - This step needs to be executed on each node where `pbm-agent` is installed. - - -A **pbm-agent** process connects to its localhost `mongod` node with a standalone type of connection. - -To set the MongoDB URI connection string means to configure a service init script (`pbm-agent.service` systemd unit file) that runs a **pbm-agent**. - -The `pbm-agent.service` systemd unit file includes the environment file. You set the MongoDB URI connection string for the `PBM_MONGODB_URI` variable within the environment file for every **pbm-agent**. - -??? tip "How to find the environment file" - - The path to the environment file is specified in the `pbm-agent.service` systemd unit file. - - In Ubuntu and Debian, the pbm-agent.service systemd unit file is at the path `/lib/systemd/system/pbm-agent.service`. - - In Red Hat and CentOS, the path to this file is `/usr/lib/systemd/system/pbm-agent.service`. - - **Example of pbm-agent.service systemd unit file** - - ```init - [Unit] - Description=pbm-agent - After=time-sync.target network.target - - [Service] - EnvironmentFile=-/etc/default/pbm-agent - Type=simple - User=pbm - Group=pbm - PermissionsStartOnly=true - ExecStart=/usr/bin/pbm-agent - - [Install] - WantedBy=multi-user.target - ``` - -=== "On Debian and Ubuntu Linux" - - Edit the environment file `/etc/default/pbm-agent` and specify the MongoDB connection URI string for the `pbm` user to the local `mongod` node. - - For example, if `mongod` node listens on port 27017, the MongoDB connection URI string will be the following: - - ``` - PBM_MONGODB_URI="mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin" - ``` - -=== "On Red Hat Enterprise Linux and derivatives" - - Edit the environment file `/etc/sysconfig/pbm-agent` and specify the MongoDB connection URI string for the `pbm` user to the local `mongod` node. - - For example, if `mongod` node listens on port 27017, the MongoDB connection URI string will be the following: - - ``` - PBM_MONGODB_URI="mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin" - ``` - - -#### Passwords with special characters - -If the password includes special characters like `#`, `@`, `/` and so on, you must convert these characters using the [percent-encoding mechanism](https://datatracker.ietf.org/doc/html/rfc3986#section-2.1) when passing them to Percona Backup for MongoDB. For example, the password `secret#pwd` should be passed as follows in `PBM_MONGODB_URI`: - -``` -PBM_MONGODB_URI="mongodb://pbmuser:secret%23pwd@localhost:27017/?authSource=admin" -``` - -### Set the MongoDB connection URI for `pbm CLI` - -!!! note "" - - This step needs to be executed only on a host that you will use `pbm` CLI at. - -Set the MongoDB URI connection string for `pbm` CLI in your shell. This allows you to call `pbm` commands without the `--mongodb-uri` flag. - -Use the following command: - -``` -export PBM_MONGODB_URI="mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin&replSetName=xxxx" -``` - -For more information about what connection string to specify, refer to the [pbm connection string](../details/authentication.md#mongodb-connection-strings-a-reminder-or-primer) section. - -### External authentication support in Percona Backup for MongoDB - -In addition to SCRAM, Percona Backup for MongoDB supports other [authentication methods](https://docs.percona.com/percona-server-for-mongodb/6.0/authentication.html) that you use in MongoDB or Percona Server for MongoDB. - -For external authentication, you create the `pbm` user in the format used by the authentication system and set the MongoDB connection URI string to include both the authentication method and authentication source. - -For example, for [Kerberos authentication](https://docs.percona.com/percona-server-for-mongodb/6.0/authentication.html#kerberos-authentication), create the `pbm` user in the `$external` database in the format `` (e.g. [pbm@PERCONATEST.COM](mailto:pbm@PERCONATEST.COM)). - -Specify the following string for MongoDB connection URI: - -``` -PBM_MONGODB_URI="mongodb://%40@:27018/?authMechanism=GSSAPI&authSource=%24external&replSetName=xxxx" -``` - -Note that you must first obtain the ticket for the `pbm` user with the `kinit` command before you start the **pbm-agent**: - -```{.bash data-prompt="$"} -$ sudo -u {USER} kinit pbm -``` - -Note that the `{USER}` is the user that you will run the `pbm-agent` process. - -For [authentication and authorization via Native LDAP](https://docs.percona.com/percona-server-for-mongodb/6.0/authorization.html#authentication-and-authorization-with-direct-binding-to-ldap), you only create roles for LDAP groups in MongoDB as the users are stored and managed on the LDAP server. However, you still define the `$external` database as your authentication source: - -``` -PBM_MONGODB_URI="mongodb://:@:27017/?authMechanism=PLAIN&authSource=%24external&replSetName=xxxx" -``` - -When using [AWS IAM authentication](), create the `pbm` user in the `$external` database with the username that contains the ARN of the IAM user/role. - - -=== "User authentication" - - ``` - arn:aws:iam:::user/ - ``` - -=== "Role authentication" - - ``` - arn:aws:iam:::role/ - ``` - -The MongoDB connection URI string then looks like the following: - -``` -PBM_MONGODB_URI="mongodb://:@:27017/?authMechanism=MONGODB-AWS&authSource=%24external&replSetName=xxxx" -``` - -## Configure remote backup storage - -The easiest way to provide remote backup storage configuration is to specify it in a YAML config file and upload this file to Percona Backup for MongoDB using `pbm` CLI. - -The storage configuration itself is out of scope of the present document. We assume that you have configured one of the supported remote backup storages. - - -1. Create a config file (e.g. `pbm_config.yaml`). - -2. Specify the storage information within. - - The following is the sample configuration for Amazon AWS: - - ```yaml - storage: - type: s3 - s3: - region: us-west-2 - bucket: pbm-test-bucket - prefix: data/pbm/backup - credentials: - access-key-id: - secret-access-key: - serverSideEncryption: - sseAlgorithm: aws:kms - kmsKeyID: - ``` - - This is the sample configuration for Microsoft Azure Blob storage: - - ```yaml - storage: - type: azure - azure: - account: - container: - prefix: pbm - credentials: - key: - ``` - - This is the sample configuration for filesystem storage: - - ```yaml - storage: - type: filesystem - filesystem: - path: /data/local_backups - ``` - - See more examples in [Configuration file examples](../details/storage-config-example.md). - - -3. Insert the config file - -```{.bash data-prompt="$"} -$ pbm config --file pbm_config.yaml -``` - -To learn more about Percona Backup for MongoDB configuration, see [Percona Backup for MongoDB configuration in a cluster (or non-sharded replica set)](../reference/config.md). - -## Start the `pbm-agent` process - -Start `pbm-agent` on every server with the `mongod` node installed. It is best to use the packaged service scripts to run `pbm-agent`. - -```{.bash data-prompt="$"} -$ sudo systemctl start pbm-agent -$ sudo systemctl status pbm-agent -``` - -For example, imagine that you put configsvr nodes (listen port `27019`) collocated on the same servers as the first shard’s `mongod` nodes (listen port `27018`, replica set name `sh1rs`). In this server there should be two `pbm-agent` processes, one connected to the shard (e.g. `“mongodb://username:password@localhost:27018/”`) and one to the configsvr node (e.g. `“mongodb://username:password@localhost:27019/”`). - -For reference, the following is an example of starting `pbm-agent` manually. The -output is redirected to a file and the process is backgrounded. - -!!! important - - Start the `pbm-agent` as the `mongod` user. The `pbm-agent` requires write access to the MongoDB data directory to make physical restores. - -```{.bash data-prompt="$"} -$ su mongod nohup pbm-agent --mongodb-uri "mongodb://username:password@localhost:27018/" > /data/mdb_node_xyz/pbm-agent.$(hostname -s).27018.log 2>&1 & -``` - -Replace `username` and `password` with those of your `pbm` user. `/data/mdb_node_xyz/` is the path where **pbm-agent** log files will be written. Make sure you have created this directory and granted write permissions to it for the `mongod` user. - -Alternatively, you can run `pbm-agent` on a shell terminal temporarily if you want to observe and/or debug the startup from the log messages. +After you [installed Percona Backup for MongoDB](../installation.md) on every server with the `mongod` node that is not an arbiter node, complete the following setup steps: -### How to see the `pbm-agent` log +1. [Configure authentication in MongoDB](configure-authentication.md). -With the packaged `systemd` service, the log output to `stdout` is captured by -systemd’s default redirection to `systemd-journald`. You can view it with the -command below. See `man journalctl` for useful options such as `--lines`, `--follow`, etc. +2. [Configure the remote backup storage](backup-storage.md). -```{.bash data-prompt="$"} -$ journalctl -u pbm-agent.service --- Logs begin at Tue 2019-10-22 09:31:34 JST. -- -Jan 22 15:59:14 : Started pbm-agent. -Jan 22 15:59:14 pbm-agent[3579]: pbm agent is listening for the commands -... -... -``` +3. [Start `pbm-agent` process](start-pbm-agent.md). -If you started `pbm-agent` manually, see the file you redirected stdout and stderr to. +## Next steps -When a message `pbm agent is listening for the commands` is printed to the -`pbm-agent` log file, `pbm-agent` confirms that it has connected to its `mongod` node successfully. +[Configure authentication :material-arrow-right:](configure-authentication.md){.md-button} \ No newline at end of file diff --git a/docs/install/repos.md b/docs/install/repos.md index e246a601..539692e4 100644 --- a/docs/install/repos.md +++ b/docs/install/repos.md @@ -1,53 +1,46 @@ # Install from Percona repositories -Use the package manager of your operating system to install Percona Backup for MongoDB: +To install the software from Percona repositories means to subscribe to them. Percona provides the [`percona-release`](https://www.percona.com/doc/percona-repo-config/index.html) repository management tool. It automatically enables the required repository so that you can install and update both Percona Backup for MongoDB packages and required dependencies smoothly. -* `apt` - for Debian and Ubuntu Linux -* `yum` - for Red Hat Enterprise Linux and compatible Linux derivatives +--8<-- "pbm-install-nodes.md" -To install the software from Percona repositories means to subscribe to them. Percona provides the [`percona-release`](https://www.percona.com/doc/percona-repo-config/index.html) repository management tool. It automatically enables the required repository for you and enables you to install and update both Percona Backup for MongoDB packages and required dependencies smoothly. +## Procedure -!!! important +:material-alert: Warning: Run the following commands as root or via the `sudo` command. +{.power-number} - Run the following commands as root or via the `sudo` command +1. [Install `percona-release`](https://www.percona.com/doc/percona-repo-config/installing.html). If you have installed it before, [update](https://www.percona.com/doc/percona-repo-config/updating.html) it to the latest version. -#### 1. Install `percona-release` +2. Enable the repository -[Install `percona-release` tool](https://www.percona.com/doc/percona-repo-config/installing.html). If you have installed it before, [update](https://www.percona.com/doc/percona-repo-config/updating.html) it to the latest version. + ```{.bash data-prompt="$"} + $ sudo percona-release enable pbm release + ``` +3. Install Percona Backup for MongoDB packages -#### 2. Enable the repository + === "On Debian and Ubuntu" -Starting with of version 1.3.0, Percona Backup for MongoDB packages are stored in the *pbm* repository. + 1. Reload the local package database: -```{.bash data-prompt="$"} -$ sudo percona-release enable pbm release -``` + ```{.bash data-prompt="$"} + $ sudo apt update + ``` -#### 3. Install Percona Backup for MongoDB + 2. Install Percona Backup for MongoDB: -=== "On Debian and Ubuntu" + ```{.bash data-prompt="$"} + $ sudo apt install percona-backup-mongodb + ``` - 1. Reload the local package database: + === "On Red Hat Enterprise Linux and derivatives" ```{.bash data-prompt="$"} - $ sudo apt update + $ sudo yum install percona-backup-mongodb ``` - 2. Install Percona Backup for MongoDB: - - ```{.bash data-prompt="$"} - $ sudo apt install percona-backup-mongodb - ``` - -=== "On Red Hat Enterprise Linux and derivatives" - - Install Percona Backup for MongoDB: - - ```{.bash data-prompt="$"} - $ sudo yum install percona-backup-mongodb - ``` +--8<-- "install-result.md" ## Next steps -[Set up PBM](initial-setup.md) \ No newline at end of file +[Initial setup :material-arrow-right:](initial-setup.md){.md-button} \ No newline at end of file diff --git a/docs/install/source.md b/docs/install/source.md index 9ce313f1..1dae101e 100644 --- a/docs/install/source.md +++ b/docs/install/source.md @@ -2,70 +2,64 @@ ## Prerequisites -Building the project requires: +To build Percona Backup for MongoDB from source, you need the following: -* Go 1.19 or above +* Go 1.19 or above. [Install and set up Go tools :material-arrow-top-right:](https://golang.org/doc/install) * make * git * `krb5-devel` for Red Hat Enterprise Linux / CentOS or `libkrb5-dev` for Debian / Ubuntu. This package is required for Kerberos authentication in Percona Server for MongoDB. -!!! admonition "See also" - - [Install and set up Go tools](https://golang.org/doc/install) - - - ## Procedure -### 1. Clone the repository +Here's how to build Percona Backup for MongoDB: +{.power-number} -```{.bash data-prompt="$"} -$ git clone https://github.com/percona/percona-backup-mongodb -``` +1. Clone the repository -### 2. Go to the project directory and build it + ```{.bash data-prompt="$"} + $ git clone https://github.com/percona/percona-backup-mongodb + ``` -```{.bash data-prompt="$"} -$ cd percona-backup-mongodb -$ make build -``` +2. Go to the project directory and build it -After **make** completes, you can find `pbm` and `pbm-agent` binaries -in the `./bin` directory: + ```{.bash data-prompt="$"} + $ cd percona-backup-mongodb + $ make build + ``` -```{.bash data-prompt="$"} -$ cd bin -$ ./pbm version -``` +After **make** completes, you can find `pbm` and `pbm-agent` binaries +in the `./bin` directory. -To verify if Percona Backup for MongoDB has been built correctly and is ready for use, run +3. Check that Percona Backup for MongoDB has been built correctly and is ready for use. -```{.bash data-prompt="$"} -$ pbm version -``` + ```{.bash data-prompt="$"} + $ cd bin + $ ./pbm version + ``` -Output + ??? example "Output" -```{.text .no-copy} -Version: [pbm version number] -Platform: linux/amd64 -GitCommit: [commit hash] -GitBranch: main -BuildTime: [time when this version was produced in UTC format] -GoVersion: [Go version number] -``` + ```{.text .no-copy} + Version: [pbm version number] + Platform: linux/amd64 + GitCommit: [commit hash] + GitBranch: main + BuildTime: [time when this version was produced in UTC format] + GoVersion: [Go version number] + ``` -!!! tip + !!! tip - Instead of specifying the path to pbm binaries, you can add it to the PATH environment variable: + Instead of specifying the path to pbm binaries, you can add it to the PATH environment variable: - ```{.bash data-prompt="$"} - $ export PATH=/percona-backup-mongodb/bin:$PATH - ``` + ```{.bash data-prompt="$"} + $ export PATH=/percona-backup-mongodb/bin:$PATH + ``` ## Post-install steps === "On Debian and Ubuntu" + {.power-number} 1. Create the environment file: @@ -153,4 +147,4 @@ GoVersion: [Go version number] ## Next steps -[Set up PBM](initial-setup.md) \ No newline at end of file +[Initial setup :material-arrow-right:](initial-setup.md){.md-button} \ No newline at end of file diff --git a/docs/install/start-pbm-agent.md b/docs/install/start-pbm-agent.md new file mode 100644 index 00000000..faebcbdc --- /dev/null +++ b/docs/install/start-pbm-agent.md @@ -0,0 +1,49 @@ +# Start the `pbm-agent` process + +Start `pbm-agent` on every server with the `mongod` node installed. It is best to use the packaged service scripts to run `pbm-agent`. + +```{.bash data-prompt="$"} +$ sudo systemctl start pbm-agent +$ sudo systemctl status pbm-agent +``` + +For example, imagine that you put configsvr nodes (listen port `27019`) collocated on the same servers as the first shard’s `mongod` nodes (listen port `27018`, replica set name `sh1rs`). In this server there should be two `pbm-agent` processes, one connected to the shard (e.g. `“mongodb://username:password@localhost:27018/”`) and one to the configsvr node (e.g. `“mongodb://username:password@localhost:27019/”`). + +For reference, the following is an example of starting `pbm-agent` manually. The +output is redirected to a file and the process is backgrounded. + +!!! important + + Start the `pbm-agent` as the `mongod` user. The `pbm-agent` requires write access to the MongoDB data directory to make physical restores. + +```{.bash data-prompt="$"} +$ su mongod nohup pbm-agent --mongodb-uri "mongodb://username:password@localhost:27018/" > /data/mdb_node_xyz/pbm-agent.$(hostname -s).27018.log 2>&1 & +``` + +Replace `username` and `password` with those of your `pbm` user. `/data/mdb_node_xyz/` is the path where **pbm-agent** log files will be written. Make sure you have created this directory and granted write permissions to it for the `mongod` user. + +Alternatively, you can run `pbm-agent` on a shell terminal temporarily if you want to observe and/or debug the startup from the log messages. + +### How to see the `pbm-agent` log + +With the packaged `systemd` service, the log output to `stdout` is captured by +systemd’s default redirection to `systemd-journald`. You can view it with the +command below. See `man journalctl` for useful options such as `--lines`, `--follow`, etc. + +```{.bash data-prompt="$"} +$ journalctl -u pbm-agent.service +-- Logs begin at Tue 2019-10-22 09:31:34 JST. -- +Jan 22 15:59:14 : Started pbm-agent. +Jan 22 15:59:14 pbm-agent[3579]: pbm agent is listening for the commands +... +... +``` + +If you started `pbm-agent` manually, see the file you redirected stdout and stderr to. + +When a message `pbm agent is listening for the commands` is printed to the +`pbm-agent` log file, `pbm-agent` confirms that it has connected to its `mongod` node successfully. + +## Next steps + +[Make a backup :material-arrow-right:](../usage/start-backup.md){.md-button} \ No newline at end of file diff --git a/docs/install/tarball.md b/docs/install/tarball.md index 85d7ff23..abc1187b 100644 --- a/docs/install/tarball.md +++ b/docs/install/tarball.md @@ -5,33 +5,35 @@ You can download Percona Backup for MongoDB from [Percona website](https://www.p * [From binary tarballs](#install-from-binary-tarball). * Manually, from the installation packages using `dpkg` (Debian and Ubuntu) or `rpm` (Red Hat Enterprise Linux and CentOS). However, you must make sure that all dependencies are satisfied. +--8<-- "pbm-install-nodes.md" ## Install from binary tarball Find the link to the binary tarballs under the **Generic Linux** menu item on [Percona website](https://www.percona.com/downloads/percona-backup-mongodb/). +{.power-number} +1. Fetch the binary tarball. Replace the `` with the required version. -### 1. Fetch the binary tarball + ```{.bash data-prompt="$"} + $ wget https://downloads.percona.com/downloads/percona-backup-mongodb/percona-backup-mongodb-/binary/tarball/percona-backup-mongodb--x86_64.tar.gz + ``` -Replace the `` with the required version. +2. Extract the tarball - ```{.bash data-prompt="$"} - $ wget https://downloads.percona.com/downloads/percona-backup-mongodb/percona-backup-mongodb-/binary/tarball/percona-backup-mongodb--x86_64.tar.gz - ``` + ```{.bash data-prompt="$"} + $ tar -xf percona-backup-mongodb--x86_64.tar.gz + ``` +3. Export the location of the binaries to the `PATH` variable -### 2. Extract the tarball + For example, if you’ve extracted the tarball to your `home` directory, the command would be the following: - ```{.bash data-prompt="$"} - $ tar -xf percona-backup-mongodb--x86_64.tar.gz - ``` + ```{.bash data-prompt="$"} + $ export PATH=~/percona-backup-mongodb-/:$PATH + ``` -### 3. Export the location of the binaries to the `PATH` variable +--8<-- "install-result.md" -For example, if you’ve extracted the tarball to your `home` directory, the command would be the following: +## Next steps - ```{.bash data-prompt="$"} - $ export PATH=~/percona-backup-mongodb-/:$PATH - ``` - -After Percona Backup for MongoDB is successfully installed on your system, you have `pbm` and `pbm-agent` programs available. See [Initial setup](initial-setup.md) for guidelines how to set up Percona Backup for MongoDB. \ No newline at end of file +[Initial setup :material-arrow-right:](initial-setup.md){.md-button} \ No newline at end of file diff --git a/docs/installation.md b/docs/installation.md index 53812226..2a956ca9 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -1,46 +1,53 @@ -# Install Percona Backup for MongoDB +# Quickstart Guides -## Supported platforms +Percona Backup for MongoDB (PBM) is an open source and distributed solution for consistent backups and restore of [MongoDB sharded clusters and replica sets](details/deployments.md). [Read more :material-arrow-top-right:](index.md). -Percona Backup for MongoDB is available for the most 64-bit Linux distributions. Find the list of supported platforms on the [Percona Software and Platform Lifecycle](https://www.percona.com/services/policies/percona-software-platform-lifecycle#mongodb) page. +Find the list of supported platforms for Percona Backup for MongoDB on the [Percona Software and Platform Lifecycle](https://www.percona.com/services/policies/percona-software-platform-lifecycle#mongodb) page. -## What nodes to install on +You can use any of the easy-install guides but **we recommend using the package manager of your operating system** for a convenient and quick way to try the software first. -### `pbm-agent` +=== ":simple-windowsterminal: Package manager" + + Use the package manager of your operating system to install Percona Backup for MongoDB: -Install [`pbm-agent`](details/pbm-agent.md) on all servers that have `mongod` nodes in the -MongoDB cluster (or non-sharded replica set). You don't need to start it on the `arbiter` node, since it doesn’t have the data set. + * `apt` - for Debian and Ubuntu Linux + * `yum` - for Red Hat Enterprise Linux and compatible Linux derivatives -### `pbm` CLI + [Install from repositories :material-arrow-right:](install/repos.md){.md-button} -You can install [`pbm` CLI](details/cli.md) on any or all servers or desktop computers you wish to use it from. Those computers must not be network-blocked from accessing the MongoDB cluster. +=== ":simple-docker: Docker" -## Installation tutorials + Get our Docker image and spin up PBM for a quick evaluation. -Install Percona Backup for MongoDB in one of the following ways: + Check the Docker guide for step-by-step guidelines. -* [from Percona repositories](install/repos.md) using the package manager of your operating system. This is the recommended way + [Run in Docker :material-arrow-right:](install/docker.md){.md-button} -* [build from source code](install/source.md) if you want full control over the installation +=== ":simple-kubernetes: Kubernetes" -* [download tarballs from Percona website](install/tarball.md) + **Percona Operator for Kubernetes** is a controller introduced to simplify complex deployments that require meticulous and secure database expertise. -* [run Percona Backup for MongoDB in a Docker container](https://hub.docker.com/r/percona/percona-backup-mongodb) + Check the Quickstart guides how to deploy and run PBM on Kubernetes with Percona Operator for MongoDB. -* [deploy Percona Backup for MongoDB on Kubernetes](install/cloud.md) + [Deploy in Kubernetes Quickstart :material-arrow-right:](https://docs.percona.com/percona-operator-for-mongodb/quickstart.html){.md-button} -After the installation completes, you have the following tools: +=== ":octicons-file-code-16: Build from source" -| Tool | Purpose | -| --------------- | ---------------------------------------------------------| -| `pbm` | Command-line interface for controlling the backup system | -| `pbm-agent` | An agent for running backup/restore actions on a database host | -| `pbm-speed-test`| An interface for field-testing compression and backup upload speed| -| `pbm-agent-entrypoint` | An entry point application that allows starting `pbm-agent` and also restarts it in case of any faults| + Have a full control over the installation by building PBM from source code. + + Check the guide below for step-by-step instructions. + + [Build from source :material-arrow-right:](install/source.md){.md-button} + +=== ":octicons-download-16: Manual download" + + If you need to install PBM offline or prefer a specific version of it, check out the link below for a step-by-step guide and get access to the downloads directory. + + [Install from tarballs :material-arrow-right:](install/tarball.md){.md-button} ## Next steps -[Install from Percona repositories](install/repos.md) +[Initial setup :material-arrow-right:](install/initial-setup.md){.md-button} diff --git a/docs/manage/overview.md b/docs/manage/overview.md index da2944d2..cff2e8a2 100644 --- a/docs/manage/overview.md +++ b/docs/manage/overview.md @@ -3,6 +3,6 @@ Need to fine-tune PBM to effectively manage your day-to-day operations? This section is the right place to be. Find what you can do with PBM in the followng topics: * [Configure PBM via pipelines](configure-remotely.md) -* [Automate access to S3 buckets for PBM[(manage/automate-s3-access.md) -* [Upgrade PBM](manage/upgrading.md) -* [Remove PBM](manage/uninstalling.md) \ No newline at end of file +* [Automate access to S3 buckets for PBM](automate-s3-access.md) +* [Upgrade PBM](upgrading.md) +* [Remove PBM](uninstalling.md) \ No newline at end of file diff --git a/mkdocs-base.yml b/mkdocs-base.yml index 7f107cfb..3c139ecf 100644 --- a/mkdocs-base.yml +++ b/mkdocs-base.yml @@ -168,50 +168,46 @@ nav: - release-notes/2.1.0.md - release-notes/2.0.5.md - release-notes/2.0.4.md - - release-notes/2.0.3.md - - release-notes/2.0.2.md - - release-notes/2.0.1.md - - release-notes/2.0.0.md - - release-notes/1.8.1.md - - release-notes/1.8.0.md + - '2022 (versions 1.8.0 to 2.0.3)': + - release-notes/2.0.3.md + - release-notes/2.0.2.md + - release-notes/2.0.1.md + - release-notes/2.0.0.md + - release-notes/1.8.1.md + - release-notes/1.8.0.md - 'Percona Backup for MongoDB 1.7.0 (2022-04-18)': 'release-notes/1.7.0.md' - - 'Percona Backup for MongoDB 1.6.1 (2021-11-04)': 'release-notes/1.6.1.md' - - 'Percona Backup for MongoDB 1.6.0 (2021-08-16)': 'release-notes/1.6.0.md' - - 'Percona Backup for MongoDB 1.5.0 (2021-05-10)': 'release-notes/1.5.0.md' - - 'Percona Backup for MongoDB 1.4.1 (2021-01-28)': 'release-notes/1.4.1.md' - - 'Percona Backup for MongoDB 1.4.0 (2020-12-24)': 'release-notes/1.4.0.md' - - 'Percona Backup for MongoDB 1.3.4 (2020-11-19)': 'release-notes/1.3.4.md' - - 'Percona Backup for MongoDB 1.3.3 (2020-11-04)': 'release-notes/1.3.3.md' - - 'Percona Backup for MongoDB 1.3.2 (2020-10-14)': 'release-notes/1.3.2.md' - - 'Percona Backup for MongoDB 1.3.1 (2020-09-03)': 'release-notes/1.3.1.md' - - 'Percona Backup for MongoDB 1.3.0 (2020-08-26)': 'release-notes/1.3.0.md' - - 'Percona Backup for MongoDB 1.2.1 (2020-07-27)': 'release-notes/1.2.1.md' - - 'Percona Backup for MongoDB 1.2.0 (2020-05-13)': 'release-notes/1.2.0.md' - - 'Percona Backup for MongoDB 1.1.3 (2020-04-14)': 'release-notes/1.1.3.md' - - 'Percona Backup for MongoDB 1.1.1 (2020-01-31)': 'release-notes/1.1.1.md' - - 'Percona Backup for MongoDB 1.1.0 (2020-01-16)': 'release-notes/1.1.0.md' - - 'Percona Backup for MongoDB 1.0.0 (2019-09-19)': 'release-notes/1.0.0.md' - - 'Percona Backup for MongoDB 0.5.0 (2019-06-17)': 'release-notes/0.5.0.md' - - Architecture: - - PBM architecture: 'details/architecture.md' - - details/pbm-agent.md - - PBM CLI: details/cli.md - - PBM control collections: details/control-collections.md - - Remote backup storage: - - Overview: details/storage-configuration.md - - details/storage-config-example.md - - details/authentication.md - - details/deployments.md - - details/versions.md + - '2021 (versions 1.4.1 to 1.6.1)': + - 'Percona Backup for MongoDB 1.6.1 (2021-11-04)': 'release-notes/1.6.1.md' + - 'Percona Backup for MongoDB 1.6.0 (2021-08-16)': 'release-notes/1.6.0.md' + - 'Percona Backup for MongoDB 1.5.0 (2021-05-10)': 'release-notes/1.5.0.md' + - 'Percona Backup for MongoDB 1.4.1 (2021-01-28)': 'release-notes/1.4.1.md' + - '2020 (versions 1.1.0 to 1.4.0)': + - 'Percona Backup for MongoDB 1.4.0 (2020-12-24)': 'release-notes/1.4.0.md' + - 'Percona Backup for MongoDB 1.3.4 (2020-11-19)': 'release-notes/1.3.4.md' + - 'Percona Backup for MongoDB 1.3.3 (2020-11-04)': 'release-notes/1.3.3.md' + - 'Percona Backup for MongoDB 1.3.2 (2020-10-14)': 'release-notes/1.3.2.md' + - 'Percona Backup for MongoDB 1.3.1 (2020-09-03)': 'release-notes/1.3.1.md' + - 'Percona Backup for MongoDB 1.3.0 (2020-08-26)': 'release-notes/1.3.0.md' + - 'Percona Backup for MongoDB 1.2.1 (2020-07-27)': 'release-notes/1.2.1.md' + - 'Percona Backup for MongoDB 1.2.0 (2020-05-13)': 'release-notes/1.2.0.md' + - 'Percona Backup for MongoDB 1.1.3 (2020-04-14)': 'release-notes/1.1.3.md' + - 'Percona Backup for MongoDB 1.1.1 (2020-01-31)': 'release-notes/1.1.1.md' + - 'Percona Backup for MongoDB 1.1.0 (2020-01-16)': 'release-notes/1.1.0.md' + - '2019 (versions 0.5.0 to 1.0.0)': + - 'Percona Backup for MongoDB 1.0.0 (2019-09-19)': 'release-notes/1.0.0.md' + - 'Percona Backup for MongoDB 0.5.0 (2019-06-17)': 'release-notes/0.5.0.md' - Get started: + - Quickstart Guide: installation.md - Install: - - Quickstart: installation.md - - Method 1. From repositories: install/repos.md - - Method 2. Build from source: install/source.md - - Method 3. Download tarballs: install/tarball.md - - Method 4. Run in Docker: install/docker.md - - Method 5. Deploy on Kubernetes: install/cloud.md - - Set up and configure: 'install/initial-setup.md' + - From repositories: install/repos.md + - Download tarballs: install/tarball.md + - Build from source: install/source.md + - Run in Docker: install/docker.md + - Set up and configure: + - Overview: install/initial-setup.md + - install/configure-authentication.md + - install/backup-storage.md + - install/start-pbm-agent.md - Backup and restore: - Make a backup : "usage/start-backup.md" - Restore a backup: usage/restore.md @@ -224,7 +220,19 @@ nav: - usage/restore-progress.md - usage/oplog-replay.md - usage/logs.md + - Architecture: + - PBM architecture: 'details/architecture.md' + - details/pbm-agent.md + - PBM CLI: details/cli.md + - PBM control collections: details/control-collections.md + - Remote backup storage: + - Overview: details/storage-configuration.md + - details/storage-config-example.md + - details/authentication.md + - details/deployments.md + - details/versions.md - Administer PBM: + - 'Overview': manage/overview.md - 'PBM configuration via pipelines': 'manage/configure-remotely.md' - 'Automate access to S3 buckets for PBM': 'manage/automate-s3-access.md' - 'Upgrade PBM': 'manage/upgrading.md' diff --git a/snippets/install-result.md b/snippets/install-result.md new file mode 100644 index 00000000..4b531460 --- /dev/null +++ b/snippets/install-result.md @@ -0,0 +1,8 @@ +After the installation completes, you have the following tools: + +| Tool | Purpose | +| --------------- | ---------------------------------------------------------| +| `pbm` | Command-line interface for controlling the backup system | +| `pbm-agent` | An agent for running backup/restore actions on a database host | +| `pbm-speed-test`| An interface for field-testing compression and backup upload speed| +| `pbm-agent-entrypoint` | An entry point application that allows starting `pbm-agent` and also restarts it in case of any faults| diff --git a/snippets/pbm-install-nodes.md b/snippets/pbm-install-nodes.md new file mode 100644 index 00000000..1db52eca --- /dev/null +++ b/snippets/pbm-install-nodes.md @@ -0,0 +1,10 @@ +## What nodes to install on + +### `pbm-agent` + +Install [`pbm-agent`](../details/pbm-agent.md) on all servers that have `mongod` nodes in the +MongoDB cluster (or non-sharded replica set). You don't need to start it on the `arbiter` node, since it doesn’t have the data set. + +### `pbm` CLI + +You can install [`pbm` CLI](../details/cli.md) on any or all servers or desktop computers you wish to use it from. Those computers must not be network-blocked from accessing the MongoDB cluster. \ No newline at end of file