3. Serving with MongoDB

MongoDB is an object based database system. Python-ezdb provides a nice higher level interface “Mongo” by using PyMongo and os commands to make managing MongoDB more streamlined and less reliant on direct connection management to MongoDB.

3.1. Creating a basic database

Disambiguation: we define a basic database as a standalone MongoDB instance with one universal administrator and one read/write user with password authentication.

While it is possible it is highly discouraged to use Nemesyst to create the users you require as this is quite complicated to manage and may lead to more problems than its worth compared to simply creating a database and adding a user manually using something like the following:

3.1.1. Manual creation of MongoDB

Files-only/ development creation of database example:
mongod --config ./examples/configs/basic_mongo_config.yaml

This will create a database with all the MongoDB defaults as it is an empty yaml file. If you would instead want a more complex setup please take a look at examples/configs/authenticated_replicaset.yaml instead, but you will need to generate certificates and keys for this so it is probably a poor place to start but will be what you will want to use in production as a bare minimum security.

3.1.2. Docker-Compose creation of MongoDB

Docker-Compose, Files-only/ development creation of database example:
docker-compose up

This similar to the Manual creation of MongoDB creation uses a simple config file to launch the database. This can be changed in docker-compose.yaml. At this point you will need to connect to the running MongoDB instance (see: Connecting to a running database) to create your main administrator user, with “userAdminAnyDatabase” role. After this you can use the following to close the Docker container with the database:

Docker-Compose, Files-only/ development, closing Docker-Compose database example:
docker-compose down

Note

Don’t worry we set our docker-compose.yaml to save its files in /containers/mongodb so they are persistent between runs of docker-compose. If you need to delete the MongoDB database that is where you can find them.

3.1.3. Connecting to a running database

To be able to fine tune, create users, update etc it will be necessary to connect to MongoDB in one form or another. Nemesyst can help you log in or you can do it manually.

Note

If there is no userAdmin or userAdminAnyDatabase then unless expressly configured there will be a localhost exception which will allow you to log in and create this user. If this user exists the localhost exception will close. Please ensure you configure this user as they can grant any role or rights to anyone and would be a major security concern along with making it very difficult to admin your database.

3.1.3.1. Mongo

To connect to an non-sharded database with autnentication but no TLS/SSL:

Bash shell example:
mongo hostname:port -u username --authenticationDatabase database name

To connect to a slightly more complicated scenario with authentication, TLS, and sharding enabled:

Bash shell example:
mongo hostname:port -u username --authenticationDatabase database name --tls --tlsCAFile path to ca file --tlsCertificateKeyFile path to cert key file

3.1.4. Creating database users

You will absolutely need a user with at least “userAdminAnyDatabase” role. Connect to the running database see Connecting to a running database.

Mongo shell create a new role-less user:
db.createUser({user: "username", pwd: passwordPrompt(), roles: []})
Mongo shell grant role to existing user example:
db.grantRolesToUser(
"username",
[
  { role: "userAdminAnyDatabase", db: "admin" }
])
Mongo shell create user and grant userAdminAnyDatabase in one:
db.createUser({user: "username", pwd: passwordPrompt(), roles: [{role:"userAdminAnyDatabase", db: "admin"}]})

Note

Since this user belongs to admin in the previous examples that means the authenticationDatabase is admin when authenticating as this user as per the instructions in “Connecting to a running database”.

3.2. From basic database to replica sets

This section will outline how to take a currently standard database and turn it into a replica set

3.2.1. MongoDB config file setup for replica sets

Files-only/ development example ./examples/mongod.d/replica.yaml:
security:
  keyFile: mongo.key
  authorization: enabled
replication:
  replSetName: rs0
processManagement:
  fork: true
net:
  bindIp: 0.0.0.0
  port: 65535
systemLog:
  path: mongolog.log
  destination: file
storage:
  dbPath: /data/db
  directoryPerDB: true

3.2.2. Checking the current status of the replica sets

The replica sets should not be initialized which we can check.

Mongo shell Check the current status of replica sets:

Command:

rs.status()

Out:

{
    "operationTime" : Timestamp(0, 0),
    "ok" : 0,
    "errmsg" : "no replset config has been received",
    "code" : 94,
    "codeName" : "NotYetInitialized",
    "$clusterTime" : {
            "clusterTime" : Timestamp(0, 0),
            "signature" : {
                    "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                    "keyId" : NumberLong(0)
            }
    }
}

There should be no config present also, which we can also check.

Mongo shell Check the current status of replica set config:

Command:

rs.conf()

Out:

2020-03-12T13:43:46.998+0000 E  QUERY    [js] uncaught exception: Error: Could not retrieve replica set config: {
    "operationTime" : Timestamp(0, 0),
    "ok" : 0,
    "errmsg" : "no replset config has been received",
    "code" : 94,
    "codeName" : "NotYetInitialized",
    "$clusterTime" : {
            "clusterTime" : Timestamp(0, 0),
            "signature" : {
                    "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                    "keyId" : NumberLong(0)
            }
    }
} :
rs.conf@src/mongo/shell/utils.js:1531:11
@(shell):1:1

If the config does not yet exist like above, or is not initialized we should initialize it.

3.2.3. Initializing and populating the replica set config

Mongo shell Initialize the config:

Command:

rs.initiate()

Now the rs.conf should exist so we are free to add members to the replica set.

Mongo shell Add a member to the config:

Command:

rs.add({host: "hostname:port"})

3.3. From plaintext database to TLS/SSL

First it is necessary to generate a key and a certificate file for our use. For now these can be self signed but in future you may want to look at getting them signed by a certificate authority such as LetsEncrypt.

3.3.1. Generating a certificate authority key, and then a self signed certificate

This example shows generating an encrypted RSA key. If you would instead prefer it to be plaintext remove `-aes-256-cbc`.

Bash shell generate encrypted RSA certificate authority private key example:
openssl genpkey -algorithm RSA -aes-256-cbc -pkeyopt rsa_keygen_bits:4096 -out ssl_key
Bash shell generate x509 certificate file valid for 365 days example:
openssl req -key ssl_key -x509 -new -days 365 -out signed_certificate

Note

It should be noted that MongoDB does hostname validation using this certificate file. The things we are aware of are the hostname must match, and in the case of replicas one thing like organization name must match between the communicating replicas if they use SSL/TLS. It should also be noted that Pymongo unlike mongo does not interpret between hostname and ip address the same way, an example can be found in MongoDB/ Serving Issues.

This should now leave you with two files, an ssl_key and a signed_certificate. We can now combine these two together to create a .pem file with both to provide to MongoDB. This new file will is the certificate-key file.

Bash shell a ckfile.pem file example:
cat signed_certificate > ckfile.pem
cat ssl_key >> ckfile.pem

3.3.2. Using our certificate and key as the server

Almost all of the required changes take place in the mongodb config file/ how you call mongod itself.

Files-only/ development mongod.conf/ mongod.yaml example:
net:
  bindIp: 127.0.0.1
  port: 27017
  tls:
    mode: requireTLS
    certificateKeyFile: ckfile.pem # this should be a path to this file
    certificateKeyFilePassword: password
    allowConnectionsWithoutCertificates: true

An example TLS/SSL enabled replica set database config file can be seen below. This however requires a few additional files for authenticating the databases and certificates for SSL/TLS that you will need to generate.

Files-only/ development example ./examples/mongod.d/authenticated_replicaset.yaml:
security:
  keyFile: mongo.key
  authorization: enabled
replication:
  replSetName: rs0
processManagement:
  fork: true
net:
  bindIp: 0.0.0.0
  port: 65535 # the highest number port possible to use
  tls:
    mode: requireTLS
    certificateKeyFile: ckfile.pem # path to ckfile.pem
    certificateKeyFilePassword: passwordIfItHasOneAtAll # password for ckfile
    allowConnectionsWithoutCertificates: true
systemLog:
  path: mongolog.log
  destination: file
storage:
  dbPath: /data/db
  directoryPerDB: true

3.3.3. Using our certificate and key as the client

Self signed certificates are just as valid, and as good as any other certificate, with one exception; only machines we can install our certificate on will trust us, unless we disable this layer of trust entirely. Thus if our certificate is self signed then the certificate file in our case signed_certificate must be installed on each machine that we desire to trust our MongoDB instance.

3.4. Troubleshooting

Please see MongoDB/ Serving Issues

3.5. Further reading

MongoDB core:

Replica sets:

TLS/SSL: