3. Serving with MongoDB
MongoDB is an object based database system. Python-ezdb provides a nice higher level interface “Mongo” by using PyMongo and os commands to make managing MongoDB more streamlined and less reliant on direct connection management to MongoDB.
3.1. Creating a basic database
Disambiguation: we define a basic database as a standalone MongoDB instance with one universal administrator and one read/write user with password authentication.
While it is possible it is highly discouraged to use Nemesyst to create the users you require as this is quite complicated to manage and may lead to more problems than its worth compared to simply creating a database and adding a user manually using something like the following:
3.1.1. Manual creation of MongoDB
- Files-only/ development creation of database example:
mongod --config ./examples/configs/basic_mongo_config.yaml
This will create a database with all the MongoDB defaults as it is an empty yaml file.
If you would instead want a more complex setup please take a look at examples/configs/authenticated_replicaset.yaml
instead, but you will need to generate certificates and keys for this so it is probably a poor place to start but will be what you will want to use in production as a bare minimum security.
3.1.2. Docker-Compose creation of MongoDB
- Docker-Compose, Files-only/ development creation of database example:
docker-compose up
This similar to the Manual creation of MongoDB creation uses a simple config file to launch the database. This can be changed in docker-compose.yaml
.
At this point you will need to connect to the running MongoDB instance (see: Connecting to a running database) to create your main administrator user, with “userAdminAnyDatabase” role.
After this you can use the following to close the Docker container with the database:
- Docker-Compose, Files-only/ development, closing Docker-Compose database example:
docker-compose down
Note
Don’t worry we set our docker-compose.yaml to save its files in /containers/mongodb
so they are persistent between runs of docker-compose. If you need to delete the MongoDB database that is where you can find them.
3.1.3. Connecting to a running database
To be able to fine tune, create users, update etc it will be necessary to connect to MongoDB in one form or another. Nemesyst can help you log in or you can do it manually.
Note
If there is no userAdmin or userAdminAnyDatabase then unless expressly configured there will be a localhost exception which will allow you to log in and create this user. If this user exists the localhost exception will close. Please ensure you configure this user as they can grant any role or rights to anyone and would be a major security concern along with making it very difficult to admin your database.
3.1.3.1. Mongo
To connect to an non-sharded database with autnentication but no TLS/SSL:
- Bash shell example:
mongo
hostname
:port
-uusername
--authenticationDatabasedatabase name
To connect to a slightly more complicated scenario with authentication, TLS, and sharding enabled:
- Bash shell example:
mongo
hostname
:port
-uusername
--authenticationDatabasedatabase name
--tls --tlsCAFilepath to ca file
--tlsCertificateKeyFilepath to cert key file
3.1.4. Creating database users
You will absolutely need a user with at least “userAdminAnyDatabase” role. Connect to the running database see Connecting to a running database.
- Mongo shell create a new role-less user:
db.createUser({user: "
username
", pwd: passwordPrompt(), roles: []})- Mongo shell grant role to existing user example:
db.grantRolesToUser( "
username
", [ { role: "userAdminAnyDatabase
", db: "admin
" } ])- Mongo shell create user and grant
userAdminAnyDatabase
in one: db.createUser({user: "
username
", pwd: passwordPrompt(), roles: [{role:"userAdminAnyDatabase
", db: "admin
"}]})
Note
Since this user belongs to admin
in the previous examples that means the authenticationDatabase is admin
when authenticating as this user as per the instructions in “Connecting to a running database”.
3.2. From basic database to replica sets
This section will outline how to take a currently standard database and turn it into a replica set
3.2.1. MongoDB config file setup for replica sets
- Files-only/ development example
./examples/mongod.d/replica.yaml
: security: keyFile: mongo.key authorization: enabled replication: replSetName: rs0 processManagement: fork: true net: bindIp: 0.0.0.0 port: 65535 systemLog: path: mongolog.log destination: file storage: dbPath: /data/db directoryPerDB: true
3.2.2. Checking the current status of the replica sets
The replica sets should not be initialized which we can check.
- Mongo shell Check the current status of replica sets:
Command:
rs.status()
Out:
{ "operationTime" : Timestamp(0, 0), "ok" : 0, "errmsg" : "no replset config has been received", "code" : 94, "codeName" : "NotYetInitialized", "$clusterTime" : { "clusterTime" : Timestamp(0, 0), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } } }
There should be no config present also, which we can also check.
- Mongo shell Check the current status of replica set config:
Command:
rs.conf()
Out:
2020-03-12T13:43:46.998+0000 E QUERY [js] uncaught exception: Error: Could not retrieve replica set config: { "operationTime" : Timestamp(0, 0), "ok" : 0, "errmsg" : "no replset config has been received", "code" : 94, "codeName" : "NotYetInitialized", "$clusterTime" : { "clusterTime" : Timestamp(0, 0), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } } } : rs.conf@src/mongo/shell/utils.js:1531:11 @(shell):1:1
If the config does not yet exist like above, or is not initialized we should initialize it.
3.2.3. Initializing and populating the replica set config
- Mongo shell Initialize the config:
Command:
rs.initiate()
Now the rs.conf should exist so we are free to add members to the replica set.
- Mongo shell Add a member to the config:
Command:
rs.add({host: "
hostname
:port
"})
3.3. From plaintext database to TLS/SSL
First it is necessary to generate a key and a certificate file for our use. For now these can be self signed but in future you may want to look at getting them signed by a certificate authority such as LetsEncrypt.
3.3.2. Using our certificate and key as the server
Almost all of the required changes take place in the mongodb config file/ how you call mongod itself.
- Files-only/ development
mongod.conf
/mongod.yaml
example: net: bindIp:
127.0.0.1
port:27017
tls: mode: requireTLS certificateKeyFile:ckfile.pem
# this should be a path to this file certificateKeyFilePassword:password
allowConnectionsWithoutCertificates: true
An example TLS/SSL enabled replica set database config file can be seen below. This however requires a few additional files for authenticating the databases and certificates for SSL/TLS that you will need to generate.
- Files-only/ development example
./examples/mongod.d/authenticated_replicaset.yaml
: security: keyFile: mongo.key authorization: enabled replication: replSetName: rs0 processManagement: fork: true net: bindIp: 0.0.0.0 port: 65535 # the highest number port possible to use tls: mode: requireTLS certificateKeyFile: ckfile.pem # path to ckfile.pem certificateKeyFilePassword: passwordIfItHasOneAtAll # password for ckfile allowConnectionsWithoutCertificates: true systemLog: path: mongolog.log destination: file storage: dbPath: /data/db directoryPerDB: true
3.3.3. Using our certificate and key as the client
Self signed certificates are just as valid, and as good as any other certificate, with one exception; only machines we can install our certificate on will trust us, unless we disable this layer of trust entirely. Thus if our certificate is self signed then the certificate file in our case signed_certificate
must be installed on each machine that we desire to trust our MongoDB instance.
3.4. Troubleshooting
Please see MongoDB/ Serving Issues
3.5. Further reading
MongoDB core:
Replica sets: