MongoDB, a NoSQL database management system, has become extremely popular. Its popularity has come with problems, especially in unsupported versions. Some of the largest organizations using MongoDB are featured in a new Datavail white paper. Datavail’s MongoDB Database Engineers and Administrators share their best practices in this blog post.
Different Flavors of MongoDB
The MongoDB community supports open-source software. In the community version, it is unsupported, or it is supported with the enterprise edition. There is also a hosted version, Atlas, which provides both storage and software.
“MongoDB is popular for content management systems (CMSs), which frequently include a wide range of content, such as images, text, tables, and advertisements. Data hubs and other big-data operations use it. MongoDB is excellent for mobile and social data management.” – Datavail white paper.
The community version is free, but you’re responsible for patching and upgrading it yourself. A support contract is recommended if you use the community version.
MongoDB enterprise requires a license per database. Support comes with license
MongoDB Atlas offers database-as-a-service for those who wish to host their databases. In Atlas, users are dependent on the software running in the cloud to perform their tasks. As a result, it’s a lot less flexible than being able to control your own configuration, which gives you more control. Nevertheless, it is an appealing option for companies with fewer employees or for those in need of a backup system.
Best Practices for MongoDB Operations
Here are some best practices for operating MongoDB that Datavail’s clients and DBAs suggest.
- Assigning Roles and Responsibilities
The roles are assigned based on the new configuration expectations. Often, end users are closely involved in configuration discussions, along with system administrators, data architects, and DBAs.
- Preparing for a MongoDB Deployment
Storage engine solutions
- It is possible to use MongoDB with a variety of storage solutions – either on-premises, in the cloud, or in a hybrid environment.
- MongoDB can be configured with a variety of pluggable storage engines. WiredTiger and MMAPv1 are included by default
- In-memory storage and encrypted storage are included in MongoDB Enterprise Advanced edition.
Schema design
A great strength of NoSQL databases is that they don’t impose schemas. It is possible to merge data in a wide variety of different formats with ease. The format in which they are merged becomes a schema and its design should be carefully considered.
It is important for database architects and schema designers to consider the type and number of documents that will be stored, how they will be grouped into collections, and how they will be indexed and validated.
MongoDB Compass allows these rules to be turned into a visual schema, making database queries easier and faster.
Data lifecycle management
By building MongoDB Zones, DBAs can create tiered storage solutions that support the data lifecycle, storing frequently-used data in memory, less-used data on the server, and archived data offline.
Managing indexes
Indexes play an important role in optimizing a database system. As well as taking up a great deal of space, they must also be removed when they are no longer required. The following are some best practices for modeling data as documents. Your application’s objectives will determine the appropriate approach.
- Store data as a single document. A single query can then be used to retrieve the entire database. While this method is efficient, there are limitations in terms of size.
- Avoid creating large documents. MongoDB supports a maximum document size of 16MB. Files larger than 16MB, like photos or videos, must be broken up into smaller files with tools like GridFS. It is not possible to replicate in memory a document if the space allocated for it exceeds half the available space. When documents become too large, you should monitor their growth and relocate them.
- Avoid long field names. Field names that are long increase the minimum amount of space that a database requires. There is a limit of 125 characters for field names. Similarly, avoid queries involving low-cardinal fields because they can result in extremely large reports.
- Use MongoDB’s new tools to manage Indexes. WiredTiger’s storage engine automatically compresses indexes. With MongoDB Compass, it is easy to identify opportunities to remove infrequently used indexes from a database.
- MongoDB Setup and Configuration
- Configuring the database. Provision MongoDB instances using tools such as Chef and Puppet. Using MongoDB’s Ops Manager and Cloud Manager, replica sets and sharded clusters can be provisioned automatically.
- Upgrading the software. If the storage engine is properly provisioned, MongoDB upgrades can be performed without causing any downtime.
- Data migration. MongoDB uses mongoimport and mongoexport to move data into and out of the database. Using mongodump and mongorestore, you can migrate from one MongoDB instance to another.
- Storage.Solid-state drives (SSDs) can be attached to MongoDB in order to provide lightning-fast performance as well as conventional hard disk storage. MongoDB does not recommend the use of RAID-5 or RAID-6. For better performance and fault tolerance, RAID-10 is recommended.
- Continuous Availability with MongoDB
In the same way that your power supply has a battery backup or a series of backups, your MongoDB software can be configured with data redundancy and system redundancy to ensure continuous availability.
- Journaling. Journal entries are recovered when data is recovered, resulting in a highly reliable database that can be quickly restored. Journals are compressed with WiredTiger and MMAPv1 to reduce storage space requirements.
- Data redundancies. In the event of a database crash, MongoDB maintains multiple replicas for instant failover. During the repair or replacement of the failed dataset, MongoDB continues to function. A replica data set enables an upgrade to occur without downtime, as well as maintenance and tuning to take place while the database is running.
- Replication. MongoDB replica data sets can be used across data centers to protect against natural and human disasters. It is possible for DBAs to specify the level of persistence they desire for data writes. Guarantees can be set at written, journaled, replicated, or even customized for replication of specific data to specific replicas.
- Scaling a MongoDB System
MongoDB offers the advantage of being able to scale your database as your needs grow. In this technique, MongoDB automatically balances data across shards, or replica data sets, so that there is always capacity and rarely a bottleneck.
There are many ways to shard a database, depending on how the data is stored and retrieved. When a database is sharded by range, it is organized according to a key value that is present in all the documents within the database. To determine the distribution of shards, hash sharding uses an MD5 of the shard key value. Sharding policies can be seen and modified using MongoDB Zones, a sharding program. There are new helpers in MongoDB 3.4 that work with Zones to provide better control over sharding.
Among the things to watch for when scaling your system are file sizes running up against the limits mentioned earlier, exceeding RAM limitations of the system, the need to write to shards can exceed disk I/O limits, and the need to specify the location of the shards.
- Managing MongoDB
MongoDB can be managed via the Ops Manager if you are using the on-premises version, the Cloud Manager if you are using the cloud version, or MongoDB Enterprise Advanced, which uses both Cloud Manager and Ops Manager. The database can be manually maintained or automated using the following procedures:
- Deploying
- Scaling
- Upgrading
- Scheduled backups
- Alert monitors
- Index rollouts
- Zone management
Monitoring. In order to monitor how actual performance differs from what is expected or required, it is important to establish performance baselines. With Ops Manager, you can monitor over 100 metrics in easy-to-customize dashboards that show you how your database is performing at a glance. Permissions can be mapped to dashboards to restrict access to data sets. When performance varies from a desired parameter, custom alerts can be configured to trigger warnings.
Backup and recovery. You should have a backup-and-recovery plan for your MongoDB systems to ensure system uptime, to prevent data loss, and to meet regulatory requirements. Using Ops Manager and Cloud Manager, your data is continuously backed up just seconds behind the master database. In the event of a problem, you can restore the system to a specific point in time. Using the mongodump tool included with MongoDB, you can perform live backups, scheduled backups, or point-in-time backups.
- Security for MongoDB
- Authentication. The Advanced Enterprise edition of MongoDB includes a challenge/response module and is compatible with LDAP, Kerberos, and Windows Active Directory.
- Authorization. Users are assigned permissions based on a highly detailed list of data they may access. It is possible that no two users will view a document in the same way, depending on the fields that they are permitted to access. The Advanced Enterprise edition of MongoDB 3.4 contains even more features for generating alerts regarding possible system compromises and more control over user access and permissions.
- Auditing.
- Creating and customizing audit trails is a simple and straightforward process. The functionality of the system includes the ability to do the following:
- Get a report of who has accessed a particular document.
- Maintain a record of each user’s changes to the database.
- Customize and store auditing tools so that frequently-run reports can be generated with just a few keystrokes.
- Encryption.
- Utilize encrypted channels for data transfer and encrypt your data on the disk and on the network. “MongoDB Operations Best Practices” states that “MongoDB supports a variety of encryption algorithms – the default is AES-256 (256 bit encryption) in CBIC mode.”. It is also possible to use AES-256 in GCM mode. It is possible to configure encryption to meet the requirements of FIPS 140-2.”
In order to keep your database secure and enable best practices, you should use MongoDB to manage your database and upgrade to the latest version. Contact Datavail today or download the white paper to learn how you might benefit from a MongoDB upgrade or managed services contract. In North America, Datavail is the largest database services provider with more than 800 database administrators. Regardless of your organization’s build, Datavail provides 24×7 managed services for applications, BI/Analytics, and databases.