MongoDB Schema Design – Many small documents or fewer large documents?
Source : Stackoverflow
An example of “one-to-few” might be the addresses for a person. This is a good use case for embedding – you’d put the addresses in an array inside of your Person object.
An example of “one-to-many” might be parts for a product in a replacement parts ordering system. Each product may have up to several hundred replacement parts, but never more than a couple thousand or so. This is a good use case for referencing – you’d put the ObjectIDs of the parts in an array in product document.
An example of “one-to-squillions” might be an event logging system that collects log messages for different machines. Any given host could generate enough messages to overflow the 16 MB document size, even if all you stored in the array was the ObjectID. This is the classic use case for “parent-referencing” – you’d have a document for the host, and then store the ObjectID of the host in the documents for the log messages.
How should I implement this schema in MongoDB?
User: username_name: string Campaign: title: string description: string link: string UserCampaign: user_id: integer camp_id: integer Click: os: text referer: text camp_id: integer user_id: integer
MongoDB relationships: embed or reference?
- Put as much in as possible
The joy of a Document database is that it eliminates lots of Joins. Your first instinct should be to place as much in a single document as you can. Because MongoDB documents have structure, and because you can efficiently query within that structure (this means that you can take the part of the document that you need, so document size shouldn’t worry you much) there is no immediate need to normalize data like you would in SQL. In particular any data that is not useful apart from its parent document should be part of the same document.
- Separate data that can be referred to from multiple places into its own collection.
This is not so much a “storage space” issue as it is a “data consistency” issue. If many records will refer to the same data it is more efficient and less error prone to update a single record and keep references to it in other places.
- Document size considerations
MongoDB imposes a 4MB (16MB with 1.8) size limit on a single document. In a world of GB of data this sounds small, but it is also 30 thousand tweets or 250 typical Stack Overflow answers or 20 flicker photos. On the other hand, this is far more information than one might want to present at one time on a typical web page. First consider what will make your queries easier. In many cases concern about document sizes will be premature optimization.
- Complex data structures:
MongoDB can store arbitrary deep nested data structures, but cannot search them efficiently. If your data forms a tree, forest or graph, you effectively need to store each node and its edges in a separate document. (Note that there are data stores specifically designed for this type of data that one should consider as well)
It has also been pointed out than it is impossible to return a subset of elements in a document. If you need to pick-and-choose a few bits of each document, it will be easier to separate them out.
- Data Consistency
MongoDB makes a trade off between efficiency and consistency. The rule is changes to a single document are always atomic, while updates to multiple documents should never be assumed to be atomic. There is also no way to “lock” a record on the server (you can build this into the client’s logic using for example a “lock” field). When you design your schema consider how you will keep your data consistent. Generally, the more that you keep in a document the better.
What is this site
I design and build stuff, and I like to share some of my thoughts about it here.
- expressjs dynamic routing – reading list
- how to cut video using nodejs and ffmpeg
- Mongodb – Reference or embed
- shitty time reset procedure (src: twitter)
- Set up a different an SSH key for git push for a give repository
- UX challenges – 7 rapid prototypinng
- 3 weird old fashion websites still alive today
- WebRTC Janus – CPU intensive
- SRS server – what it is – and install
- How to Setup OBS with NGINX on Windows for RTMP Streaming + VPS Hosted by Amazon AWS
- Content Creation
- Future Society
- Geek stuff
- Hardware Hacks
- Hustle Hacks
- Internet History
- Rethinking education
- Understanding the world
- Print your shoesDecember 12, 2018 - 4:10 am
- 5 talks to get up to speed on the JAMStackNovember 20, 2018 - 11:52 am
- MongoDB Schema designNovember 17, 2018 - 4:55 am
- Open source UI Kits & themed frontend for the webOctober 17, 2018 - 8:26 am
- Install docker on debian with AnsibleSeptember 29, 2018 - 5:59 am