Beyond Databases

• Do databases have an infinite capacity?
• Lots of databases (e.g. Access) are restricted in the size (2 GB).
• Is there data bigger than that?
• What do you do if you've got data that's bigger than that?
• The key here is that the data has to fit in memory to process it.
• You've got to get it onto a hard disk and then into RAM.
• If you have to swap from hard disk to RAM too often, your system thrashes, and it slows down a lot.
• File analysis.
• Distributed databases.
• Hadoop Distributed File System