Flexibility vs. Strictness in Data

There’s almost always a set of trade-offs in software development.  There are trade-offs between performance and memory consumption, trade-offs in making it fast, cheap, and good, and trade-offs between using different technologies.  I don’t often hear about trade-offs between structured vs. unstructured data.

My software development path has been interesting.  When I first started programming, I was doing graphics programming in C++ (and DirectX).  I then learned Java, VB.Net, C#, and have now found myself using Python.  Somewhere along there I picked up SQL (including a bit of both Transact SQL and PL-SQL).  More recently, I’ve dabbled in using other tools like Redis and ElasticSearch.  Along the way, it’s been interesting to see the different methodologies when it comes to storing data.

In the strict camp, database models have a set number of fields, with specific types and rigid relationships between data.  Either values are present, or they are null.  But you can always assume that there will be a certain amount of structure.  This is great, until a schema change comes along.  Schema changes are generally fine, so long as you know what the future of your data should look like.

In the flexible camp, you’ve got no guarantee that a particular key will exist, but you have the flexibility to add it if it isn’t there.  As business requirements change, so can your data – quite easily.  Need to add some new attributes to a database model?  No problem.  Not sure what the attribute names are going to be?  Also not a problem – just wing it!  This is great for rapid development, but then it comes back to haunt you when you have to continually check for the presence of something before you try to use it.

I generally like the approach of starting strict, then loosening restrictions as necessary.

If you don’t have clearly defined requirements, including clearly defined requirements of the data you need to store, that’s a pretty good indication that you shouldn’t be starting to code yet.  To solve a problem, you first have to understand the problem.  Although flexible data allows you to discover how to solve the problem, it’s often a good idea to revisit how you are storing your data once you’ve figured out the solution.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s