is Ben Rudgers

Remarks: Epigram 5

This is part of a writing exercise around Alan Perlis‘s Epigrams in Programming.

If a program manipulates a large amount of data, it does so in a small number of ways.

Contemporary Example: Youtube.

Youtube involves huge amounts of data, yet it’s publicly observable behaviors have evolved slowly over the past few years. Upload, play, pause, rewind, search, leave a comment, make a playlist, that’s pretty much always been the extent to which users can manipulate the data.

What Google does behind the scenes is undoubtedly more sophisticated, but still the ways in which it has manipulated the dataset seem limited relative to its scale at this time. Which points out that what is large today is larger than what was large yesterday – it would probably be disorienting to treat the large data sets when Perlis wrote the epigram as large today.

The world wide web illustrates the effects of scale on data manipulation. At it’s massive scale, there are only a few simple manipulations – Get, Post etc. – while at the small scale of the DOM, there are all sorts of standard manipulations one can do.

Perhaps the small number of manipulations goes hand in hand with large sets of data because the only way for a human to deal with large amounts of data is via abstractions. Our purpose in examining data is to find patterns (or their absence)  and a pattern is an abstraction.