« The Book is Done! | Main | Caching Doesn't Work »
June 13, 2008
When requirements can lead you astray
posted by ari
First an aside:
I was reading European Car (June 2008) magazine this morning. Okay. Okay. I am an über geek. I don't just geek on software and infrastructure but on all things engineering. I admit it.
Anyways, the contributing editor's letter was all about top-down vs. bottom-up design. He thought till last week that auto-driving highways is a near impossibility for now because we need perfectly precise GPS, huge compute horsepower to keep track of all the cars everywhere in real time, etc. He then drove a BMW that was using radar and cameras to follow the car in front (this car is a production car available at your local BMW dealer...nothing secret and experimental). He realized immediately that the cars can work in a mesh of self-piloting little brains all working with a few inputs and internally handling special cases like emergency stop, etc. all on their own.
I found it relevant to software design from a requirements gathering perspective. If the requirement from the business is stated as "make all the cars drive on their own from Los Angeles to Las Vegas and make sure no one gets killed," how are we as engineers supposed to know that we should use a centralized or decentralized brain for the implementation? The editor called this a top-down vs. bottom-up sort of thing but he was wrong. Its not about agile, top-down, requirements first, or anything else. Its about good engineering. One engineer can take the requirement and come up with a centralized approach while another can do exactly the opposite.
The point? Yesterday I got a requirement from a customer to store 1TB of data churning every 3 hours and then enable users to report on it, generating ad hoc queries. So, I dug around a bit and figured out that (a) the reports are canned and (b) the reports are all about statistical analysis of the data (mins, maxes, averages, what have you). So we flipped the requirements to packing the data into report-ready form sort of like cubing in an OLAP system. We then did away with the raw data saying to the users that if you install a new reporting dimension into the system, it will be ready to view 3 hours later after it has been populated.
Exact same problem as the car-self-driving thing. User asked, "how do I show I retain a window of 3 hours of data for users to report against." The answer turns out to be "generate the reports in stream as the data is flowing by. Then forget about the raw data."
The punchline: the user was struggling with a distributed in-memory SQL engine on top of Terracotta when none was required.
Don't jump to quickly to design and implementation. IMHO, it is important to keep asking questions long after you think you understand the use case. Poke at all the requirements. And if you want a more formal framework, I would say, if you know what is hard and easy based on lots of QA and stress testing and performance analysis, use that info to poke and prod on the requirements. Cast a business problem toward your strengths, not your weaknesses, and never assume that the business cannot relax a rule or two.
--Ari
Trackback Pings
TrackBack URL for this entry:
http://blog.terracottatech.com/cgi-bin/mt/mt-tb.cgi/61