Technology improves all the time, and companies like IBM especially are making sweeping efforts to streamline and evolve existing processes. With these steady improvements always coming, the question has been raised if poorly formed SQL even matters anymore. Adding fuel to the question is IBM’s announcement of BLU Acceleration, which combines techniques to improve analytical performance and simplify administration. A blog post by Tony Baer goes into articulate depth about where analytics are headed and what it means for SQL. As the blog states, “BNSF railroad, a BLU beta customer, reported performing a 4-billion row join in 8 milliseconds.” The future is fast.
BLU touts columnar and in-memory processing, data compression, and data skipping. As this technology and others that complement it head to market, the mindset in engaging challenges will shift:
For the DBA, the multiple personalities of data platforms are changing the nature of problem-solving: instead of writing the best SQL statement, focus on defining and directing the right query, to the right data, on the right engine, at the right time. For instance, a hot new mobile device is released to the market with huge fanfare, sales initially spike before unexpectedly dropping through the floor. Such a query might fuse SQL (from the CRM analytic system) with sentiment analysis (to see what customers and prospects were saying), graph analysis (to understand who is friends with, and influences, who), and time series (to see how sentiment changed over time). The query may run across SQL, Hadoop, and possibly another specialized data store.
In essence, the way Baer sees it, it really will not be the end of the world if your SQL is not quite up to snuff when there are so many other elements working in tandem with it. Consult his full blog post for a lengthy and thoughtful reflection on where technology is headed and how to adapt with it.