Agility has become the buzz word around the enterprise. whether it is agility around storage, networking, cloud operations, or most any other IT service is not really the point here, it all comes down to agility as an ideology.
Take for example the burgeoning data analytics market, which is driven by big data and business intelligence, where implementing agile ideologies could be the secret to success. After all, an agile business needs to be able to react to trends and discoveries to remain competitive, and waiting on analytics does not bode well for those looking to make intelligent decisions as quickly as possible.
In other words, best of breed analytics solutions must bridge the gap between data science and production to unify development and deployment into an agile methodology. With that in mind, Florian Douetteau, CEO of Dataiku, has put together an interesting guidebook that discusses how to achieve that level of synergy to build a data project that embodies the ideologies of agility.
Douetteau has identified the key strategies that illustrate how to bring agility to a data science project, those strategies include adopting:
– Consistent Packaging and Release
– Continuous Retraining of Models
– Multivariate Optimization
– Functional Monitoring
– Roll-Back Strategy
– IT Environment Consistency
– Failover Strategies
– Auditability and Version Control
– Performance and Scalability
Ultimately, the goal here is to bring agility to the data team, where a data science team and IT production can work hand in hand to deliver results in an agile fashion.
In an Interview with GigaOM, Douetteau offered additional advice, he said “One of the most valuable tips I can offer is that IT should provide a common platform, which gives users across the different groups access to the tools and technologies they are familiar with. Ideally, visual drag and drop tools for should be provided for less technical team members, while the ability to code, should be provided for advanced members. What’s more, monitoring, security options and role based administration tools should be made available to those responsible of deployments.”
Nonetheless, previous attempts to achieve the goal of agile decision making has been an almost impossible task, thanks to the silos surrounding data science development and the deployment of operational applications that can illustrate results.
Douetteau says “the biggest challenge of most data science projects is getting everyone on the same page in terms of business goals, technical requirements, project challenges, and responsibilities. More often than not, there is a disconnect between the worlds of development and production. Some teams may choose to re-code everything in an entirely different language while others may make changes to core elements, such as testing procedures, backup plans, and programming languages.”
It is that isolationism that prevents many data science projects from becoming an overall success, and worse yet, lead to incorrect conclusions and assumptions. Much of the blame can be placed upon the waterfall development ideologies of the past, which have hampered the adoption of agility in the area of data sciences.
Douetteau adds “preventing failures takes a manager who is willing to act as tech stack and programming language dictator, who will force the team into a fixed technology for a solution. That manager should also ensure that team members adopt a big picture approach, where they are able to help each other complete tasks outside of their comfort zone. Individual silos of knowledge will hinder a team’s effectiveness, and collaboration is the key to success.”
For enterprises to truly become agile, they must eschew those waterfall development processes and switch to agile methods across the board. However, data science projects seem to be the most opportune place to start in today’s on demand, instant results world.
Douetteau adds “Providing a platform that caters to all members of the team promotes collaboration and communication, two elements that are essential to the success of any devops/data analysis project that involve multiple departments.”
What’s more, the lessons learned on data science projects can be readily applied to other areas of IT and business operations, making agile an achievable goal, as long as you know where to start.
Douetteau says “Finding a common ground between your data team and IT department will undoubtedly ease the process of creating a data product for your organization. If all of your teams are aligned from the start of a project, each department knows their role and what technologies they are familiar with and specialize in to accomplish the task. Data scientists can build a solution and the IT department can deploy it. Once a best practices procedure is established it can be reproduced and your organization can more quickly and effectively make use of new predictive data opportunities… making your organization truly agile”
