Big data is on almost everybody’s mind at the moment, with IT departments wrestling with ever increasing volumes of both structured and unstructured data.
But despite big data’s place as a mainstream IT phenomenon, the bulk of big data projects still fail, as organisations struggle to find ways to capture, manage, make sense of and ultimately, derive value from their data and information, says Yolandi Nortje, executive of Intuate Group.
Taming big data to get the business insight you need is a daunting task all by itself; with one of the main causes of failure for big data projects being lack of knowledge.
Many of the technologies, approaches and disciplines around big data are so new that “mainstream” IT teams frequently lack the knowledge about how to actually work with the data and how to use it for analysis purposes in order to derive a real business benefit.
Hiring a data scientist is one possibility for addressing this knowledge shortfall; however, if that is not feasible for your organisations then existing project managers may have to gain new skills, refresh some others, and learn how to adapt to the needs of big data projects. Here are some foundational skills and experience you need to become an effective big data project manager.
Education and training
When reviewing the prerequisites in many big data project manager job postings, the minimum requirement is a Bachelors in computer science, with the majority of postings even specifying a Masters or PhD in the same field along with statistics, mathematics, science, and engineering.
Familiarity with algorithms, data structures, basic probability theory, artificial intelligence, database systems and linear algebra was also called for.
If further study is not feasible right now because of project timelines and budgets, then it is time to look into one of the growing number of graduate certificate programmes about big data to take your career to the next level.
For example, the US-based Institute for Operations Research and the Management Sciences (INFORMS), a 10 000-member international organisation of analytics professionals, recently launched its Certified Analytics Professional (CAP) programme, which provides the rigorous certification needed in the burgeoning big data field.
On the other hand, project managers with a background in training can play a major role in establishing big data in-house training programmes for their organisations, and there are a number of off-the-shelf packages to get you started.
Project managers will also need to manage the continuing education and professional development of their big data team to ensure the success of the employees and the overall effort. This may include representing training needs when budget time comes around and becoming an overall champion for why big data training makes good business sense.
Team experience
The big data project team may not be exactly like the traditional software development teams that you are used to leading. Big data project teams must draw on multidisciplinary talents from engineering, operations, business analysis, and even power users from the business side of the house.
The team’s core mission is to make sense of all that data; if you are at the helm of a big data project, you will have to lead a diverse group of talented professionals to make that happen, so any cross-functional team management experience will be vital.
Skills
With all the potential changes and new business offerings that come with big data, you will need marketing and sales skills to help your company to compete for the big data projects.
It is easy to become over immersed in the technical and analytics side of big data, but a project manager with customer-facing experience will be able to help with proposal efforts, sales presentations, and customer briefings – a valuable commodity for a company.
Similarly, project managers with a background and experience in advising internal and external customers about solutions will find a place in the big data world.
With big data developments and direction being so unpredictable, project managers need to monitor trends in the emerging fields of enterprise data management, data warehousing and business intelligence, and then be able to advise customers and upper management about the direction big data is heading, and how the business and project can take advantage of the big data advancements.
Big data still lends itself to a fair amount of experimentation, which is why there is a need for a project manager with at least some technical skills and aptitude. These technical skills can come from prior experience, as a big data project manager could be drawn from either the business or technology side of the company, and built upon through in-house and vendor training.
Conclusion
With the rise of technological advances in processing power and storage capability, the demand for effective and sophisticated knowledge discovery techniques has grown exponentially. Businesses need to transform large quantities of information into intelligence that can be used to make smart business decisions.
Big data offers the solution, and with it many opportunities for experienced project managers to play an instrumental role in establishing a company’s competency in big data projects to strike while the iron is hot.
The right big data project manager will also be able to nurture a team working on the cusp of these new and emerging technologies and business practices, to ensure many projects’ success down the line.