Big Data and Cloud Problems Are an Employment Problem
February 16, 2016

Brad Kolarov
B23

Regardless of the industry, we are finding these days all businesses are becoming data businesses. To keep up with competition, companies need to collect and analyze data, gain insight on that data and act on that insight.

That's easier said than done. At the moment, many companies can't keep up with their data, let alone their competition. That's not a technology problem; it's an employee skillset problem.

The proliferation of data and the pace of innovation within the open source software ecosystem are driving the convergence of Big Data, Cloud Computing, and DevOps/Automation. Proficiency across these functional areas is essential in this rapidly evolving, converging IT environment. At the moment, however, there's a lack of qualified talent with these cross-disciplinary skills.

To an extent, the emergence of DevOps itself has led to a skills imbalance across disciplines.

In the recession of 2007–2009, business cost cutting led to many technical efficiencies. This included the migration from waterfall to agile development – which created the need for DevOps, bridging software engineering, technology operations and quality assurance. DevOps, therefore, emerged as an active and growing skillset (DevOps is the convergence of software development, IT Operations and Quality Assurance). Many technologists immersed themselves in the skills related to DevOps.

In fact, though, DevOps is only a subset in the much larger convergence of technical skills needed in today's data-centric business environment.

Here's the real need:


The center of these overlapping circles is where the real action is. But it's also a real challenge.

Finding qualified candidates who can work with Big Data (Hadoop, NoSQL, Storm, Kafka, etc.) tools is very difficult. Understanding Cloud (AWS, Azure, Google, etc.) is just as hard. Those with experience with DevOps/Automation (Chef/Puppet/Ansible/Salt, etc.) is a little easier but nonetheless challenging.

And it's not enough to have a certification in certain technology skills. Certification and experience are not synonymous. Of course, certifications are a great prerequisite, and a start towards a true cross-disciplinary skillset, but certifications are not necessarily an indicator of real world experience.

■ In Big Data, what's important is having built a production Big Data system. What's not is something like Hadoop Certification (nice but not essential).

■ In Cloud Computing, what's important is being able to use SDKs and APIs, and to implement with best practices in your coding. What's not is something like AWS Certification (nice but not essential).

■ In DevOps/Automation, what's important is being able to work with full stack automation. What's not is working with software installation scripts (do we even need to explain why that's not important?)

It's quite uncommon to have a great developer come to the table with all three of these skill sets. Softer skills are often a good indicator of future capability. For example, when our company hires new technology talent, the most desirable traits among our qualified candidates are  proven aptitude and motivation to learn.

As rare as it is to find people with the ability to work in today's converging technologies, it is still vitally important. Businesses with a shortage in these skills are definitely at a technical disadvantage compared with their better-heeled competitors.

For most companies, hiring individuals with these cross-disciplinary skills is not scalable. To level the playing field, businesses should create opportunities to gain these technology benefits.

Collaboration with integrators and consultants that have invested specifically in product and service offerings across the three disciplines of Big Data, Cloud and DevOps/Automation can go a long way to building real world capability in your own organization, without having to scale up in either internal cost or human resources.

Brad Kolarov is a Co-Founder and Managing Partner at B23.

The Latest

January 17, 2019

To better align business and IT objectives, enterprise organizations should focus on the core "problems" that individual business units face today in driving out real consumer value. Until the roadblocks and inhibitors — and, ultimately, the resultant technical debt — are removed from the equation, large enterprise organizations will continue struggling to succeed ...

January 16, 2019

Technical debt is what results when legacy platforms or highly integrated and dependent systems and processes inhibit large enterprise organizations from meeting the needs of internal business stakeholders. In many cases, the core objectives that drive real, monetizable business value are not aligned to the esoteric IT goals of "automation" and "Agile development." This creates a fundamental disconnect between business and IT ...

January 14, 2019

Budget season is an important time of the year for businesses because it gives senior IT and security leaders time to reflect on what went right this year and what initiatives need to be given priority in the new year. Recent research from Threat Stack shows security budgets are expected to increase by 19 percent over the next two years, but business leaders are still facing challenges determining where to allocate this budget in the face of rapidly evolving infrastructure ...

January 10, 2019

As organizations of all sizes are embracing hybrid and multi-cloud infrastructures, they are experiencing the many benefits of a more agile, distributed and high-speed environment where new applications and services can be built and delivered in days and weeks, rather than months and years. But as the adoption of these next generation architectures continues to grow, so do the complexities of securing the cloud workloads running on them ...

January 09, 2019

DEVOPSdigest invited DevOps experts for their predictions on how DevOps and related technologies will evolve and impact business in 2019. Part 9, the final installment, covers microservices, containers and APIs ...

January 08, 2019

DEVOPSdigest invited DevOps experts for their predictions on how DevOps and related technologies will evolve and impact business in 2019. Part 8, covers microservices and containers ...

January 07, 2019

DEVOPSdigest invited DevOps experts for their predictions on how DevOps and related technologies will evolve and impact business in 2019. Part 7, covers the Cloud ...

December 20, 2018

DEVOPSdigest invited DevOps experts for their predictions on how DevOps and related technologies will evolve and impact business in 2019. Part 6, covers DevOps Analytics, including AI and Machine Learning ...

December 19, 2018

DEVOPSdigest invited DevOps experts for their predictions on how DevOps and related technologies will evolve and impact business in 2019. Part 5 is all about testing ...

December 18, 2018

DEVOPSdigest invited DevOps experts for their predictions on how DevOps and related technologies will evolve and impact business in 2019. Part 4 covers Agile, CI/CD and automation ...

Share this