Push Technology announced the launch of a new Kafka Adapter for their Diffusion Intelligent Data Mesh.
DevOps brings enormous competitive advantage to businesses – a fact that's backed up by independent research. For example, the 2018 Accelerate State of DevOps report found that top-performing elite organizations were able to deploy code 46 times more frequently than low performers, had lowered lead times from committing to deploying changes of less than one hour (compared to 1-6 months in low performers), and recovered from incidents 2,604 times faster.
However, at the same time there is an intense and growing focus on data security and privacy. Whether it's through regulations such as the GDPR or increasing consumer expectations that their personal information will be kept safe and used responsibly, it's business-critical that organizations take these concerns seriously.
This is a particular issue when it comes to database DevOps where you also have two different cultures. Database Administrators (DBAs) have traditionally focused on ensuring stability and the security of mission-critical data and information, while developers are looking to quickly create and launch new features to drive competitive advantage.
How can you balance these potentially conflicting demands, yet achieve greater speed through DevOps and still protect the data? While creating a collaborative culture is important, so is technology. Essentially, changing the development process is vital, focusing on four key areas:
DBAs and developers can both agree that standardizing processes increases speed and ensures consistency. Therefore, look to adopt industry-standard tools for coding, comparison and version control, and roll them out across the wider team so that everyone is using them. At the same time, adhering to naming and design standards along with consistent coding practices will help to streamline the development process.
Tools such as static code analysis software can help by ensuring everyone follows the same standards and known best practices when they develop. This means that whether it's an experienced DBA writing T-SQL code or a less practiced developer, you have a consistent programming structure that can be easily understood, nipping any potential problems in the bud. At the same time, merge code early and often with version control tools to ensure you have a known, shared codebase on which to work.
Particularly when it comes to database deployments, you need a repeatable, reliable and low risk process to ensure trouble-free DevOps implementations. What I call ‘continuous stuff' (continuous integration, continuous delivery and continuous deployment) plays a leading role in achieving this.
Continuous integration gives quick feedback to the development team, while continuous delivery ensures that changes are made quickly, and continuous deployment demonstrates that the organization trusts its teams to deliver DevOps.
Automation dramatically brings down time to market, but (going back to standardization), you need the right tools and processes in place if it's to be truly effective.
DevOps is a continuous process and as an organization you need to continually improve how you operate. Monitoring operations therefore has two key roles. Clearly, it highlights potentially disruptive operational and performance issues, allowing you to fix them quickly, hopefully before they develop into more serious problems. More importantly, it gives you vital insights that allow you to optimize operations moving forward.
That means you need to add continuous monitoring to your other tools, with DBAs and other operational staff regularly sharing reports with developers to improve performance. We're now living in a feedback-driven society, and development is no different. We all need to know whether we're doing a good job, and how we can get better. It could be something as small as adjusting coding habits, but there's no way that teams will progress if they don't know there's an issue.
As I said at the beginning of this article, there's growing pressure to protect data and ensure the highest levels of security. This should be good practice for every organization – and if you needed any further reason to take it seriously, look at the enormous fines you could be liable for under regulations such as the GDPR.
Yet sharing accurate, real-world data is crucial to successfully developing new code and features. That means adopting tools for data masking and anonymization that can protect confidential and personal information, while still providing data for development and testing that has the same size and shape as the real data in the production database. You also need to be able to audit where data is shared within your organization and ensure it's protected at every point.
Changing the development process to ensure DevOps flourishes requires a combination of collaboration and technology. Following these four steps is therefore crucial to starting on your DevOps journey, bringing DBAs and developers together through standardization, automation, monitoring and data protection.