Keys to Communicating with Business Stakeholders

Keys to Communicating with Business Stakeholders

For an enterprise CIO, the job description can be long. Among the many urgent and shifting priorities and responsibilities, communications may often be pretty far down on the list. However, in an age of BizOps and digital transformation, the role of communicator is one that can have an increasingly pivotal role in a CIO’s ability to lead change and deliver value. In my previous post, we examined why communicating with business leadership is such a vital mandate for the modern CIO, and some of the challenges that have impeded these efforts. In this second post, I’ll offer some key strategies for optimizing these communications.

Understand the Audience

To communicate effectively, it’s critical to know your audience and tell a story that creates empathy. To do this, a CIO can adopt an approach out of the marketing team’s playbook: persona-based marketing. In marketing, we focus on tailoring communications to specific types of prospects. In a similar way, CIOs should focus on aligning their communications with several key personas, such as board members, CEOs, CFOs, and line-of-business leaders. Based on an understanding of these key contacts, CIOs can then adapt and tailor their communications to these individuals.

Objectives and Strategies

In particular, take time to study the key audience members and their strategies, including the following areas:

  • KPIs. How does this persona track their team’s progress? What metrics does this persona’s leadership use to measure his or her success?
  • Objectives. What are their primary goals and objectives? Adopting a new business model? Responding to new competitive threats? Increasing new customer acquisition or retention? Increasing customer lifetime value?
  • Challenges. What are the top challenges they’re confronting? What issues are impeding their progress?

Communications Processes, Habits, and Preferences

To enable optimized communications, CIOs should look to the persona’s existing communications, including the following three factors:

  • Terminology. What types of terminology do they use, particularly to describe the topics they care most about?
  • Communication methods. How does communication happen with their organizations? This can be invaluable in aligning communications most seamlessly. This can help with determining whether to focus on in-person meetings with accompanying reports, formal presentations, or emails.
  • Timing. Look at the audience to guide timing, both in terms of frequency and dates. On a practical level, you want to ensure you’re communicating on a frequency they’re accustomed to, and, for example, that you’re not communicating with the CFO during a cycle close or with a chief revenue officer in the days preceding the end of the fiscal quarter.

By gaining an understanding of the audience’s strategies and communications preferences, CIOs can begin to better align the content and format of their communications.

Articulate Business Value

When it comes to communications, planning, and strategies, it will be incumbent upon CIOs to focus on business value. At the highest level, it can be helpful to think about business value along three key domains:

  • Value creation. This can include the delivery of new services, perhaps to keep pace with, or differentiate from, competitors. How can these efforts translate to key business metrics, such as customer acquisition or retention? For example, how will an upgrade to an e-commerce back-end’s scalability impact business metrics, such as increased checkouts per hour? How will enhanced responsiveness enable reduced shopping cart abandonments?
  • Resource optimization. While this can often fall into a cost-reduction discussion around IT budgets, there can be far broader business impact and value. For example, from an IT standpoint, an AIOps initiative that helps automate event management and remediation may boost the productivity of IT staff and reduce a team’s reliance on outside contractors and associated costs. However, this type of automation may also translate to cost reduction in other areas of the business, for example reducing staff calls into the help desk or avoiding disruption. Alternatively, maybe the automation will free up high-level staff to deliver against some other business-impacting effort.
  • Risk mitigation. A wide range of efforts can fall under this category. While security-related investments and initiatives come immediately to mind, there can be a number of other areas that can benefit. For example, the automation initiative outlined above can help with risk mitigation. Maybe persistent service issues are leading to increased customer churn. The automation of event management can reverse this trend by enabling faster remediation, reducing the scope and duration of outages. Further, intelligent automation can even set the stage for the detection of potential problems and the ability to preempt them before customers ever notice there’s an issue.

By thinking about IT initiatives within each of these categories, CIOs can help frame their efforts in a way that resonates most with the business stakeholders they need to collaborate with.

Optimize Communications

Finally, look to ensure communications are optimized for maximum impact. Look to ensure your communications adhere to these principles:

  • Relevant. Ensure your communications map to the strategies and business values outlined above.
  • Clear and concise. To the greatest extent possible, try to eliminate ambiguity and deliver clear, succinct messages.
  • Credible. Provide key points that are supported with evidence, including metrics and examples.

Conclusion

As organizations continue to pursue digital transformation and BizOps initiatives, the alignment of IT and business grows increasingly vital. Consequently, as I outlined in my first post, [link to follow] communicating with business leaders represents an increasingly strategic component of the CIO’s job description. By gaining an understanding of the audience, clearly articulating the business value of IT initiatives, and optimizing their interactions, CIOs can begin to establish the communications that foster alignment, and help facilitate successful BizOps initiatives.

If you’re interested in learning more about BizOps, be sure to review our eBook, “The Definitive Guide to BizOps: Meeting the Digital Transformation Imperative Through 2021 and Beyond.” This guide examines why BizOps is becoming such an important imperative and it reveals the essential requirements needed to make BizOps a reality.

Why Communications Break Down Between CIOs and Business Leaders

Why Communications Break Down Between CIOs and Business Leaders

When asked to describe the skills most critical to the CIO’s success, “communications” may not be the first phrase to jump to mind. However, in an age of BizOps and digital transformation, the role of communicator is one that can have an increasingly pivotal role in a CIO’s ability to lead change and deliver value. As a marketing and communications executive, I’ll share two posts perspective on why communicating with business leadership is such a vital mandate for the modern CIO and offer some key strategies for optimizing these communications. In my first post, I’ll examine why communicating with business leadership is such a vital mandate for the modern CIO, and some of the challenges that have impeded these efforts.  In my second post, I’ll offer some key strategies for optimizing these communications.

The BizOps Imperative, and the Implications

The lack of alignment between business and IT has been a reality forever, or at least as long as there’s been IT and business. However, the reality is that schisms between these organizations have persisted, and alignment isn’t what it can and should be.

BizOps can be integral for addressing this alignment challenge since it represents a strategic architecture that transforms decision making. Much the way DevOps can accelerate digital delivery, BizOps can support the establishment of a strategic foundation that can accelerate and scale digital transformation.

As organizations pursue BizOps, the demands on CIOs will continue to evolve. Moving forward, one of CIOs’ most critical roles will be that of communicator, and of particular importance will be their ability to communicate with business leadership.

The Challenge

Historically, IT and business leaders have operated in an isolated fashion, with distinct organizations, teams, processes, and so on. Part of this distinction also arises in terms of communications and language.

It shouldn’t be a surprise that CIOs are comfortable with, and default to, technology speak. In their day-to-day jobs, these executives are inherently focused on concepts like capacity, availability, and performance. However, It is this fundamental reality that can mean that a CIO’s communication style isn’t aligned with the business leaders receiving them.

The inability to communicate optimally with business leadership can be significant and far reaching and can lead to a number of potential ramifications. Maybe a CEO isn’t sold on the business value of a technology initiative, and ultimately that initiative doesn’t get funded. Or within the larger business, the perception of IT’s value is diminished or stays low. Perhaps support for ongoing IT initiatives and policies isn’t sustained or maybe business leaders choose to go around IT altogether. Maybe line-of-business leaders don’t buy in to IT approaches, so they don’t advocate for adherence to policies, and compliance lags.

These implications are increasingly costly for organizations. Ultimately, it limits the ability of IT to contribute to critical business objectives, which can have a direct and significant impact on the business’ prospects and degree of success. Further, these obstacles will continue to stifle transformation efforts, and are very much anathema to vital initiatives like BizOps.

Conclusion

As organizations continue to pursue digital transformation and BizOps initiatives, the alignment of IT and business grows increasingly vital. Toward that end, it’s more important than ever for CIOs to communicate effectively with business leaders. In my next post, I’ll offer a practical look at how CIOs can begin to optimize their communications with business leaders. Also, if you’re interested in learning more about BizOps, be sure to review our eBook, “The Definitive Guide to BizOps: Meeting the Digital Transformation Imperative Through 2021 and Beyond.” This guide examines why BizOps is becoming such an important imperative and it reveals the essential requirements needed to make BizOps a reality.

Video – What is AIOps and Why it Matters

Video – What is AIOps and Why it Matters

In this video, you’ll learn how AIOps is helping our customers increase retention, drive acquisition and increase revenue by predicting and fixing your most common IT problems before they impact customer experience.

From Data Overload to Data-Driven: Three Key Trends Shaping Decision Making in 2020

From Data Overload to Data-Driven: Three Key Trends Shaping Decision Making in 2020

When is too much of a good thing a bad thing? It’s a question we’ve all asked ourselves after over-indulging in a holiday meal. (Maybe it was before the third helping I should have asked that very question).

Many enterprise decision makers are feeling the same way about the volume, velocity and variety of data in their organizations. While data is integral to every key business outcome, more isn’t necessarily a good thing.

As we look toward 2020, it’s clear a lot of changes are in store. The way enterprises are making data driven decisions is fundamentally changing. We need new ways to store, analyze and report on data that is augmented by AI and machine learning and ties back to business outcomes. Let’s take a look at some of the key trends forcing change in this area.

#1 Contending with Spiraling Data Volumes

Technology environments continue to grow more dynamic, composed of more disparate yet interrelated systems and elements. Environments now tend to feature a diverse mix of virtualization technologies, clouds, containers, microservices, orchestration systems, and more. Increasing layers and types of security technologies continue to be implemented as well.

For all these reasons, the volume, variety, and velocity of data that needs to be correlated and analyzed continues to grow dramatically. Decision makers need to accelerate their ability to sift through massive amounts of data, and gain the actionable insights they need to optimize performance, service levels, applications, and investments.

Relying on more silo’d point solutions, teams are drowning in data, while lacking real insights into how technology performance ultimately equates to business outcomes. If your teams haven’t reached a tipping point, with trends like IoT and 5G making their presence known, they will soon.

#2 Adopting a BizOps Approach

To meet their agility imperatives, technology and business leaders are adopting an emerging methodology called BizOps. IDC analysts define BizOps as “a data-driven decision-support mechanism that connects business and technology functions together to drive business outcomes.”

Through BizOps, teams can ensure that digital infrastructures and software provide the operational characteristics—including speed, scale, efficiency, and agility—that business services require.

Teams that employ BizOps practices need to create a data-driven decision-support mechanism that connects business and technology functions together to effectively achieve desired business outcomes. This isn’t easy: Enterprise leaders pursuing BizOps objectives are stymied by organizational silos that separate people, processes, and technologies. The result is that the data required is also locked in silos—stifling collaboration, insights, and innovation.

#3 Employing SRE Models

While the SRE model has been around for more than ten years, the reality is that some enterprises are just now beginning to pursue this approach. Within these organizations, IT leaders are wrestling with ways to maximize agility, and the SRE model is emerging as a key enabler.

As teams seek to pursue SRE initiatives, the tools in place can offer significant support—or pose a massive detriment. The reality is that many organizations looking to adopt SRE models are either employing in-house developed open-source tools or loosely connected toolchains. Quite often, these decisions are made at departmental level. This results in tool sprawl and, due to the ensuing heterogeneity, makes it harder for staff to find a single source of truth for problem solving. In addition, by introducing a multitude of tools, integrations, and administrative privileges that are difficult to monitor and manage, these approaches can pose significant security risks.

Data Driven Decisions. Done Right.

As we look forward to 2020, enterprise decision making needs to undergo fundamental transformation, growing increasingly event-driven and automated. Teams need to embrace a new AI-driven software intelligence platform that provides breakthroughs in the way decision making is accelerated and automated within an enterprise. Through these capabilities, enterprises can establish the continuous insights and collective intelligence to optimize decision making.

Broadcom offers a new innovative Digital BizOps solution that supports today’s emerging decision making imperatives. Digital BizOps uses AI-driven insights to deliver intelligent recommendations across Business, Development and Operations to transform customer experience, increase employee productivity, improve operational efficiency and speed innovation.

Digital BizOps is powered by Automation.ai – the industry’s first AI-driven software intelligence platform that harnesses the power of advanced AI, machine learning and internet-scale open source frameworks to transform massive volumes of enterprise data from disparate toolsets into actionable insights.

Learn more.

How can your Center of Excellence help to reduce alarm fatigue?

How can your Center of Excellence help to reduce alarm fatigue?

When I talk with people about Digital Transformation there is always the thinking about how they can digitize business processes, increase the speed of delivery and reduce costs. This is a very simplistic view and does not provide enough value to the business – simply scripting will not increase your quality and in the cloud world a tough challenge.

Your services are running everywhere, from your datacenter to multiple cloud suppliers, each has its own set of management and monitoring tools which makes control and visibility of your business processes more complex.

At the same time, the expectation of IT has grown dramatically, it’s no longer about downtime and availability. It’s about agility, quality, and speed. Slow is the new downtime. For example, 53% of visits are abandoned if a mobile site takes longer than three seconds to load. Downtime is very expensive. According to Gartner, the average cost of IT downtime is $5,600 per minute, each of these has significant potential for reputational damage and lost revenues.

So simply trying to go faster is not the best for the business.

Insight and Growth of Complexity

With the distribution of processing across hybrid environments everything has got far more complex. An ever-increasing number of monitoring tools that are disconnected from the enterprise processes has significantly increased the number of alarms we have to react to. That has created more pressures for Enterprise IT to deliver the services the business and our customers expect.

72% of IT organizations rely on up to nine different IT monitoring tools to support modern applications. Keep in mind: this is the situation before they started their digital transformation initiatives. According to the same survey, 47% experience on average more than 50,000 alerts per month. Whenever an alert activates, it requires identification and verification to initiate (if necessary) the right remediation processes which take a lot of our time.

But it is not all doom and gloom, there is a way to move forward in your Digital Transformation, be able to embrace the latest cloud services and deliver an exceptional user experience without breaking the bank. Noise and Silence of Alerts But …wait… 50,000 alerts per month? How do you handle your alerts today? What are your plans, when the number of alerts grows? And it will happen with your digital transformation initiatives.

This is where artificial intelligence for IT Operations (AIOps) coming into play. AIOps is the future for IT Operations and combines big data, machine learning, and automation to observe, analyze, and act. It reduces noise while collecting and correlating data from disparate sources like different performance monitoring tools to be effectively analyzed. With machine-learning-based insights, AIOps allows identifying abnormal behaviors or potential risks in an early stage.

However, most companies still remediate these alarms with manual effort, so the meantime to detection has been sorted but time to resolution is still a problem. “You know my methods, Watson” Automation is the right power tool to create and manage the foundation of your digital transformation. In AIOps, the right processes will be initiated when abnormal behavior, potential risks, or alerts come up – this is autonomous remediation without any human effort.

It is time to combine the knowledge and experience of the IT Infrastructure and Operations (ITIO) teams with automation. To centralize and digitize all existing documentation and orchestrate the existing tools to manage our environment which allows us to remediate alerts automatically. This reduces workload, mean time to repair, and enforce best practices across your organization. With a systemic approach to automation your “Automation Center of Excellence” then enables the agility across the enterprise. Automation is the backbone of the IT organization and its focus goes beyond Continuous Delivery and Digital Business Automation. AIOps is a part of it – an important one for your digital transformation initiatives.

Video – AIOps Silicon Insights Delivers Unparalleled Network Visibility and AI-Driven Remediation

Video – AIOps Silicon Insights Delivers Unparalleled Network Visibility and AI-Driven Remediation

Broadcom is simplifying Hybrid Multi-Cloud and SD-WAN deployments by combining AI and ML with rich granular data captured at the chip level for one-of-a-kind AIOps analytics and automation.

Our customers are experiencing challenges today with managing complex network architectures like SDN, SD-WAN and NFV to deliver an innovative, reliable and responsive digital experience.

Cost is one major challenge. It is expensive to run network operations today and requires many personnel to architect, monitor, understand patterns and triage these new architectures.  Recent studies show that an enterprise loses an average of $9k every minute during a data center outage. Some of our customers have reported up to $20M in losses to data center outages per year.

Speed is another challenge. Today’s digital experience requires a new level of network operational responsiveness to be able to identify, diagnosis, react and solve problems quickly to avoid bad customer sentiment and mounting operating losses.

It is critical to bring AI and ML through an advanced AIOps solution to network operations today to solve for reducing costs from operating, avoiding outages and to speed up root cause analysis for faster triage and operations.

AIOps can eliminate hours, days and weeks spent on manual pattern identification of trend charts looking for any unusual activity in the network. It can enable advanced root cause analysis and anomaly detection through intelligent thresholding and alarm noise reduction. Furthermore, AIOps enables advanced predictions, correlations and automated network triage. AIOps can learn from analyzing network activity to automate, repair and tune the network for reduced operational costs and faster triage to deliver consistent and exceptional digital experiences.

AIOps latency

Figure 1: Clicking on the affected metrics tab displays the power of our Machine Learning (ML) algorithms used to identify anomalies in network behavior.

We all understand that no one solution fits all problems, and network monitoring is no exception. Some next-gen use cases such as dynamic traffic engineering require granular visibility at flow or packet level in real time. Broadcom’s Inband Flow Analyzer (IFA) is a flexible, scalable and real-time monitoring solution that leverages Inband Telemetry technology to deliver granular telemetry at flow or packet level. Broadcom also supports several key out-of-band telemetry use cases such as streaming telemetry directly from the ASIC. AIOps for SD-WAN and Hybrid Multi-Cloud from Broadcom enables one-of-a-kind network monitoring, leveraging state of the art in-band and out-of-band telemetry technologies, to enable use cases such as Packet Path Tracing with Route Change Detection Alarms, Microburst Detection with Proactive Congestion Monitoring, Mirror On Drop providing deep visibility into packet drops and Elephant Flow Monitoring.

BroadView deep flow per packet analysis

Figure 2: Timed stamped packet path tracing enables per hop packet latency visibility.

AIOps for SD-WAN and Hybrid Multi-Cloud is powered by Broadcom’s industry-leading silicon including Trident 4 and Jericho2 families that enables disruptive advances in Enterprise data center and campus networks.

Highlights of Trident 4 include:

  • A scalable architecture, with bandwidth from 2.0 to 12.8 terabits per second
  • Extensive compile and run time programmability, allowing changes to behavior during switch operation
  • An extremely rich feature set with a high degree of feature concurrency
  • Large, fungible databases and an industry-leading on-chip packet buffer
  • Advanced Telemetry and Instrumentation

Highlights of Jericho2 include:

  • 10 Tb/s in a single device, delivering hundreds of Mb/s (Megabits per Second) per subscriber
  • Elastic Pipe, delivering advanced packet processing, programmability, virtualization and investment protection
  • Large scale, carrier grade, expandable IP-MPLS forwarding
  • Hundreds of milliseconds of deep buffering
  • Advanced Telemetry and Instrumentation

Trident 4 and Jericho2 support programmable dataplane leveraging state of the art Network Programming Language (NPL).  IFA complements programmable dataplane through innovative mechanisms to enable next-gen use cases, such as measuring real-time application latency in cloud networks, in a scalable way.

AIOps Silicon Insights operationalizes complex Cloud and SD-WAN environments by combining AI and ML with rich, granular packet data captured at the Broadcom chip level for one-of-a-kind network analytics and automation.

LEARN MORE

See AIOps Silicon Insights in action here:

Actively Manage Your Products. Or Fail.

Actively Manage Your Products. Or Fail.

Organizations have limited amounts of money available for investment, and that investment has to generate a substantial return – achieving all of the goals and objectives that have been set for the reporting period, while also moving the organization closer to achieving its long-term vision. There isn’t room for investment dollars to be wasted on initiatives that don’t contribute, or even on projects that contribute in a less than optimal way. That’s where the integrated product portfolio comes in.

It informs the decisions around the product portfolio by allowing leadership to ensure:

  • Investments are focused on innovation and growth – that’s the only way corporate performance can advance without €œthrottling” (cost reduction is important at times but will always have limited upsides because all costs have a floor).
  • Investment is balanced across different elements of the portfolio. Investments must be distributed across products at different stages of the lifecycle – ensuring those products that are currently enjoying high levels of adoption and sales remain attractive to fund the development of new products while ensuring that declining products remain profitable by limiting investment. The same logic applies to distributing investments across different market segments, geographical regions, etc.
  • Project managers and teams have access to the full context of why their initiatives were approved. Instead of simply having a high-level explanation of project purpose the teams can see how their product or service fits into the enterprise product portfolio and how their work directly benefits the organization’s ability to succeed.

Of course, all of this is dynamic. Modern business is continuously evolving as technology advances, customer expectations increase and competitors drive innovation. The product portfolio must therefore be managed actively, ensuring not only that each product or service is accurately represented in terms of current positioning, but that future directions and plans are realistic in that product’s environment. Each individual adjustment will also impact the overall balance of all products or services, requiring further adjustments to retain an integrated product portfolio that distributes opportunity, risk and effort appropriately.

This relationship between product and project portfolio management is at the heart of why the product portfolio must be actively managed – continuously adjusting it to reflect the current operating and competitive environments, and positioning it to optimize the ability to respond to every opportunity and threat. The integrated product portfolio defines the plans, positioning and expectations of all the organization’s offerings – both individually and collectively. The project portfolio represents the work that is being undertaken to deliver on those plans, maintain or enhance the positioning and fulfill the expectations. Success only comes when both of those aspects are aligned.

Organizations recognize that maintaining alignment in the project portfolio requires ongoing adjustments, and successful organizations have built that approach into their strategy execution approaches. They maintain an actively managed portfolio backlog, conduct planning reviews on at least a quarterly basis, and practice business agility to ensure the need to adjust goals and objectives is identified as early as possible.

That approach must now be extended into the product portfolio. Product owners should already be aware enough of the markets their offerings operate within to identify as changes are starting to occur. They must also be able to interpret the implications of those changes – does the product need to evolve and adapt, does it need to be repositioned, or can the change be ignored as insignificant? Organizationally, all of those ongoing adjustments must be brought together, evolving the entire integrated product portfolio and ensuring it always represents the current situation, opportunities and threats for all products in all markets.

Anything less risks failure.

Fraud Prevention: What’s Holding You Back?

Fraud Prevention: What’s Holding You Back?

Some people like to count it, whilst others like to hoard it and some (rather irritatingly) like to jangle it.

To what am I referring? Cash, of course.

However, one thing’s abundantly clear, we’re all doing less counting, less hoarding and, mercifully, less jangling. Because cash is no longer king.

There is a pub in South London where, if you’re only carrying cash, you’re going to stay thirsty.

“Apologies, but it is the digital age”

-Crown & Anchor

Because in October, the Crown and Anchor went fully cashless. Only debit or credit cards are now accepted behind the bar.

And what’s more; there’s been no discernible downturn in takings.

Over in Sweden, whilst they are stepping towards it more cautiously, they are the closest of any nation to becoming a cashless society. A fifth of all Swedes no longer use an automated teller machine.

Across Europe, about one in five people say they rarely carry money. In Belgium, Denmark and Norway, debit and credit card use has hit record highs. In the U.K. cash use has halved in the last ten years with cash now used in only three out of every ten transactions.

Concerns might be raised over the impact of the reduction in the use of cash on parts of the population but there’s an even greater existential threat as a result of increased use of plastic and digital payment methods: fraud.

Retail fraud is on the increase. CNBC reported that last year the “level of fraud as a percentage of retailers’ revenues has climbed to 1.58 percent year to date from 1.47 percent,” compared to 2016.

Card-Not-Present

However, fraudsters have now turned their attention away from point of sale fraud to card-not-present (CNP) fraud which is becoming an organised crime activity.

According to Financial Fraud Action UK losses on UK issued cards totalled £618.0 million in 2016, a 9% increase from £567.5 million in 2015; the fifth consecutive year of increase and higher than the peak of £609.9 million seen in 2008.

At the same time, total spending on all debit and credit cards reached £904 billion in 2016, with 19.1 billion transactions made during the year.

In the U.K. the police’s Dedicated Card and Payment Crime Unit (DCPCU) has prevented £25m of payment fraud in the first half of 2018, including disrupting seven organised crime groups.

In addition, 8,651 stolen card numbers were recovered.

However, retailers say it’s difficult to verify someone’s identity online and the less advanced, less sophisticated legacy solutions still in use are no longer fit for purpose in the face of increased sophistication on the part of fraudsters.

The Game Just Got A Whole Lot Tougher

We take much for granted in this digital age and we embrace the ease and convenience which it presents to us.

But this also means that merchants and retailers must be ever more vigilant in the fight against fraud.

Techniques such as falsifying their location to get past standard fraud detection or making efforts to push fraud through in bursts before the new fraud patterns can be understood are becoming more commonplace.

The biggest trend the DCPCU are seeing is brokerage, or “fraud as a service”. Brokerage occurs when fraudsters carry out fraud on behalf of others.

For example, they might advertise an iPhone X for £750 on Facebook when the actual retail price is £999, use compromised card details to purchase the phone on behalf of the consumer and pocket the difference.

Often the consumer has no idea that there’s a fraudster playing middle man due to the sophisticated scale of operations and marketing tactics they use.

Take Action

In the fight against CNP and online fraud there is however some good news.

First invented by CA Technologies, A Broadcom Company and Visa, the just-updated EMV® 3-D Secure protocol brings a new approach to authentication through a wider range of data, biometric authentication and an improved online experience.

With the new protocol, CA uses its patented neural networks to analyse fraud patterns in real time, using unique data such as transaction velocity.

Crucially, it is possible to share data between banks and merchants silently in the background, meaning authorization rates can be increased with no perceivable change to the checkout flow by customers.

So, if you’re thinking of cheating a retailer, better check first to see if they are running CA Technologies EMV 3DS fraud protections. Chances are they may very well be – in which case, the game’s up.

For more information please click here.

Breaches of Data, Breaches of Trust

Breaches of Data, Breaches of Trust

Which of these two recent trends disturbs you the most? 

With the recent breach announcement from Marriott, you could almost predict the fallout.  First, the news outlets all jumped in to report the story and highlight what you should do if you may have been impacted, or basically restate what was already stated in the Marriott press release or make hypothetical predictions about the attack.  Second, every security vendor immediately released social tweets, blogs, and press about how their app/software/hardware/expertise could have prevented this breach from happening.  Third, every non-security person with an opinion weighed in – I like to call this group the gluten-free, organic, low sodium, nut-free peanut gallery.  I had a good friend who loved using this quote from Mark Twain:

“It is better to keep your mouth closed and let people think you are a fool than to open it and remove all doubt.”

– Mark Twain

I cannot help but think of this quote whenever I read comments from the peanut gallery.  But to get back to my point, there seems to be much more attention and focus on breaches of data compared to the recent breaches of trust.  And I wonder – is this a good thing?

The Impact of Breaches of Data

For organizations that suffer a breach, the repercussions can be costly.  In the recent Global State of Online Digital Trust Report, Frost & Sullivan found that 48 percent of consumers will stop using a service after a breach is reported.  Additionally, almost every organization that had been involved in a publicly disclosed data breach shared that the breach had “long-term negative impact to their revenues and to consumer trust.”  Zig Ziglar said that “if people trust you, they will do business with you.”  Trust is even more critical to online business because Frost & Sullivan found that consumers “with the highest levels of digital trust increased their net spending online significantly more compared to those with the lowest.”  The bottom line is that data breaches lowers a consumer’s digital trust in an organization, and this has negative impacts to revenue.  But what about the other breaches of trust that we have seen recently?

Breaches of Trust

Another alarming trend, although it does not seem to get the same level of exposure, is selling personal data for profit – which is, admittedly a breach of trust.  Facebook has been the poster child for this new trend because they were caught selling personal data, and more recently because it was discovered that there were discussions to further profit on collected data.  GoogleApplecell carriers, and more recently, the Weather Channel have all come under fire for how they are handling, or potentially mishandling personal data they are collecting.  However, this trend is more widespread than you think.  Frost & Sullivan found that “43% of business leaders indicated that they sell personally identifiable customer data to other organizations.”

Many argued that this practice was being done legally because this was clearly spelled out to users within their online terms and conditions agreements – you know, those pesky legal disclaimers that are written in tiny font and written in a language that only an attorney could decipher; the ones we all click through as fast as possible so we can actually use the app or service.  Yes, this is where most organizations are telling us that they are going to treat our personal data like a commodity on the open market.  It’s like shaking on a deal with one hand, while you have your fingers crossed on your other hand behind your back (the universal sign that you plan to renege on your deal).  One bright piece of news on this front is that the European Union is looking closely at these agreements as “they could be interpreted as forced consent by the General Data Protection Regulation (GDPR).”

Summary

Most studies are focuses on the impact of data breaches, but few examine the impact of breaches in trust. Does the public view these equally or not? Which is worse – to accidentally lose data because your organization was targeted, or to purposely sell that data to third-parties for profit?  In my mind, there is no contest – one is clearly worse than the other.  To use a legal analogy, breaches of data compared to breaches of trust are like manslaughter is compared to murder; it all comes down to intent.  And as such, the public scrutiny, penalties, and fines should be much harsher for one versus the other.  What do you think?

A Pendulum Shift – API Security v API Management

A Pendulum Shift – API Security v API Management

Two Security Experts Discuss Industry Direction in This Podcast

As API security issues are becoming a regular headline – and as a result, more and more industry analysts are beginning to shift from an emphasis on full lifecycle API management towards a focus on API Security as the key baseline for any digital transformation project.

Recently, Jay Thorne, Product Advisor for Layer7 at CA Technologies, a Broadcom Company, had a chance to sit down with Alissa Knight – cybersecurity sluth and market analyst for Aite Group to talk about that exact subject.  Alissa interviewed  Jay on API Security, API Gateways, the future of security in the new API economy, and the scoop behind the growing number of API breaches.

I invite you to listen in on this episode here.

Your DevOps Initiatives Are Failing: Here’s How to Win

Your DevOps Initiatives Are Failing: Here’s How to Win

Let’s start off by stating the obvious. Right now, it’s highly likely that your organization is struggling to deliver quality software on time, and on budget. Scope, cost, schedule — you’re told you can have control over two, but you’re likely struggling to nail down even one of the three.

And you aren’t alone. In fact, over 70 percent of IT projects fail. This failure dooms careers and companies, causing a collective drain of over $1.7 trillion on the global economy annually – an eye-popping sum that exceeds the GDP of all but nine countries.

While we all know that having a continuous testing process in place will dramatically reduce the risk of failure, a lot of organizations still struggle with the adoption of continuous testing methods. So how can you ensure your team wards off failure and avoids becoming another statistic?

Read on to find out how to avoid some of the most common hurdles and pitfalls.

Make proper requirements & planning a top priority

The single biggest reason projects fail is a lack of actionable requirements.

Right now, it’s likely a huge number of your requirement tickets only consist of a short summary sentence in the title. Or maybe someone has attached a monolithic PowerPoint presentation to a single ticket, documenting new features across multiple sprints. Perhaps instead you’ve got a few thousand auto-generated defects from an accessibility audit that was completed two years ago? It doesn’t matter. Regardless of what format they take, inconsistent requirements force developers to rely heavily on their own subjective interpretations of what to build. And this eats into their productivity, and significantly decreases your chances of success.

How do I get it right?

To be successful, requirements need to be clearactionable, and testable. This can help you avoid:

  1. Developers building something other than what was intended,
  2. time wasted on meetings to align on requirements,
  3. dramatically slower production times,
  4. missed schedules due to having to rebuild during UAT, and
  5. an increase in problems making it through to your customers in production.

To reduce risk, keep your requirements as minimalistic as possible and build towards MVP. And at an absolute minimum, ensure you have a user story, acceptance criteria and a testing plan to keep you on track.

The goal of requirements should be to have enough information in every ticket for any developer to pick it up and run with it, without having to spend time deciphering email threads or setting up meetings to understand the ask. Having a good set of requirements templates will help, along with having a unified project-wide guide documenting workflows, so everyone involved knows how your process works.

Refine requirements before you build. All new requirements should be reviewed by your architects, developers (ideally the same ones who will be doing the work), and your quality assurance teams. Make sure to allocate time for them to ask questions and give their feedback. It’s also important for someone from your QA team to be engaged during planning to help create testing plans, write test scripts and identify test data sets needed. Managers and sales teams simply passing on requirements to developers without making themselves available for discussion, or encouraging developers to suggest improvements is a horribly outdated way to work (and almost guaranteed to end in failure).

Create a unified source of truth for your requirements. To achieve company-wide alignment on what “done” means, and the process for how to get there, make sure every team touching the code works with it in the same way. Far too often “build squads” and “maintenance squads” each have their own way of doing things, their own tools for requirements or defect tracking or their own deployment work-flows. This inconsistency will bring chaos to your project.

Get everyone on the same page. Consider giving everyone in your organization access and training on your requirements planning system. Insights can come from anywhere, and keeping systems transparent and open can help you get important information faster. When the right people have the means to communicate directly, it removes the unnecessary time and risk associated with a complicated chain of command.

Create consistent issue templates. Once everyone is trained on the process and tools (make sure to build this into your on-boarding process!) you can create consistent issue templates. Start simple, with “new feature”, “enhancement”, and “defect”. It’s likely some issues won’t have every single necessary detail, so developers can be encouraged to enhance issues with their assumptions before coding. However, make sure everyone knows your “getting to done” process, and make sure it’s culturally acceptable in your organization for people to send tickets that don’t meet standards back to the creator for more details.

Always ask for feedback. Every team will have slightly different needs and requirements, so it’s important to get feedback on your templates. Make sure you’re documenting what everyone needs, and if your software has to go through accessibility testing, security testing or any other compliance testing, make sure those requirements are stated up front so everyone can build with those standards in mind. Likewise, make sure your coding standards, approved licenses and code review processes are publicized and kept up to date.

Now that you have a set of requirements templates that your team is happy with, set up a process for the development lifecycle. We’re all familiar with the kanban methodology columns – “To-Do”, “In Progress”, “In Review” and “Done” – so consider adding a “Planning” column to ahead of this and mandating that nothing moves into “To-Do” until it’s been reviewed and everyone on the team is happy with it. This will keep your “In Progress” tickets out of the “To Do” pipeline and let everyone know that the to-do list is actually ready to be worked on.

Automate provisioning, builds, and data generation to automate testing

Continuous delivery requires automation as you move code between environments. It’s important to work out a plan with your DevOps team to ensure environments can be automatically provisioned and code deployed to them for testing. While the industry on the whole is improving, far too many enterprise-grade companies still have a manual build process – and while being able to make “daily builds” was great 20 years ago, the modern standard should be more aligned with, “Can you create a build on-demand, with one command?”

These builds don’t just require an environment and code. They also need to be tested at every step, to help you catch problems early and at the source. Testing requires access to quality test data, and far too often, tests are run on small data sets that only test perfect-path results.

Expect the unexpected

Perfect path testing leaves a lot to chance. What happens when an API you depend on fails? If you don’t know, or didn’t build in the right fallback or user messaging, you’re going to have unhappy customers. So make sure you have a process for creating comprehensive and realistic test data, or pulling sanitized data from production that you can use as part of testing. Having realistic tests, imperfect-path tests, or even some chaos testing, can really help you at this stage.

Also bear in mind that modern systems are complicated and often interconnected. We are not just building one stand-alone application; we are building something that relies on numerous back-end systems and 3rd party integrations, which may or may not come with a sandbox mode for testing. This means that in addition to “internal” test data, you may also need simulated virtual or “mock” services for those times you can’t test against production or a 3rd party service.

By introducing mock services, you can write unit tests that perform negative testing or chaos testing, ensuring you know how your code will perform under a variety of real-world circumstances. Sounds complicated? It’s not if you use the right tools. And if your organization is struggling with unit test coverage, focus on API-endpoint testing as a starting point.

Reduce, reuse, recycle

While automation can help, another thing your team can do is optimize and reuse code in order to reduce overall workload. This decreases the number of tests required and the effort needed to fix bugs and make enhancements.

Look to common areas – like user sign in pages or managing profile forms – and ensure you’re reusing validation and error handling. This will help give a consistent user experience, as well as simplify the amount of code you have to support. Given that 70% of testing is still manual, small steps like this can help free up your QA engineers, giving them more time to work on automation tasks.

Why automate?

Allocating time to create automated tests should be high priority. Once you create a test, you only have to touch it again if the underlying code it’s testing has changed. Therefore, it is endlessly reusable with a tremendous ROI. The effort savings will typically pay off within one sprint – imagine the savings you will get over a year, or the lifespan of that code.

Automation is not just about time savings; it also significantly increases quality. Even a simple dependency update can have a knock-on effect on other parts of the system, with potentially sweeping ramifications. The more you automate, the easier it is to do full-system tests to ensure your “simple change” has the intended outcome (with no unwelcome extras).

It gets even better – many modern tools allow you to record front-end UI tests, and replay them, meaning with the right tools it’s a zero-cost exercise to build the infinitely reusable automated tests vs. just doing manual testing one time. That’s a no brainer.

Work smarter, not harder

Assuming you have clean requirements, another huge win is the ability to automatically generate your test cases. Manually building and managing test cases is a massive time sink – and therefore it seldom gets done thoroughly. However, with the right tools and requirements, you can shrink the effort down a size or two.

During your automation transition, you’ll likely experiment with a number of tools, and the cost of open-source tools makes them appealing. But when you’re ready, look for enterprise tools that will enable you to import any tests you created. Being able to reuse things you’ve already got on hand will make it much easier to keep momentum.

Don’t feel like building everything yourself from scratch? Many frameworks and platforms have great tools for sanity checking their own best practices or a predefined set of functional tests. While these tools won’t always be tailored for your specific needs, some test coverage is better than none. For web projects, Lighthouse is a great tool that can be easily be added to your build process to check web pages and report on general performance, security, or usability issues.

Bring tools to developers

During development, the cost of fixing a bug is dramatically cheaper than trying to fix it once it goes into production. A stitch in time saves nine (or in this case 1000 – yep, by some accounts it can cost 1,000 times more). Recent failures by Boeing and Samsung highlight the damage a lack of proper testing can have on your company’s reputation.

It’s not hard to see how these sorts of issues happen. With the old way of doing things in linear siloed phases, testing wouldn’t happen until developers were finished. Given that testing was typically time-boxed, it was rare all the tests would run; then on top of that, the resulting data may not make it back to developers until the following sprint. These delays introduce a high level of risk, with plenty of time for the original developer to forget context or for the project to have been handed over completely to somebody with no prior knowledge of the requirements.

Instead, a quick win is finding tools that easily integrate with your developers IDEs, or can be run from the command line or in code. By integrating all the tools together, you remove the hurdle of multiple systems and environment variables, making it easy for developers to find bugs in their own code and immediately fix them. To reap these benefits however, it’s important that everyone writing code has access to the testing tools, knows how to use them, and will use them.

Build a collaborative culture of rapid feedback

The goal of continuous testing is to create rapid feedback loops, but that’s not limited to code. Since software development does not end at launch, it is important to cultivate a culture of continuous feedback; and opening your development stand-ups to anyone who has work slated that sprint will ensure your team is better able to get the answers they need. Having cross-functional retrospectives will also help ensure your continuous testing initiatives are working for everyone, and will give you the opportunity to fine tune as you go. If you can’t find a time for everyone to attend, offer the ability to submit feedback via a “suggestions box” (you can easily set this up with a Google Form).

Remember, retrospectives are not just for calling out things that need to be improved – they’re also a great opportunity to call out the people who are doing things right. Teamwork is key, and nothing builds rapport like gratitude. Things will not always run smoothly, but one key metric to follow is how fast you can address issues that are causing pain – and the easiest way to do this is to capture the mood during a retrospective.

Adjusting to the new normal

Even with support from all stakeholders, simply mandating that everyone switch to use continuous testing methodology without training and time will cause your initiatives to fail. Continuous testing cannot be mandated to just one team, or by one team. It is a fundamental commitment to quality that must be embraced by everyone in the organization. Your teams will need time to adjust, adapt to new processes, learn new tools, and ask questions of one another – and all of this takes time.

Since most existing automated tests and testing data tends to reside with quality assurance teams, it is great to lean on them to champion change and share their testing knowledge across your organization. Quality assurance testers and engineers will also have a lot to offer and, just as we have agile coaches, can function as continuous testing coaches throughout the organization.

With all the required training, you may see an initial slowdown of productivity during the transition. It is important not to panic. A lot of things contribute to story point decline, but it’s important to let devs track non-development tasks – like meetings, training sessions, and retrospectives – to the story point count so you are tracking an accurate picture of how much the team is getting done. Change will happen fast, but it will not happen overnight. In the meantime, it’s important for everyone to stay flexible.

Conclusion

The best way to ensure your continuous testing initiatives are successful is for your organization is to:

  • Be proactive with requirements,
  • work to share knowledge and access across teams,
  • automate as much as you can (and make sure all teams can reuse the work)
  • generate appropriately robust test data, and
  • be kind to your future selves by budgeting time for continuous feedback and improvement.
Ultimately, software development is always going to be a complex process, and it won’t ever be perfect. But by taking these steps, you’re sure to leave everything a lot better than you found it.
Secure or Streamlined: The Digital Experience Paradox

Secure or Streamlined: The Digital Experience Paradox

Based on our research, a large majority of enterprises admitted that their digital transformation efforts frequently traded security for a streamlined experience and quick time-to-market. But behind this choice made at the altar of velocity lies a hidden truth about digital transformation – no matter how carefully projects are executed, delivering a better user experience almost always results in greater digital risk for the enterprise.

Digital Transformation versus Cybersecurity

recent study we conducted with Frost & Sullivan unsurprisingly revealed that the fastest-growing enterprises have fully embraced digital transformation and are succeeding, relative to their peers, at delivering satisfying experiences to users in the cloud, on the web, and on mobile or IoT endpoints. But this success has come at a price. More than two-thirds of respondents from these organizations admitted that the unrelenting pressure to release new apps or app updates quickly had negatively impacted quality and security.

In addition, among the most advanced companies – digital disrupters heavily invested in modern architectural patterns such as APIs, microservices and containers – almost 90% reported trouble securing these newer technologies. Clearly, businesses are aware that a serious problem exists, and most are struggling with it.

Speed of Releasing New Applications and Updates and the Consequences on Quality and Security

IT-Driven Cybersecurity Solutions are Band-Aids

Since the advent of digital transformation, the most common response to this challenge has been to delegate it to technology practitioners, which seems obvious, since these professionals tend to be subject matter experts on cybersecurity solutions.

The problem is that in the current era, where “every business is digital”, C-level and other business executives still deputize the entire challenge of cybersecurity, seeing it as a “technology problem” rather than as a critical business imperative. Our study found that very few business executives – only 16 percent – even considered security one of their main challenges. Even more surprisingly, among IT professionals, only a third listed cybersecurity as one of their top 3 challenges.

Cybersecurity a Top Challenge for Organizations by Business Function

Our conclusion is that security is most often viewed within the enterprise as “someone else’s problem”, to be solved via tactics and point solutions rather than as a business strategy. But as we navigate an era where a single breach or incident can result in massive liabilities, multi-billion dollar fines, new government regulations, and material impacts on a company’s value, this thinking must change.

The Better the Experience, the Greater the Risk

One of the most compelling reasons that key executives must be involved in cybersecurity decision-making lies in this paradox – delivering a better digital experience in terms of convenience, reduced friction, and customer satisfaction almost always leads to substantially increased exposure to digital risk. This is because current products and services tend to require the collection, storage and processing of vast amounts of sensitive, private information – PII, locations, conversations, behavior, preferences, transactions, financial, and health information. Against this backdrop, brand reputation, consumer trust, and regulatory compliance today are largely factors of how well an enterprise can maintain custody of this data once collected. Breaking this down further, executives must consider and balance this paradox from several different perspectives:

More Data, More Risk

Personal information is often the key to delivering compelling, convenient digital experiences. But individual data elements are also liabilities for as long as they are held, with potential reputational, financial, and legal costs should the enterprise ever lose control of them. Failure to consider these costs strategically can amplify security incidents, as was the case in the recent Capital One data breach, which was made far worse due to the likely-unnecessary storage of seventeen years’ worth of credit applications in a live production system.

More Partners, More Risk

Once data is collected, the “magical experiences” that today’s users demand often require additional processing or supplementary services beyond the ability of any one enterprise to deliver on its own. Examples are everywhere, from mapping and social network integration to outsourced natural language processing, fulfillment, and delivery. Yet every partner that must be integrated to deliver a digital experience substantially increases the cybersecurity threat perimeter for the originating enterprise – and business executives need to be aware of this as another potential liability to be factored into their decision-making.

More Scalability, More Risk

Every enterprise is in the cloud today, and for good reason – the cost-effectiveness, agility, and scalability afforded by today’s infrastructure options are unbeatable. But migrating storage and processing to the cloud often increases operational complexity – and thus data risk – in unexpected ways. In their rush to reduce costs and overhead, many executives fail to appropriately budget and account for increased liability in domains such as privileged access. When business executives choose to collect highly personal information to deliver a digital experience, it is vital that they work with technology practitioners to fully consider the entire lifecycle of that data – no matter where it is stored, processed or transferred.

A New Hope: Consistent Security Across the Digital Experience Lifecycle

To tackle these problems successfully, C-suite and business executives should elevate cybersecurity concerns to the same strategic level as their customer experience – considering both sides equally as they work through the infrastructure and processes needed to deliver a desired digital product or service throughout its design, development, and operation.

Our team models this pipeline as an eight-stage Digital Experience Lifecycle, through which all digital experiences must continuously flow as they are created and iterated.

The Digital Experience Lifecycle

It’s essential for executives seeking to build trust and reduce risk in their digital transformation efforts to carefully consider cybersecurity and privacy throughout the digital experience lifecycle. Questions such as “do we really need to collect this particular data from customers”, “how long should we keep it”, and “who should be able to access this and how?” should be continuously asked – not just by IT practitioners, but by business executives whose P&L could be drastically affected by the answers.

At Broadcom, we believe that the best technology approach for large, complex enterprises is to seek or build an enterprise-wide platform specifically designed to protect data within the context of the entire digital experience lifecycle, rather than leaving it to IT to tackle individual challenges in separate silos such as identity, authentication, privileged access, and API security. We call this building a new architecture of trust, and I invite you to learn more about it in our short solution brief and two minute video.

Continuous Testing and Continuous Delivery in a DevOps World

Continuous Testing and Continuous Delivery in a DevOps World

Around five years ago, I was attending my very first conference when I first heard the term Continuous Testing. I had been living in a waterfall world: developers weren’t testing their own work, and life as a tester was less than amazing. Testing was then thought of as an activity only done by testers, who really just ensured the application worked met requirements (and we had a LOT of work coming into my team).

As I heard the speaker talk about this new (to me) phrase, I thought, “Great, now I have to test everything, all the time?” If we think about how many of us experienced ‘over the fence’ testing, this could be long, long cycles of manual checking. At one point for me, this took a team of almost 45 people, 3 weeks at the end. And that was just ONE CYCLE (we had as many as 6 cycles before something was released to production, if we were lucky, once a year). Even when we began automating, at best it was a nightly run for thousands of tests, and days of debugging if one failed. Since I didn’t know much about how Continuous Testing fit into a DevOps world, I took it to mean we had to figure out how to do that much “testing” all the time. I could think of nothing worse!

This session  turned out to be a (very disappointing) vendor pitch, and I wish I had heard more about Continuous Delivery, DevOps, and how testing fits into that world.

Flow and Feedback

To understand Continuous Testing and Continuous Delivery (CD), it’s important to look at the world of DevOps – the culture driving how we build, test, release, and operate software. That culture thrives by focusing on collaboration and communication, flow, feedback, and continuous learning and improvement. So how does testing and CD fit in?

Let’s take a look at the concept of flow and feedback. How does an idea go from a concept to something of value for our customers? As we look at the entire flow or process, think about how we develop our ideas. How do we explore those thoughts and discover the unknowns? What risks can we think of? How ethical is what we are trying to build? Asking ourselves these questions (and more!) is a form of exploration to discover the unknowns and give feedback – which, even if it is not me sitting at a computer, is still testing our ideas and assumptions.

Once we are ready to build, do we know how our code flows from that first commit into production? This is where Continuous Integration (CI), Continuous Delivery (CD), and Continuous Deployment (hey! Also CD! Confusing, right?) come into play.

I started in a world where we didn’t have CI. Devs could go days, even weeks, without merging their code. This made it SUPER hard to know which change broke any test we may have had running at the time. Enter Continuous Integration. This brought us the practice of introducing smaller changes, several times a day, with tests kicked off on each individual change (not a bunch of changes together). As we evolved and further matured our pipeline, we started looking at Continuous Delivery. We used to have once a year releases (nightmarish to test), then quarterly (still bad, but better than the annual releases). When CD came into the picture, coupled with the smaller changes that CI brought us, we were able to release much more frequently. These smaller releases were WAY easier to test, but we were still waiting until we felt we had enough changes of value. Enter Continuous Deployment. We started using feature flags. We could deploy on every small change to production.

As we began to understand our feedback loops throughout our CI/CD pipelines, we could prevent failures from going any further down stream if our feedback told us that change broke something along the way. We no longer had to think of feedback as one huge giant testing suite checking requirements. We could have smaller suites along the way, giving much more frequent feedback.

If I think of Continuous Deployment as driving a car down the road at night, and automation and testing as my headlights, I could _technically_ drive without those headlights turned on – but I might end up in a ditch. By using automation to improve my flow, and testing to give me feedback, I have a much better chance of getting where I’m going! (I’ll ignore the fact i have gotten quite lost along the way sometimes).

What About Testing in Continuous Delivery?

Guess what?! We haven’t even talked about what happens once we’ve given our clients something of value. Just as we can test ideas at the beginning of our process and as we build and deploy, how do we discover more unknowns and see what’s happening in the wild? We need to test in production! We log. We monitor. We introduce chaos. We OBSERVE. If we see something, we understand what we missed and we start the cycle from idea to production over again. As we seek to get feedback, we are testing. We are constantly learning, and improving, and testing.

Dan Ashby has one of my favorite models that helps me ensure I’m thinking about the feedback I need at every step in our process, and as we repeat the cycle. We test our ideas, our artifacts, our design, our infrastructure and code, our operations, our environments, our pipelines, and more!

Image Credit: Dan Ashby, Building a Quality Culture through Continuous Testing 

As you can see, Continuous Testing is not just about the testing that occurs in your CI/CD pipeline. It’s everywhere. Testing is the feedback we seek, in all aspects of design, developing, deploying, and maintaining our applications. It requires everyone to embrace testing, which has been one of the most positive mindsets I have seen evolve in my career.

For more resources and community, I highly recommend:

Don’t Become a Statistic: How to Save Your Failing Software Development Initiatives

Don’t Become a Statistic: How to Save Your Failing Software Development Initiatives

Let’s start off by stating the obvious. Right now, it’s highly likely that your organization is struggling to deliver quality software on time, and on budget. Scope, cost, schedule — you’re told you can have control over two, but you’re likely struggling to nail down even one of the three.

And you aren’t alone. In fact, over 70 percent of IT projects fail. This failure dooms careers and companies, causing a collective drain of over $1.7 trillion on the global economy annually – an eye-popping sum that exceeds the GDP of all but nine countries.

While we all know that having a continuous testing process in place will dramatically reduce the risk of failure, a lot of organizations still struggle with the adoption of continuous testing methods. So how can you ensure your team wards off failure and avoids becoming another statistic?

Read on to find out how to avoid some of the most common hurdles and pitfalls.

Make proper requirements & planning a top priority
The single biggest reason projects fail is a lack of actionable requirements.

Right now, it’s likely a huge number of your requirement tickets only consist of a short summary sentence in the title. Or maybe someone has attached a monolithic PowerPoint presentation to a single ticket, documenting new features across multiple sprints. Perhaps instead you’ve got a few thousand auto-generated defects from an accessibility audit that was completed two years ago? It doesn’t matter. Regardless of what format they take, inconsistent requirements force developers to rely heavily on their own subjective interpretations of what to build. And this eats into their productivity, and significantly decreases your chances of success.

How do I get it right?
To be successful, requirements need to be clearactionable, and testable. This can help you avoid:

  1. Developers building something other than what was intended,
  2. time wasted on meetings to align on requirements,
  3. dramatically slower production times,
  4. missed schedules due to having to rebuild during UAT, and
  5. an increase in problems making it through to your customers in production.

To reduce risk, keep your requirements as minimalistic as possible and build towards MVP. And at an absolute minimum, ensure you have a user story, acceptance criteria and a testing plan to keep you on track.

The goal of requirements should be to have enough information in every ticket for any developer to pick it up and run with it, without having to spend time deciphering email threads or setting up meetings to understand the ask. Having a good set of requirements templates will help, along with having a unified project-wide guide documenting workflows, so everyone involved knows how your process works.

Refine requirements before you build. All new requirements should be reviewed by your architects, developers (ideally the same ones who will be doing the work), and your quality assurance teams. Make sure to allocate time for them to ask questions and give their feedback. It’s also important for someone from your QA team to be engaged during planning to help create testing plans, write test scripts and identify test data sets needed. Managers and sales teams simply passing on requirements to developers without making themselves available for discussion, or encouraging developers to suggest improvements is a horribly outdated way to work (and almost guaranteed to end in failure).

Create a unified source of truth for your requirements. To achieve company-wide alignment on what “done” means, and the process for how to get there, make sure every team touching the code works with it in the same way. Far too often “build squads” and “maintenance squads” each have their own way of doing things, their own tools for requirements or defect tracking or their own deployment work-flows. This inconsistency will bring chaos to your project.

Get everyone on the same page. Consider giving everyone in your organization access and training on your requirements planning system. Insights can come from anywhere, and keeping systems transparent and open can help you get important information faster. When the right people have the means to communicate directly, it removes the unnecessary time and risk associated with a complicated chain of command.

Create consistent issue templates. Once everyone is trained on the process and tools (make sure to build this into your on-boarding process!) you can create consistent issue templates. Start simple, with “new feature”, “enhancement”, and “defect”. It’s likely some issues won’t have every single necessary detail, so developers can be encouraged to enhance issues with their assumptions before coding. However, make sure everyone knows your “getting to done” process, and make sure it’s culturally acceptable in your organization for people to send tickets that don’t meet standards back to the creator for more details.

Always ask for feedback. Every team will have slightly different needs and requirements, so it’s important to get feedback on your templates. Make sure you’re documenting what everyone needs, and if your software has to go through accessibility testing, security testing or any other compliance testing, make sure those requirements are stated up front so everyone can build with those standards in mind. Likewise, make sure your coding standards, approved licenses and code review processes are publicized and kept up to date.

Now that you have a set of requirements templates that your team is happy with, set up a process for the development lifecycle. We’re all familiar with the kanban methodology columns – “To-Do”, “In Progress”, “In Review” and “Done” – so consider adding a “Planning” column to ahead of this and mandating that nothing moves into “To-Do” until it’s been reviewed and everyone on the team is happy with it. This will keep your “In Progress” tickets out of the “To Do” pipeline and let everyone know that the to-do list is actually ready to be worked on.

Automate provisioning, builds, and data generation to automate testing
Continuous delivery requires automation as you move code between environments. It’s important to work out a plan with your DevOps team to ensure environments can be automatically provisioned and code deployed to them for testing. While the industry on the whole is improving, far too many enterprise-grade companies still have a manual build process – and while being able to make “daily builds” was great 20 years ago, the modern standard should be more aligned with, “Can you create a build on-demand, with one command?”

These builds don’t just require an environment and code. They also need to be tested at every step, to help you catch problems early and at the source. Testing requires access to quality test data, and far too often, tests are run on small data sets that only test perfect-path results.

Expect the unexpected
Perfect path testing leaves a lot to chance. What happens when an API you depend on fails? If you don’t know, or didn’t build in the right fallback or user messaging, you’re going to have unhappy customers. So make sure you have a process for creating comprehensive and realistic test data, or pulling sanitized data from production that you can use as part of testing. Having realistic tests, imperfect-path tests, or even some chaos testing, can really help you at this stage.

Also bear in mind that modern systems are complicated and often interconnected. We are not just building one stand-alone application; we are building something that relies on numerous back-end systems and 3rd party integrations, which may or may not come with a sandbox mode for testing. This means that in addition to “internal” test data, you may also need simulated virtual or “mock” services for those times you can’t test against production or a 3rd party service.

By introducing mock services, you can write unit tests that perform negative testing or chaos testing, ensuring you know how your code will perform under a variety of real-world circumstances. Sounds complicated? It’s not if you use the right tools. And if your organization is struggling with unit test coverage, focus on API-endpoint testing as a starting point.

Reduce, reuse, recycle
While automation can help, another thing your team can do is optimize and reuse code in order to reduce overall workload. This decreases the number of tests required and the effort needed to fix bugs and make enhancements.

Look to common areas – like user sign in pages or managing profile forms – and ensure you’re reusing validation and error handling. This will help give a consistent user experience, as well as simplify the amount of code you have to support. Given that 70% of testing is still manual, small steps like this can help free up your QA engineers, giving them more time to work on automation tasks.

Why automate?
Allocating time to create automated tests should be high priority. Once you create a test, you only have to touch it again if the underlying code it’s testing has changed. Therefore, it is endlessly reusable with a tremendous ROI. The effort savings will typically pay off within one sprint – imagine the savings you will get over a year, or the lifespan of that code.

Automation is not just about time savings; it also significantly increases quality. Even a simple dependency update can have a knock-on effect on other parts of the system, with potentially sweeping ramifications. The more you automate, the easier it is to do full-system tests to ensure your “simple change” has the intended outcome (with no unwelcome extras).

It gets even better – many modern tools allow you to record front-end UI tests, and replay them, meaning with the right tools it’s a zero-cost exercise to build the infinitely reusable automated tests vs. just doing manual testing one time. That’s a no brainer.

Work smarter, not harder
Assuming you have clean requirements, another huge win is the ability to automatically generate your test cases. Manually building and managing test cases is a massive time sink – and therefore it seldom gets done thoroughly. However, with the right tools and requirements, you can shrink the effort down a size or two.

During your automation transition, you’ll likely experiment with a number of tools, and the cost of open-source tools makes them appealing. But when you’re ready, look for enterprise tools that will enable you to import any tests you created. Being able to reuse things you’ve already got on hand will make it much easier to keep momentum.

Don’t feel like building everything yourself from scratch? Many frameworks and platforms have great tools for sanity checking their own best practices or a predefined set of functional tests. While these tools won’t always be tailored for your specific needs, some test coverage is better than none. For web projects, Lighthouse is a great tool that can be easily be added to your build process to check web pages and report on general performance, security, or usability issues.

Bring tools to developers
During development, the cost of fixing a bug is dramatically cheaper than trying to fix it once it goes into production. A stitch in time saves nine (or in this case 1000 – yep, by some accounts it can cost 1,000 times more). Recent failures by Boeing and Samsung highlight the damage a lack of proper testing can have on your company’s reputation.

It’s not hard to see how these sorts of issues happen. With the old way of doing things in linear siloed phases, testing wouldn’t happen until developers were finished. Given that testing was typically time-boxed, it was rare all the tests would run; then on top of that, the resulting data may not make it back to developers until the following sprint. These delays introduce a high level of risk, with plenty of time for the original developer to forget context or for the project to have been handed over completely to somebody with no prior knowledge of the requirements.

Instead, a quick win is finding tools that easily integrate with your developers IDEs, or can be run from the command line or in code. By integrating all the tools together, you remove the hurdle of multiple systems and environment variables, making it easy for developers to find bugs in their own code and immediately fix them. To reap these benefits however, it’s important that everyone writing code has access to the testing tools, knows how to use them, and will use them.

Build a collaborative culture of rapid feedback
The goal of continuous testing is to create rapid feedback loops, but that’s not limited to code. Since software development does not end at launch, it is important to cultivate a culture of continuous feedback; and opening your development stand-ups to anyone who has work slated that sprint will ensure your team is better able to get the answers they need. Having cross-functional retrospectives will also help ensure your continuous testing initiatives are working for everyone, and will give you the opportunity to fine tune as you go. If you can’t find a time for everyone to attend, offer the ability to submit feedback via a “suggestions box” (you can easily set this up with a Google Form).

Remember, retrospectives are not just for calling out things that need to be improved – they’re also a great opportunity to call out the people who are doing things right. Teamwork is key, and nothing builds rapport like gratitude. Things will not always run smoothly, but one key metric to follow is how fast you can address issues that are causing pain – and the easiest way to do this is to capture the mood during a retrospective.

Adjusting to the new normal
Even with support from all stakeholders, simply mandating that everyone switch to use continuous testing methodology without training and time will cause your initiatives to fail. Continuous testing cannot be mandated to just one team, or by one team. It is a fundamental commitment to quality that must be embraced by everyone in the organization. Your teams will need time to adjust, adapt to new processes, learn new tools, and ask questions of one another – and all of this takes time.

Since most existing automated tests and testing data tends to reside with quality assurance teams, it is great to lean on them to champion change and share their testing knowledge across your organization. Quality assurance testers and engineers will also have a lot to offer and, just as we have agile coaches, can function as continuous testing coaches throughout the organization.

With all the required training, you may see an initial slowdown of productivity during the transition. It is important not to panic. A lot of things contribute to story point decline, but it’s important to let devs track non-development tasks – like meetings, training sessions, and retrospectives – to the story point count so you are tracking an accurate picture of how much the team is getting done. Change will happen fast, but it will not happen overnight. In the meantime, it’s important for everyone to stay flexible.

Conclusion
The best way to ensure your continuous testing initiatives are successful is for your organization is to:

  • Be proactive with requirements,
  • work to share knowledge and access across teams,
  • automate as much as you can (and make sure all teams can reuse the work)
  • generate appropriately robust test data, and
  • be kind to your future selves by budgeting time for continuous feedback and improvement.
Ultimately, software development is always going to be a complex process, and it won’t ever be perfect. But by taking these steps, you’re sure to leave everything a lot better than you found it.
To learn more and get started with complete continuous testing today, for free, visit www.BlazeMeter.com.
AIOps: Helping SREs Predict the Future

AIOps: Helping SREs Predict the Future

As a kid I grew up reading a lot of science fiction. My forbearing parents used to let me take out from the library the max number of books each week they would allow (30, I still remember that number). And each week I would go back for more. Given this constant consumption of augury you would think something I read would have prepared me for the future we now face within the Operations space.

While there are definitely some inklings in the science fiction canon about computer systems constructed at such scale that they would be hard for humans to understand, there is precious little attention paid to what it would take to operate them in production. Welcome to my world (and your reality, too, I bet).

At the AIOps Virtual Summit we discussed two separate approaches to handling this level of complexity and how they intersect. The first is the engineering discipline known as Site Reliability Engineering (SRE) which aims to engineer failure out of the system. The second, AIOps, is a newly coined term for the application of a class of advanced algorithms to the massive corpus of operational data we are now accumulating just as part of the ordinary day-to-day activity of running all of these systems and services.

One goal of the former is to construct a set of operational practices that allow us to navigate the tricky path between a desired feature velocity (iterating the software as fast as possible to provide the features a business needs to provide to its customer base) and a desired level of operational stability (keeping the system available for those customers). This is trickier than it sounds for at least three reasons:

  1. There are often completely different sets of people working on these problems.
  2. They have very different incentives around the work.
  3. Communication between these groups is often, shall we say, a little dicey.

SRE, like many other engineering disciplines, is a data-driven approach. It uses data (in ways we’ll talk about in the upcoming session) to help create productive conversations and decision making easier between these different groups.

AIOps similarly tries to use operational data to provide a big win for an organization. It attempts to address the hard problem of “we have all of this data on the operational status and performance of our infrastructure, what can we learn from it?”

Can the record of the past help us understand how things are working in the present or even help predict the future? Is there information in the data I have already that might provide some insight into how my systems are behaving? For example:

  • Is this just a spike in traffic or an indication my systems are about to experience a tailspin into failure?
  • Are there any difficult-to-see patterns in the load in my system that could help me optimally provision my resources so I don’t pay more than I need to?
  • Have we ever seen a outage like the one we are experiencing? (and how did we deal with it last time?)

Some of this is real today, some of it is easily imagined. There are definitely limits on what AIOps can offer our operations practices, but we surely haven’t taken it to its full potential yet.

Watch a replay of myself and Todd, Palino, who’s a senior SRE at LinkedIn discuss How AI is Helping Site Reliability Engineers Automate Incident Response. We discuss both approaches and their potential to bring a little bit of the future into your present.

Automation.ai as your trusted guide

What if you could empower SREs with the insights needed to drive improvements? What if instead of the typical war rooms and on-call burn out, SREs had a trusted guide to quickly fix problems?

Broadcom provides a “just add water” approach that can help your IT teams automate incident response through our AI-driven, self healing platform Automation.ai. Leveraging our deep domain expertise, we can help your SRE teams prevent alert fatigue by triaging alerting rules continuously using a combination of notification rules, process changes, dashboards and machine learning (ML) to proactively monitor the SRE four golden signals and measure what really matters for customer experience.

Site Reliability Engineering Is a Kind of Magic

Site Reliability Engineering Is a Kind of Magic

A site reliability engineer (SRE) can be considered the IT equivalent of a wizard, or as Andrew Widdowson, an SRE at Google, described it “Like being part of the world’s most intense pit crew… changing the tires of a race car as it’s going 100 mph.”

So how is a site reliability engineer (SRE) different from traditional IT operations, and can a discipline originating from the world of web-scale, cloud-native unicorns ever apply to steady as she goes state of Enterprise IT?

Yes, it can. The scale out way is really the new way of managing enterprise IT. The notion that Enterprise IT exists behind closed walls doesn’t exist anymore. Now, the only way to create and conduct business at scale is through engineering reliability managed in an unprecedented manner. The demand for mobile experiences and the advent of complex cloud architectures has shifted the operational focus. It’s no longer about keeping the lights on. It’s instead about performance. The apps have to work well, the experience great and the infrastructure behind it needs continual monitoring.

Reliability like any feature isn’t something that’s retrofitted after deployment; it’s established and enhanced as software is developed, tested and released. That means establishing a new discipline, which Ben Treynor — Google’s original SRE lead — describes as “what happens when a software engineer is tasked with what used to be called operations.”

A Sobering Reality

It’s easy to throw out yet another three-letter acronym and claim it’s a magical elixir for all the problems involved with running complex IT systems. In reality, engineering reliability into distributed systems with thousands of containerized applications and microservices is a tough gig. Not least because of all the moving parts, but also because any preconceived notions about predictable system behavior no longer apply.

Take for example keeping watch over a modern software application. This might consist of business logic written in polyglot languages and linked to the legacy ERP system (custom built or packaged or both). There’ll also be a raft of databases — traditional relational for transactional support, yes, but more likely a smorgasbord of NoSQL data stores — be that in-memory, graphing or document — perhaps fronted by recently adopted Node.js.

Some of this componentry will be on-premise, some will be containerized and moved to the public cloud — that might mean Docker and Kubernetes on AWS, but maybe Azure and Mesos — heck, why not both for some hybrid-style resilience?

But like the old Monty Python sketch, “you’ll be lucky” if this is all you ever have to manage. Depending on the nature of the business, there’ll also be a glut of third-party services — including payment processing and reconciliation. That’s not to mention all the new web and mobile apps interacting with the core business systems through an API gateway and possibly some analytics horsepower delivered by the likes of Hadoop and Elasticsearch. It’ll take a lot of operational wizardry to keep all that performant.

Fortune Favors the Bold

In a wonderful talk at SREcon, Julia Evans from Stripe described the realities of managing today’s complex distributed systems. What was refreshing about her presentation was the open admission that she often finds the work difficult and how there’s always a ton of new stuff to learn. As she says in her abstract, she doesn’t always feel like a wizard (echoing the protestations of Harry Potter).

This honesty illustrates what’s exciting about being an SRE. With systems like the ones described above causing any number of thorny problems, it’ll be the inquisitive and brave that keep business on track. Being an SRE isn’t for the faint-hearted or those happy with a fire-fighting status-quo. It’s for those within our ranks who get bored easily — those super sleuths who keep asking reliability questions, crafting improvements — and learning as they go.

So, if we consider a typical business-critical problem that could impact our modern application — let’s say some latency issue is causing an increasing number of mobile app users to abandon a booking service? How would teams address the issue? Problems like this might go unnoticed for some time, or there could be a deluge of alarms. Even when a problem is identified, where do teams find the root-cause? Is it a problem with a new code release or at the API gateway? Is it down to some weird microservices auto-scaling issue and was that earlier CPU increase we thought was OK but actually was really bad?

With an SRE-style approach, business critical problems are never addressed in knee-jerk fashion. Using modern tooling in areas such as AIOps and application performance monitoring, SREs can observe the real-time behavior of applications, with systems collecting and correlating information from all related components. Rather than react after the fact, these solutions continuously identify anomalous patterns (like those mobile app abandonments) and compare them to historical trends — meaning SREs are alerted well before the business is impacted.

But beyond exposing new normal application weirdness and “unknown-unknowns,” modern tools also encourage and stimulate more of the SRE detective work — the real valuable stuff. These tools won’t just detect anomalies and then leave teams scrambling to find the needle in a haystack of needles. Instead, they’ll analytically gather all the evidence and lead teams in fact-based fashion towards a solution. Like for example, using an SRE inspired monitoring service to detect a performance anomaly introduced with a new software build and then tracing to the actual code causing the problem.

Like Harry Potter, operations professionals might have a hard time accepting they’re wizards. But ask yourself this — do you want to remain a silly muggle getting burnt out by constant fire-fighting? Of course not, it’s career limiting and sucks. Time then for some SRE magic — gaining the skills and tools needed to adopt new tech like containers and microservices — becoming an essential part of future-proofing your business.

Automation.ai as your trusted guide

What if you could empower SREs with the insights needed to drive improvements? What if instead of the typical war rooms and on-call burn out, SREs had a trusted guide to quickly fix problems?

Broadcom provides a “just add water” approach that can help your IT teams automate incident response through our AI-driven, self healing platform Automation.ai. Leveraging our deep domain expertise, we can help your SRE teams prevent alert fatigue by triaging alerting rules continuously using a combination of notification rules, process changes, dashboards and machine learning (ML) to proactively monitor the SRE four golden signals and measure what really matters for customer experience.