This month, IAB Canada hosted ‘The Experimentation Edge’, an executive roundtable sponsored by The Globe Media Group. This event gathered a group of thought leaders from some of Canada’s leading agencies and brands to share perspectives on and experiences with how dramatic shifts in the B2B landscape is calling for a culture of experimentation.
Given the magnitude of impact that AI has had and will continue to have on the digital advertising sector, the discussion centered on how AI and ML have been approached by various organizations.
The current state of AI integration
AI has become a focal point in boardrooms, leadership discussions, and casual conversations among employees seeking to boost their productivity.
And while all attendees confirmed that their organizations were embracing AI and experimenting with the technology, only a few have formalized the use of Generative AI with early frameworks to help balance the opportunities with the known risks. The frameworks take several attributes into consideration including privacy, data ethics, and transparency.
“Many organizations are leveraging AI to streamline workflows and reduce low-value work through automation,” says Katherine Scarrow, manager of creative strategy for Globe Content Studio, the content marketing arm of The Globe and Mail.
Some key examples included the media accounting process which has been notoriously onerous for media agencies to manage. Plugging in-house systems into solutions like Prisma and Media Ocean has significantly lightened the load.
Another area of application was integration of AI into dynamic creative optimization (DCO). Leaning on Machine Learning (ML) to inform the building of multiple iterations of ads using training data and assets has become an exciting area of innovation for agencies. One case study cited by Cairns Oneil reported a 30 per cent lift in effectiveness from the use of trained AI vs. creative that had been developed and optimized only by humans. Interestingly, results from another experiment conducted at The Globe Media showed how important the human element was in the actual training process of AI showing a 30 per cent decrease in effectiveness when creative generation was left solely to the technology.
On the planning front, agencies like Cossette and GroupM have deployed tools to help with predictive modeling which has allowed for increased efficiencies and the ability to refocus talent on other more sophisticated areas of the business. Other reported activity in the market included strategic partnerships with AI pioneering giants like IBM Watson. With major investments into the tools being developed, there was a sense that SAAS-based tools and offerings were becoming fundamental differentiators in the space.
When exploring the topic of client sentiment towards the use of AI to deliver media services, the group felt that there has been a great amount of interest and traction given the mainstreaming of generative AI in the past year. AI capabilities are making their way into agency RFPs which has drawn CIOs and CTOs into the decision-making team at the pitch table. While advanced AI offerings are not yet viewed as table stakes, early investments into the technology have paid off. As clients begin to explore the capabilities, those agencies with established solutions and frameworks built to meet the emerging demand have a head start on experience and trained data sets. One notable selling factor that would appeal to brands raised by GroupM, was the new ability for agencies to quickly explore “rabbit holes” with fast feedback loops that were once unfeasible due to time and budget constraints. A perfect segue into a discussion on the Experimentation mindset.
Challenges and considerations
Beyond the hype and alluring possibilities of gen AI, a stark reality emerges – the majority of AI projects end in failure. In fact, according to research from Harvard Business Review, the failure rate is as high as 80 per cent.
When considering the challenges that lie ahead as our industry adopts AI, several themes emerged. There was consensus about the importance of developing frameworks and codes of practice that address areas of privacy, transparency, and ethical use of data. With every new technology, we have learned from experience that regulatory guardrails are both necessary and welcome. However, there was some discussion about the pending legislation in Canada and some concern about the risks of vague guidelines, inconsistencies with other laws and potential over-regulation at a time when we are called to invest in innovation.
More practically, the group agreed that there remain some fundamental challenges on the ground with the implementation of the technology itself. Participants agreed that understanding what information is available and deciding which data will be useful is a major undertaking. Many organizations operate within decentralized data structures which presents major hurdles when it comes to gathering data and normalizing it to make it ready for use. Connecting data streams into one centralized hub is imperative to developing a fulsome solution that is ready to train AI systems.
When exploring the risk associated with testing in live environments Scotiabank raised the importance of tightly controlled tests that are centered on business priorities. The theme of guardrails was expressed by all. In large corporate environments and some agency/client arrangements where experimentation was encouraged without standards and parameters in place, Dentsu expressed that it becomes almost impossible to agree on the results which often impact investment decisions.
The group discussed the inherent fear of replacing creativity and content creation with AI. The Globe suggested that there was a need to distinguish between creativity and efficient creation. The group agreed that there was a need to strike a balance. PHD spoke of guardrails and playbooks that are in place and how important it is to tread lightly and stay within the framework to ensure the limits of outsourcing are being respected. Everyone agreed that the use of AI should be focused on eliminating low value work to save time, drive efficiencies and streamline the value chain.
The Importance of developing a culture of experimentation
All participants agreed that it was important to develop a culture of experimentation and that cultivating an agile or iterative approach — testing hypothesis, adapting strategies, and, most importantly, embracing failure as a stepping stone toward success is a new imperative for leadership. From a talent perspective, the freedom to explore and noodle around was described almost as a new form of currency – one that both attracts and retains talent.
Aber Group raised the point that a healthy culture of experimentation ensures innovation and the long-term sustainability of an organization. Active International agreed adding that a competitive advantage can only be achieved by improving methodologies as they become outdated more quickly with the acceleration of technical advancements and deepening fragmentation of the media landscape. The Globe Media Group added that experimentation leads to the ability to diversify revenue sources and build upon an organization’s core strengths. A culture of experimentation is a culture of entrepreneurialism and growth.
More specifically, when discussing the B2B audience, Dentsu commented on how critical it is to test and understand what lies below the common denominator of a title. Advanced media strategy demands and is capable of, discovering the value of the other more nuanced attributes that exist within an audience cluster. The only way to get to this understanding is through a framework of experimentation and a healthy dose of curiosity.
Looking ahead
The group took the opportunity to share best practices for the development of an Experimentational Culture. Five key themes emerged from the thought leaders’ commentary.
- Creating a learning environment
When looking at the top attributes that help to support a culture of experimentation, the group unanimously agreed that curiosity was at the top of the list. Curiosity drives hypothesis and inspires testing activity. From a B2B standpoint, curiosity shows passion for a brand and an innate drive to solve challenges.
Learn, unlearn and re-Learn – a key part of learning is to unlearn. Having an open mind that is ready to be changed with new information is essential to forward-thinking and progress. Having the plasticity to re-learn is a key competitive advantage in a market that is flushed with data.
- Providing a safe space
Everyone agreed that creating conditions for learning and incentivizing helping promotes experimentation and creativity. “Providing team members with the trust to experiment and play freely, without the worry of negative outcomes, empowers the organization to explore new possibilities,” Scarrow says.
The group discussed how important it is to learn from failures and to recognize that success rarely follows a linear path. A culture that learns from failures is moving forward with the benefits of experience.
- Establishing framework for experiments
Experimentation can feel overwhelming. The Globe Media Group shared valuable insights into the content life cycle and how organizing testing opportunities by stages of the life cycle (planning, production, promotion, and performance) can make them more manageable. When applying tools like AI to the process, it becomes clear which stages can benefit most in terms of driving efficiency and better outcomes.
The design of the experiment itself is critical. Ensuring that all elements of the experiment are mapped out to include: an agreed upon hypothesis to test, a valid control data set and an outcome that aligns with the overarching business needs were all key elements to consider. The group also recommended starting small before scaling as results come in.
- Embracing diversity and inclusion
Discussing the importance of diversity in a successful culture of experimentation was a highlight of the session. When diverse groups come together to collaborate on innovation, barriers get broken. GroupM’s Launch Pad program which attracts and retrains talent from diverse backgrounds to welcome them into the media sector was celebrated as a great example of how diversity inspires new perspectives, ideas, and growth.
Scotiabank described how positive changes to develop an agile culture that is set-up for cross-collaboration within the organization has already made a significant impact on discovering how to innovate against customer touch points. The structural change has positioned the organization well for a culture of experimentation.
Summary
A clear take away from the roundtable discussion was that developing a culture of experimentation is not only top of mind across the media value chain, but also in practice. Whether it’s an agency staying ahead of the curve and preparing for a paradigm shift in service offerings or a brand that is pushing the boundaries to capture and retain market share, embracing experimentation is the way forward.
This article was co-authored by Katherine Scarrow, manager of creative strategy, Globe Content Studio, and Sonia Carreno, president of IAB Canada.
