In the modern world of constant change, change is the only constant for every fast-paced business that must provide a compelling and engaging solution to its target market. There are limitations in the waterfall actualization model, such as the inability to effectively meet the timelines and rush that dynamic customer operations often present. But moving to Agile is more than just a method; it’s a new style of thinking about the interactions between systems, processes and technologies in the organization.
In this paper, I present the views of how I participated in migrating a global beauty e-commerce brand from traditional e-commerce architecture to a more integrated approach governed by agile principles. But it wasn’t just about adopting Agile — it was about redesigning processes, integrating systems, and building up processes in order to create an environment that looks and feels seamless.
The Technical Need for Evolution Beyond Waterfall
Traditional e-commerce architectures, CRM systems, and dedicated loyalty applications provide the main stack for e-commerce to deliver digital assets. Even if those systems worked, they were not adaptable to the constant loops of development cycles that are typically present now. It was clear, however, that deploying agile approaches in day-to-day practices did not address the more complex architectural issues arising from this embedded infrastructure.
Some of the critical pain points resulting from this were:
- Disconnected Systems: The management of omnichannel experiences and syncing of customer data was inefficient because of limited interoperability between platforms.
- Bottlenecked Release Pipelines: The rigidity of development processes and vertical dependencies were a major bottleneck in product releases and gave the teams little room to adjust to the changing market.
- Resource Constraints: The technology stack was expensive in terms of resources, with constant manual updates and a lot of resources consumed to maintain uniformity across the environments.
The solution required more than an Agile transformation; it required an engineering reconfiguration to offer the necessary tools for modular systems, CI/CD pipelines, and intra-team real-time collaboration.
Addressing Agile Transformation in a Legacy-Driven Ecosystem
The migration exercise for the client commenced by reviewing their current systems and ongoing projects. This included assessing the attributes of each digital asset and platform and their capabilities to support Agile integration while not interrupting existing project workflows. The solution was to introduce changes in technical improvements simultaneously with the deployment of Agile techniques so as not to disrupt operational processes. We made the following modifications:
1. Modularity in Infrastructure
Legacy systems were transformed and modularized to microservices supported by containerization platforms — Docker and orchestration platforms. This made it possible to perform parallel updating and deploying which also accelerates delivery and minimizes disruptions at the system level while enhancing its scalability and fault tolerance.
2. Data Flow Optimization
APIs and middleware solutions such as MuleSoft and Apache Kafka were described as enabling connections between isolated, disjointed systems. These integrations allowed the sharing of data on the heartbeat and improved omni-station activity by plugging customers’ e-commerce websites, clients’ relationship management systems, or customer data platforms.
3. Integrated Toolchains
For CI/CD processes, tools such as Jenkins and GitLab were configured and utilized to deploy build and release cycles in accordance with the Agile approach. This enabled consistent and reliable deployment pipelines, allowing maximum manual errors and rapid development time frames because more time was saved in the processes.
Solving these basic technical problems allowed organizations to take full advantage of Agile while overcoming the problems associated with the traditional models.
How to Prioritize and Re-Engineer Digital Assets for Agile Integration
In order to assist the client with such a significant change when many projects were already underway, we adopted the Scaled Agile Framework (SAFe) as it helps in integrating Agile practices within very large enterprises. We modified it to enable gradual implementation of Agile processes so that Agile teams targeted the most critical digital assets while continuing with other essential activities. This custom approach enabled us to roll out development more efficiently without interfering with other processes.
The SAFe is a comprehensive methodology designed to help organizations successfully implement Agile practices at an enterprise scale.
1. Developing a Digital Asset and Platform Taxonomy
We began with mapping each digital asset, such as product display pages, recommendation engines, and customer profiles, to the customer journey. This mapping was crucial in identifying which assets had the greatest impact on user experience and business value. By understanding these touchpoints, we enabled the client to strategically prioritize assets for re-engineering, ensuring Agile integration efforts aligned with customer needs and organizational goals.
To create this robust digital product and platform taxonomy, we leveraged insights from established frameworks and research, following four streamlined steps:
- Define Objectives and Scope: Clearly articulated the purpose and boundaries of the taxonomy to align with organizational goals.
- Identify Platform Attributes: Focused on key attributes such as centralization, market sides, and offering orientation to differentiate platforms.
- Develop a Taxonomy Schema: Constructed a structured framework to categorize platforms based on shared features and distinctions.
- Validate and Refine: Applied the taxonomy to existing platforms, refining it for accuracy and practical application.
This taxonomy enabled the creation of pods dedicated to specific functionalities and aligned them with overarching business goals:
- Early Wins: Pods focused on front-end components like ratings, reviews, and loyalty rewards interfaces, delivering quick, high-impact results with minimal complexity. These projects improved user experience and built stakeholder confidence, setting a positive tone for the larger transformation. By demonstrating tangible progress early, the team gained the trust needed to tackle more complex initiatives.
- Complex Integration: Back-end systems, such as the CRM platform and data lake, were tackled later, requiring deep integration to ensure scalability and consistent data across channels. These efforts established the foundation for advanced capabilities like personalized recommendations and real-time analytics. Addressing these core systems ensured the stability and flexibility required to support evolving business needs.
Creation of Digital Assets and Platform Taxonomy revealed that ~30 pods comprising of ~270 full-time employees will be required to enable the front-end products for the e-commerce giant.
2. Enhancing Technical Readiness
To ensure the organization was technically prepared for transformation, we focused on three key areas:
- Cloud Migration: Non-critical workloads were transitioned to cloud-based services such as AWS Lambda for serverless compute tasks. This migration reduced reliance on local infrastructure, optimized resource allocation, and lowered operational costs while providing the scalability needed for dynamic workloads.
- API-Led Connectivity: Legacy databases were modernized by exposing them via APIs, using tools like Postman for testing and AWS API Gateway for deployment. This approach enabled seamless integration into Agile workflows without requiring a full system overhaul, offering a cost-effective and incremental path to modernization.
- Scalability: Through Infrastructure-as-Code (IaC), infrastructure was codified using tools like Terraform and AWS CloudFormation. This approach ensured consistent environments across development, testing, and production, enabling automated provisioning, version control, and repeatable deployments to enhance efficiency and scalability.
3. Building Multi-Disciplinary Agile Pods
To drive the Agile transformation, we adopted and customized the Spotify Model to create specialized Agile pods, designed not just to be cross-functional but also aligned with the technical and business scope of the transformation, with a strong focus on mobile and web-focused digital products. Each pod archetype addressed domain-specific complexities while embedding Agile principles across the entire product lifecycle — from design and development to testing and deployment of various digital assets and platforms for e-commerce. By customizing the Spotify Model, we established autonomous pods operating as squads while integrating a tribe structure to promote cross-pod collaboration.
This approach fostered alignment across customer-facing platforms, backend systems, and testing pipelines. It also allowed us to accelerate delivery timelines and drive consistent value creation, ensuring the Agile transformation aligned with the client’s strategic goals.
In addition to their technical specialization, each pod archetype was tailored to the client’s operational needs and designed to include a balanced mix of roles spanning product, digital, and technical domains. This composition ensured that every pod could address the unique challenges of its domain while contributing to the broader transformation objectives.
- Product Roles: Product Owner, Business Analyst, Domain SME
- Digital Roles: Scrum Master, Customer Experience Specialist, UI Designer, UX Designer
- Technical Roles: Tech Lead, Software Developer, Platform Engineer, Solution Architect, Testing/QA Engineer
Agile Pod Type |
Focus area and tools used |
Example Agile roles |
---|---|---|
Testing Pods |
Ensuring quality and seamless delivery by focusing on comprehensive testing and validation, Testing Pods play a critical role in Agile transformations. Their responsibilities include developing automated test cases, maintaining test pipelines, and ensuring CI/CD workflows are efficient and reliable. These pods use tools like Selenium, JUnit, and Postman to automate and manage testing processes, while frameworks such as Test-Driven Development (TDD) and Behavior-Driven Development (BDD) guide their approach to creating robust and effective testing strategies. |
|
Digital Assets Pods |
Driving innovation in customer-facing products such as websites and promotional systems. Focused on enhancing user experience, implementing marketing campaigns, and delivering dynamic content platforms. These pods are built after the concept of Digital Product Pods, which emphasizes cross-functional collaboration to deliver impactful digital solutions and enable rapid prototyping and iterative improvements. Their work often includes leveraging user data to personalize experiences, optimizing platforms for scalability, and ensuring seamless integration with other business systems. This approach not only accelerates time-to-market for digital initiatives but also drives meaningful engagement and measurable business outcomes. |
|
Platform Pods |
Supporting scalable infrastructure by managing backend systems, cloud platforms, and data pipelines. Ensures system reliability, high availability, and performance optimization. We implemented the Infrastructure as Code with Agile principles to create these pods, automating infrastructure management and enabling dynamic scalability across enterprise platforms. These pods develop and maintain key platforms such as Kubernetes-based container orchestration systems, cloud storage solutions like Amazon S3, data streaming tools like Apache Kafka, and CI/CD pipelines powered by Jenkins or GitLab. By providing a robust foundation, these pods enable seamless operations and empower other teams to innovate without infrastructure bottlenecks, driving enterprise-wide digital transformation. |
|
The Digital Assets Pod included experts in product design and development who worked collaboratively to deliver intuitive and scalable customer-facing platforms. Meanwhile, the Platform Pod supported the infrastructure needs of these systems, enabling seamless performance under dynamic business conditions.
4. Bridging Resource and Capability Gaps for Transformation
Role Distribution and Expertise
To address resource gaps effectively, we conducted a detailed assessment of role distribution and expertise for the client, ensuring alignment with their strategic objectives. Drawing insights from industry best practices in resource planning and team structuring, we recommended retaining critical roles such as Solution Architects and Domain Subject Matter Experts (SMEs) in-house to safeguard long-term innovation and ensure continuity of institutional knowledge. For mid-tier technical roles like Software Developers, a selective insourcing approach was adopted, prioritizing project-specific needs to ensure expertise in key areas of development.
To optimize resource allocation further, we advised outsourcing standardized tasks such as Testing and QA engineering, leveraging external vendors to handle repetitive but essential work efficiently. This balanced approach drew upon frameworks outlined in resources like the PMI Talent Triangle and Agile team structuring principles, enabling the client to maintain agility while addressing resource constraints strategically.
Upskilling for Agility
To build agility within the organization, we focused on upskilling internal teams with targeted training on Agile principles, CI/CD tools, and API integrations. Certification programs were introduced to enhance technical capabilities, including container orchestration with the Certified Kubernetes Administrator (CKA) and cloud architecture skills to align with the demands of the transformation. Additionally, team members pursued Agile-focused certifications such as the PMI Agile Certified Practitioner (PMI-ACP) and Certified ScrumMaster (CSM) to deepen their understanding of Agile methodologies and enhance their ability to drive iterative, value-driven development.
Conclusion: A Technical Blueprint for Digital Transformation
This experience highlights the critical need to address both methodological and technical challenges in Agile transformations. For the global beauty e-commerce client, success depended on rethinking workflows and reengineering the foundational systems that support them. By implementing modular architectures, automating release processes, and aligning cross-functional teams, the transformation delivered faster product rollouts, improved customer experiences, and a scalable technology foundation for future growth.
Organizations embarking on similar journeys should treat Agile transformation as a holistic effort, balancing immediate wins with long-term investments in technical infrastructure. For insights into managing complex transitions, Large-Scale Scrum: More with LeSS by Craig Larman and Bas Vodde provides practical strategies for scaling Agile effectively while addressing organizational challenges.
Opinions expressed by DZone contributors are their own.