What is Refactoring? How does it affect the Product?
Refactoring is nothing but modifying the internal structure of the code without altering its external behavior. I’ve seen multiple projects where developers storm with full enthusiasm to develop features as soon as they see what is needed. Moreover, when the team size increases the effects of less maintained code gets compounded as more and more developers starts committing the code changes. As you can imagine, when technical debts loom large, accommodating features becomes extremely challenging. You will start to have unhappy stakeholders as you will not be able to move faster as the technical debt starts pulling you down. If you are not making effort to curtail the technical debts, you would have to pay down doubly in the longer run. I would say, paying down technical debts in one of the easiest ways of improving the quality of the code.
Should it be a Standard Process?
Wouldn’t it be easy to add refactoring as a part of definition of done. This will assist developers to have a checkpoint on how much code has been committed and cleaned up for every iteration. You could also use a metric like number of files checked in vs number of files refactored. Refactoring is always a collective effort mimicking the collective code ownership from eXtreme programming. This way each and every individual gets to know what is happening around each and every feature technically. Refactoring as a process can provide comfort to developers in improving the throughput as the tests gets executed faster without any side effects. Imagine your house is untidy and would it look good in adding decorative items or furniture’s without cleaning it in the first place? Just like how you clean your house to look great, it is also an art of cleaning up your code base habitually to enhance the product outlook technically.
The art of software craftsmanship lies in the practicing the principles and methods tirelessly for longer time. Gaining the knowledge and skill to develop cleaner code alone would not entail success and makes your product (render your service) better but sweating over it would. You’d need to practice it tirelessly over and over (fail of course couple of times) to learn art of refactoring or writing better piece of code. Establishing refactoring as a process, would certainly make all developers follow it meticulously without bypassing it.
Should it be an item on the Product Backlog or Stretch Goal?
On the other hand, as a Scrum Master or Project Manager, you could also negotiate with product owner or stakeholders to underline the importance of cleaning up the code base for better stability. This can be done in several ways and can be left to the team on how to allocate time and effort for refactoring every iteration. Considering an ideal team of 9 members and 2-week iteration length, the team can allocate 5-7% of the team’s capacity or M size story could potentially added every iteration for refactoring effort. But this could also lead to conflicts between the product owner / stakeholders and the development team due to prioritization challenges. Say for e.g. Marketing team would say “I need these features to go in next sprint and is non-negotiable as it would help the company in reorganizing the portfolio to serve customers better”.
This can be achieved based on mutual agreement between both the parties and the effort should come down slowly as the team matures and learning curve flattens. The team can also think of different approaches to solve the problem either through pair programming, peer code reviews etc. This can help the team weed out some of the technical debts every now and then. All of the activities carried out for the refactoring effort should be factored in the story size accordingly. Alternatively, team can also pick up the refactoring effort as a stretch goal and accomplish within the iteration. This can help avoid the friction between the development team and stakeholders. The team will uncover many things during the iteration and would probably spend extra time and energy to accomplish what has been committed for the iteration. These changes can make the team lose focus on the refactoring and eventually the purpose of cleaning up the code base becomes defeated. The stretch goals make the team feel like crushing the weight instead of a lofty aspiration in these circumstances.
Should it be a Shutsuke (Self-Discipline)?
Having explored the above methods, let’s also look at making the refactoring a daily habit to the developers through constant coaching and some development aids. Every team will go through Tuckman model of Forming, Storming, Norming and Performing stages in each and every project. Initially the team may not have the time and energy to focus on cleaning up as they will have a lot to deal with in terms of design thinking, user experience, demos etc. But it is also equally important to seed in the thought of making the refactoring also a habit to each and every development team member. It gives clear advantage to ingrain these thoughts early in the development stages along with other activities. Most of the team will never think about refactoring as a primary responsibility and always give step motherly treatment.
There are also lots of development tools (open source) that can aid developers in reducing the technical debts. It can quickly throw a report on how many static code violations are present categorized by critical, major and minor. The team can address these violations as a part of each iteration to accomplish the goal of cleaner code. Manual code review has become a thing of past and there are tools as well that can help annotate code from any code repositories and provide comments on the modularity, quality, reusability as well. Technical leads can use these tools that can come handy to provide their review comments to the developers. Scrum Master can also provide coaching to the development team and reiterate on the importance of cleaner code for product quality. Like I said earlier, the development team should practice hard on these engineering practices to become better at software craftsmanship.
Refactoring is certainly one of the critical engineering practices that can help pay down technical debts earlier in the game. It is often said that the sum of good design and development effort is equal to poor design, development and operations effort combined. It is imperative that developers understand the impact of refactoring early stages of the project and becomes a habit in their daily routine. As mentioned earlier, refactoring is a collective effort by the team that can be accomplished through the methods suggested above. It is always left to the best interest of the team to see how it can he accomplished depending on the nature of the team and project. But it is always a healthy practice to clean up the code then and there instead of someone helping to formalize the process of tidying up. It is said that your daily routine becomes your habit and eventually your behavior.
“Everything that we have electrified, we are now going to cognify. And I would suggest that the formula for the next 10,000 startups be very very simple: take X – and add AI” – Kevin Kelly.
Technology has picked up steam lately and is aggressively pushing borders beyond the realms of human imagination. Artificial intelligence is a much broader term which embraces automation (rule, intelligent), predictive analytics, sophisticated algorithms with heuristics and deep learning capabilities etc. Organizations that were in existence even before the pre-internet era like GE, major automakers including GM, Toyota etc. have streamlined their processes (take for ex. assembly line) manned by labors for operational efficiencies. The project managers also have executed the projects for a long time with these standard practices and methods. There were also lots of manual processes (repetitive) at enterprise level to fulfill the application needs (for ex. invoice, claims and tax processing if you take Finance domain). The applications that were developed in those days were also on legacy systems, too tightly coupled and made it difficult to remodel or keep pace with the evolution. With limited set of tools, it was virtually impossible for the project managers to derive insights from massive amount of data. The dawn of world wide web and access to internet coupled with mobile explosion defined the beginning of internet era or the inflection point. The organizations that were so lucky to be born in the early internet era (Amazon, Netflix) developed their business models around customers and data. The advent of open source tools and cloud further accelerated the development by slashing the costs and also freed the developers from typical top down approach. What important role does artificial intelligence play today especially in the area of project management?
Role of AI in Project Management Profession
AI and its application has wide range of benefits across all sectors. There is no doubt that the technology power behind the AI will overtake us and the fear of losing the battle is completely plausible by some of the brightest minds in the industry. It would never be a great surprise, if AI sneaks into your cube and potentially replace you from doing mundane work. Standish group’s chaos report claims that 52.7% of projects cost 189% of their original estimates.
PMI’s Talent Triangle
Technical Project Management, Strategic & Business Management and Leadership skills are three sides of talent triangle defined by PMI ®. Artificial Intelligent can aid project managers in all of these three sides for better efficiencies.
Artificial intelligent with digital management assistants like bots and various algorithms can play role in providing project or program level insights and recommendation to manage the activities. Project manager’s effort towards operational activities would be significantly reduced.
Business and Strategic
Project Managers also need to be visionaries and think strategically as the industry is changing rapidly based on the customer needs. Project managers should align themselves with the values that they can bring to the organization through the project success. Artificial Intelligence can greatly help project managers in predicting market trends and provide various business models depending on the needs of the project.
Leadership skills are critical to project managers when managing stakeholders in the project as well as motivating the team members. These skills have profound influence and impact on the project outcomes and can be supplemented with AI tools for maximizing benefits.
Managing a project well takes more than just making astounding plan in advance and running through to it. Interdependencies within the project and external changes could make outcomes erratic. Estimates and forecasts are at best perception; at worst, guestimates. Modern management techniques such as agile and continuous delivery target to reduce unlikelihood by working incrementally, but that still doesn’t promise final delivery. Project managers now have machine learning algorithms at their disposal that can help in predicting the outcomes of projects using data the projects already have, such as the planned start and end dates of various phases of the project to learn the completion rate of the team and predict the likelihood of schedule delay or completion. The fine-tuned and tailored algorithms can be integrated to any application life cycle management tools to predict confidence of task estimates and the likelihood of completion.
Artificial intelligence can help project managers greatly in maximizing the value to the clients by optimizing the effort there by maximizing project success. It can also help the team members in freeing up from repetitive operational (mundane) tasks. With the help of AI tools project managers would be able to work towards accomplishing project goals, improve quality, optimize cost, align seamlessly with multiple stakeholders and their expectations. As Elon Musk rightly put “Humans must merge with machines or become irrelevant in the AI Age”.
It took more than sixty years for artificial intelligence to gain popularity and make itself omnipresent, but now that it’s here, it’s here to stay. Gartner also predicts that by 2020, artificial intelligence will actually create more jobs than it eliminates. “Cognification” will lead to change of job roles, rather than their eradication. It’s also a curious thought to see what AI can bring about in upcoming decades and what it will leave humans with. Project management is one of the professions that has dramatically improvised over years. Project management has become highly efficient with the usage of tools and technology and has no doubt that the workplaces have become quite productive. It’s high time that project managers replace their day to day manual project management tactics and techniques with smarter ones driven by technology and intelligence.
Scrum no doubt has become the popular framework for software product development. Scrum for software development was modelled after "The New New Product Development Game" by Hirotaka Takeuchi and Ikujiro Nonaka published in the Harvard Business Review in 1986. They beleived that the cross-functional teams engaging in the dynamic conflict of ideas that generates "ba," the energy flow that surfaces knowledge that forms new products.
Dr. Jeff Sutherland is one of the inventors of the Scrum software development process. Together with Ken Schwaber, he created Scrum as a formal process at OOPSLA'95. The Scrum Guide was published first in 2010 by Jeff and Ken and further editions of the guide clarified many of the practices and guidelines. In due course of time, people tried to take advantage on the popularity of the scrum framework and have created their own versions. This has led to lots of confusions in the minds of many professionals across industries.
Common Myths about Scrum
Some common myths about Scrum are,
Let’s explore the some of the myths in detail below,
Scrum and Agile are same
Scrum is one of the most popular frameworks and more than 80% of the projects use scrum due to its simplicity and proven results. While Scrum is one of the widely adopted agile methodologies to develop software products, people often think that Scrum and Agile are same. While Agile is a bigger umbrella under which there are multiple methodologies like Scrum, XP, DSDM etc. for software development, Scrum is one of those frameworks.
So Scrum and agile are not synonymous.
Requirements should be written only in the form of User Stories
User stories are powerful constructs expressed from end user perspective for identifying requirements for a software product. The user stories have to be independent and modular so it can be developed and released (potentially) without much dependencies by the development team. User stories are part of eXtreme Programming practice and were adopted in Scrum when people tried to combine Scrum and XP together to realize greater benefits. There is no mandate by Scrum Guide that requirements should be written only in user stories. As long as the requirements are well understood by all the stakeholders, it can be written in any form.
So requirements need not be written in the form of user stories in Scrum.
Only User Stories should be part of Product Backlog
Another common misconception is, only User Stories should be a part of Product Backlog. Product Backlog is nothing but list of anything and everything in order to deliver the product vision. The product backlog could contain defects, nonfunctional requirements like performance improvement features, code refactoring, infrastructure setup etc. There would be often prioritization challenges with the business stakeholders between functional requirements versus rest due to various reasons. It is also the responsibility of the development team to appraise the product owner on the importance of non-functional features which could lead to greater stability of product before adding more features.
So items other than user stories / functional requirements can also be present in the product backlog.
Scrum Masters should have technical knowledge
Now-a-days, it has become imperative that Scrum Master should also have technical knowledge due to the infusion of technology and tools to a great extent in the project. There are lot of job postings which lists the need for Technical Scrum Masters. Scrum Masters are servant leaders and enables the development team to become self-sufficient. Knowing the technical details would actually make scrum master suggest solutions, adopt certain methodology based on his/her prior experience and start managing technical tasks of the team members which beats the very definition of the scrum master. Rather scrum master is a team coach, a facilitator, an impediment remover. Scrum master should be a process expert rather than a technical expert and aid the team in continuous improvement and innovation.
So Scrum master need not have technical knowledge.
Scrum Master in addition can play Product Owner role
Organizations sometimes make a single person play a role of both Scrum Master and Product Owner. This could potentially lead to lots of confusions with the team as the roles and responsibilities are different between both the roles. It’s actually giving too much power to one person which could potentially derail the entire project. The person will not also have enough bandwidth to focus on the priorities as well as facilitate the team. It also requires the scrum master to acquire the domain skills of the product owner which may or may not be his/her forte. The dual role will dilute the responsibilities (both) and the person will not be able to provide justice to either of the roles.
So, a single person can never play a dual role of Scrum Master & Product Owner
Scrum Master is fully responsible for facilitating Scrum Events
Organizations and projects have begun to believe to an extent that Scrum Master (alias Project Manager) is responsible for facilitating and owning scrum ceremonies. Certainly, Scrum Master can facilitate some of the ceremonies initially to break the ice and bring the team together for cohesiveness but it is ultimately the development team’s responsibility for facilitating all of the ceremonies themselves. Scrum master can help in removing the impediments on the way to make development team effectively run these ceremonies on their own. Scrum Master can coach the team on best practices, provide performance feedback (outsider view), resolve conflicts that arise during these meetings and shield the team from external interferences.
So Scrum Master need not facilitate always the Scrum Events, it is the responsibility of the development team
We’ve seen some of the common myths about Scrum but there are other myths as well about Scrum like burn down charts are part of Scrum, planning poker is part of Scrum etc. While people really hold on to these myths to a larger degree, but failing to understand some of the core principles of Scrum which is inspect and adapt. It is important to understand what practices are mandatory and what are optional for any organization that uses Scrum for product development.
Based on your experience in projects, what do you think are other myths that are followed in Scrum based project/product development?
Organizations across the globe are embracing agility using different methodologies and frameworks to meet customer demands and improve time to market. Scrum is one of the most popular frameworks within which people can solve complex adaptive problems, while productively and creatively delivering products of highest possible value. Scrum was developed to work best within a team size of 5 to 10 members and has to be cross functional to be effective and efficient. It becomes easy for the Scrum team to achieve synergy and resonate with each other if the team is collocated further. The Product Owner, Scrum Master and Development team work in tandem with fewer problems to deal with however and can release most valuable features quickly. The challenges and impediments increases with the addition of every Scrum team. There is also a general myth that Scrum may not be suitable for large scale projects or programs and people recommend or prefer other methodologies like LeSS or SAFe.
Challenges with Scrum in Large Program/Product Development
Can Scrum be used for large programs (involving more than 5-6+ scrum teams) and still address top priorities like fulfilling customer needs, improving time to market and reducing cycle time?
The answer is "YES"
Nevertheless, some of the questions that comes to our mind quickly when we think of scaling delivery using Scrum are below,
Let’s look at how the roles and artifacts are scaled when it comes to large scale product or program development using Scrum
Scrum Master and Development Team
Scrum guide mandates a small and cross functional team of 6-9 members for effective delivery. When the program demands more features to be developed in shorter time frame, one of the easiest way to achieve this to scale the team members to multiple scrum teams. This allows the scrum teams to pick up independent features from the product backlog for quick turnaround. Similarly, the scrum master could also be scaled to take care of the individual teams. However, it is up to the organizational maturity in agile adoption and execution as how many scrum masters are needed for a particular program setup and sets of teams. The scaling of the development team and the scrum masters will certainly help in development of features to meet the vision of the product.
In paper, Product Owner is just one person who is responsible for the product vision and maximizing the ROI. But in real world with complex product development, it’s often a shared effort and ownership. There needs to be an understanding on how to develop the product without hassles and inconsistencies through shared ownership between the Product Owners.
In general, when the product is new (in early stage) and hasn’t reached product-market fit—or is close to achieving it—one product owner will be the best person in charge of the product. This is due to the level of experimentation it requires during this stage and effective decision making is of paramount importance. Having multiple product owners during this stage would dilute and elongate the decision making process leading to ineffectiveness. At this stage, the single PO could help in making decisions quickly without wasting time in the process. When the product growth starts after entering the market stage, more features will be needed and it would become difficult to manage the product with single product owner. In that case, it would be good to have couple of product owners to manage the product. During this maturity stage, changes would be needed less frequently allowing the product owners to prioritize the changes and share the responsibilities easily.
Product Backlog is prioritized list of items or requirements needed to fulfill the product vision. Product Owner owns the product backlog and ensures that the Product Backlog Items are stacked in the order of priority to maximize the return on investment. In the Fig 1.1 shown below, during the initial stages of the product development (entering market fit stage), it is imperative that only one product backlog exists for the teams to work on. The product discovery stage is a separate journey altogether which involves user interviews, impact mapping, what and how to develop and prioritization techniques. When the product evolves over period of time (as mentioned above) or enters maturity, the product backlog can be owned by multiple product owners for managing and prioritizing features effectively.
Sprint Backlog will have prioritized list of features needed for development for that sprint towards achieving the product vision. The prioritized list of items will be picked up by the individual teams for development of the product features during the sprint planning meeting. Individual teams can have sprint planning meeting separately (to save time) along with Product Owner & Scrum Master to prioritize what is needed for the sprint.
One Product Increment is always recommended during the initial stages of the development of the product to ensure the minimum viable features are integrated well. This helps the product owner to quickly validate the assumptions about the product. Having single product increment hashes out all the issues with dependencies and integrations and enables users to validate the functionality end to end.
The Scrum ceremonies like planning, refinement, retrospectives, reviews can be done separately by the individual team along with PO and other stakeholders. You might wonder how the dependencies across the teams will be resolved as there is no common meeting to discuss those? Well, Scrum of Scrum can help the development teams to sort of internal dependencies where the Scrum Masters from individual team’s members represent to discuss about the dependencies and work towards resolving them.
There is also no doubt that Scrum would definitely be an ideal fit when it comes to large scale product development with some considerations listed above. The Scrum Framework is very light weight and has lots of process built within for a great product development. Although other scaling methods exists like SAFe, Less, Scrum@Scale etc. the organization need to carefully consider the options to better suit their culture and the nature of the product being developed.
Projects have been executed for decades for many reasons including customer requirements, technological advancements and compliance requirements etc. The success of the projects has been driven by mostly by conformance to plan for plan driven projects and the value of delivery for agile projects. There are many factors that attribute to the success of the projects and had helped project managers steer projects in right directions, take corrective and preventive actions. Metrics is one of the most important aspects project management which can assess if your existing project or program is doing enough to justify your existence. A metric, by definition, is any type of measurement used to measure some quantifiable component of performance. A metric can be collected through observation, such as delay in days, or number of defects; or the metric can be derived from directly observable quantities, such as defects per “x” lines of code, a cost performance index (CPI), or a schedule performance index (SPI) Metric is also called as an indicator, or a key performance indicator (KPI). We will see how metrics can aid automation projects realize its goals and objectives.
We’ve witnessed so far application or product being built to solve a customers’ problem. Most of the applications including e-commerce are now being based on human centered design or user centric. Today’s systems are simply becoming too big and complex to run completely manually, and working without automation is largely unsustainable for many enterprises across all industries. Moreover, automation enables the ability to adapt and improve the service experience without manual intervention. Engineers and high-skill IT personnel will be tasked with identifying automation targets and variables in each workflow, gathering inputs and outputs, as well as simplifying and troubleshooting automated workflows.
What to Automate?
This is one of the most common inquiries around automation, as well as its primary barrier to implementation. While the answer will be business-specific, a logical starting point for many will be repeatable and mundane processes. Identify tasks within each department—be it marketing, finance, sales, or even legal—that people manually perform on a daily basis, and ask if they could be automated. Frequently, these repetitive tasks consume a considerable amount of time from junior team members; organizations with an accurate and granular look at their service delivery, combined with solid bench marking metrics, will be able to hone in on opportunities for automation.
Managing Automation Projects
Every organization has its own approach for managing automation projects and have come up with assessments and frameworks to deliver benefits to their clients on their automation journey. There are multiple phases associated with the assessment framework and the below framework can help any automation projects reach its goal and measure the maturity level.
The phases are,
This is where the what part of the automation use cases are defined. Risk are identified along with mitigation plans.
The use cases for automation are brainstormed for BOTs development.
Use cases are finalized based on different techniques.
Architecture is finalized along with dependencies with related components.
BOTs are certified, deployed and marked for transition to operations team.
Monitor logs, failures, exceptions and get customer feedback on implementation
ROI, Productivity improvement, FTE reductions details are gathered.
How Metrics can help Automation projects realize its goals?
Measurement is key to successful project management. As the old adage says, “You can't manage what you don't measure.” Collecting and measuring data is at the heart of any worthwhile endeavor. With a New Year’s resolution to lose weight, you may find yourself counting calories or—go you! —watching the pounds drop on the scale. Race car drivers track their miles to the second.
There are generally 6 factors that managers generally measure to create metrics that determine project success:
Automation metrics are no different than traditional project metrics but have few variations on what metrics needs to be tracked to effectively manage the projects and deliver the desired outcomes.
With successful automation workflows, additional customers and services can be covered by the existing team, pushing margins higher while improving the consistency and speed of the service. Ultimately, people can do more with less. The metrics management approach and defined set of metrics may be suitable for a particular automation team and not suitable for another team depending on the nature of the automation, but it has worked out pretty good for me so far.