Agile? Waterfall? - Pick Carefully or Pay the Price!

Managing projects across an enterprise is an art as well as a science. There are several well known paradigms used by organizations across the world - waterfall to agile with variations in-between. I have always told those that have asked that there is not one silver bullet to magically solve the project management puzzle. In fact, my recommendations to people have always been to create a project management paradigm that fits the culture and needs of the organization.

My teams work in an industry that is heavily regulated by state and federal agencies and also must conform to several industry governing entities that set standards within the payments space. Many of the projects that we work on are implementing standard interfaces between the various payment networks, acquirer, issuers and other players within the financial industry space. Implementing these interfaces is a technical exercise with very little to no user interface. You either get the spec right or the money doesn't move. These initiatives really are not setup to be run utilizing agile techniques.

Now, let me shift over to some of the other initiatives we run within the organization. Our mobile and internet applications. As you can probably guess, these applications have significant user interfaces. These project efforts can utilize and benefit from agile techniques that can give our 'user representatives' much quicker access to potential solutions as well as drive increased cooperation between the development, user and quality assurance teams. This is a place where our teams are experimenting with agile techniques as a way to accelerate delivery through the development channel.

One organization utilizing different project management paradigms based on the unique needs of the initiative!

I'm running at this from two different directions at the same time:

1. I'm working with our project management organization to review our overall defined lifecycle to recommend changes that will allow projects that might benefit from being run utilizing agile techniques to take advantage of a slimmed down process.  We still need to figure out how to satisfy the documentation requirements of the various state/federal agencies and industry governing bodies, but overall, they are supportive of our desire to create multiple lanes that can be managed differently based on the overall risk of the effort.
2. I'm encouraging our project managers and team members to challenge the process. When working any individual initiative within the overall portfolio to make recommendations to skip various project artifacts, project steps or entire slices of the overall project lifecycle. The key is to document the decision as to why decisions are being made to skip certain pieces of the lifecycle based on risk and impacts to the overall effort.

This is what works in the enterprise that I currently work for. 

On a regular basis, I have to sit across the table from auditors and prove to them that all of the appropriate documentation has been generated. That there is traceability though the project effort from requirements, through design and testing. I have to show them our implementation and fallback plans as well as prove that those plans were followed during actual implementations. Our auditors will randomly pick projects and then go through all of the documentation and measure it against the documented lifecycle to ensure that it's all there. If something is missing, there has to be proof - either through project meeting notes or via change requests - that show the decision was made to skip the document/project step(s). This proof also needs to discuss the risk to the product, the enterprise, our financial institutions, acquirers and/or processors. It is the goal of our project management organization to manage these risk issues and ensure that we are safeguarding our ability to process payment transactions and move money within the payments space.

In past lives with other organizations I've run the gamut from running lifecycles that are even more formal than the one my teams currently use to running very informal lifecycles similar to the agile paradigm. The key is understanding the culture of the organization, the risk tolerance within the organization and the  time to market pressures within the industry.  As leaders within your organization, it's your job to discuss these issues and build processes that match the particular needs of your organization, manage the risk to the organization and your customers and build a project lifecycle paradigm that can deliver.

If you'd like more information on my background: LinkedIn Profile

Don’t Hide Behind the Process

Well, it’s been a while since I’ve discussed process issues.  Time to dig in and have some fun!

Most of you that have followed my previous posts know that I’m a process kind of guy!  I’ve introduced formal lifecycle management into several organizations.  The teams that I’ve led have successfully utilized process enhancements to drive improvements in productivity, overall capacity and quality within various organizations large and small.  Utilization of a known process allows you to begin to measure various pieces of the lifecycle – looking for inefficiencies and issues that impact your overall ability to deliver for the organization.

Now all that said, each organization needs to find the lifecycle process and development paradigm that will work with their culture.  I’m not going to try and tell you what development paradigm you need to follow because I don’t know your organization.  Within the organization I currently work for we largely follow a waterfall process with some agile techniques utilized within various teams.  This works for us and the style of applications that we create/support – mostly technical interfaces within the payments industry.  Our overall lifecycle has various checkpoints/gating that we use to validate we are ready to move to the next stage.  This works for us!  We’ve significantly increased our productivity and capacity over the last decade and have improved the overall quality of applications that we are delivering to our production environment.

While all this has been good, I occasionally notice when people are hiding behind the process versus just getting the work done!  Not every project needs every step of the lifecycle.  Some are small enough and straight forward enough that we can skip various parts of the process or make the decision not to create some of the standard artifacts/documentation.

Decisions need to be made up front when the project is defined.  Those in the know, need to direct the teams when a project needs the full pomp and circumstance of the lifecycle and when the teams have the option to skinny up the process and skip parts of the lifecycle.  That means at the very earliest stages of Discovery, the Project Sponsor should be giving direction to the team on what the risks are associated with the project which can then drive decisions on how much of the process can or should be used on the project.

Additionally, once a project has been initiated, I think individual team members need to be cognizant of what is happening and make recommendations to the Project Manager if they feel certain pieces of the lifecycle could be skipped or maybe some documentation might not need to be created.  I’m not saying that we’ll always make the choice to skip over pieces of the lifecycle or to not create various pieces of documentation, but as leaders, we need to listen to the recommendations of the people closest to the work and if it makes sense, say yes!  When we decide that we can skinny up the process used within an individual project or that we can eliminate some of the standard documentation, those decisions should be documented.  That way, there is a record of the decision and the reason is documented and shared with the Project Sponsor.

Overall, we still need to deliver for the organization!  Delivering the requested solutions with the necessary features/functionality; delivering a solution at or below the expected budget; delivering a solution with the highest levels of quality.  Our teams should be held accountable to ensure we are delivering successful solutions at the end of the project.

If you'd like more information on my background: LinkedIn Profile

Plan for Success ... Be Aware of Reality!

I run hard! My teams run hard!  We’ve got a lot on our plates and the pipeline is overflowing.  As soon as something is done, there are 5 more things waiting in the wings that need attention.  That’s a good thing – it tells me that the business values the services that my teams deliver on and that we can continue to be a difference maker within the organization.

As we go about our business – we plan for success.  Delivering product and services that our Financial Institutions can use to succeed in the market and make a difference with their customers.  Over the last several years, we have built a process around activity that ensures that we know why we are building something, we know the success factors around what it is we are building and that we then can design, build and QA the stuff we are moving into our production environments.

The teams have done a great job of reducing the number of iterations built before code can move into production.  Translation, they’ve reduce the number of critical errors found during testing that requires additional iterations prior to implementation!  They’ve also significantly reduced the number of issues that require after hours support.  Our teams have worked together to reduce the overall amount of time needed to regression test our products.  We are doing automated nightly builds on our code and are well on our way to having a full suite of regression tests run along with the automated builds.

So why am I talking about all of this?  Well, to put it all in perspective, it doesn’t seem to matter how well we refine the processes and manage the activity, we still have things go wrong!

1. Requirements identified after development is under way. 
2. Designs between teams that sometimes conflict. 
3. Estimates of project activity that end up being woefully incorrect. 
4. Resource constraints across teams just based on the sheer volume of activity. 
5. Data issues within our test environments. 
6. Validation exercises that end up taking longer than planned.

You know the drill.  You’ve all seen similar stuff happen to your projects.  So what’s a team to do?  Well as much as we want to keep our rose colored glasses in place and see sunshine and rainbows, it’s our job as leaders and technical experts to measure and apply a dose of reality within our projects.  The organization is looking to us to deliver new features/functions into the production environment.  They don’t really care to hear about how we make the sausage or why it might be taking longer.  What they do want is a semi-reasonable date as to when the features/functions will be available and what it means to our Financial Institutions and our Acquirers.

So, translating back to our internal teams, that means we need to make reasonable estimates along the way and refine as we move through the process.  As technicians, looking at and identifying the tasks needed to complete the activity – you can’t assume everything will go the way you want and your estimates need to include the breathing room you’ll need when something unexpected pops its head up.  As project managers, you can’t believe everything you’re being told – you need to validate what is being said and ensure people are on track.  Depending on the complexity of the project, you may need to put in some buffer time between the various activities to ensure that the teams have time to stay on track and recover from unknown issues.  This may not be needed on smaller less complex projects, but as the complexity gets bigger, you’d better start thinking seriously about buffer space.

I am not advocating that you needlessly pad time into your projects to make a 6 week effort end up taking 10 weeks.  What I am advocating for is that project resources accurately evaluate their tasks in a realistic fashion and feed those numbers back up to the project manager – that includes identifying the riskiest tasks and ensuring that you’re not being overly optimistic on the estimates.  Additionally, the project manager needs to review those estimates coming in and ensure that the numbers are ‘good’ and make judgements on where additional time may be needed to address parts of the process that look to be difficult.

I’ll give you a perfect example within my own organization.  With our large enterprise wide projects, we perform an integration test within development prior to moving the code into our formal QA and Parallel environments.  We began this formal ‘integration’ test effort probably a year and a half ago.  This testing required us to ensure mock data was setup across various platforms and application silos that would allow us to take transactions across the entire ecosystem.  This was initiated to ensure that by the time the code hit QA that we had successfully tested input/outputs and application handoffs between the various silos.

When we initiated this new step in our overall lifecycle – we were extremely aggressive in how long we thought it would take us to build the environment, populate the environment with the correct data and then execute the integration test plan.  I kept pushing the team to get all this activity done within a short time window so that we could then get into the ‘real’ testing and move the project forward.  Whether planned or not – the time associated with this integration testing took longer than planned and we eventually had to account for that within our planning.  Now that we’ve been through several iterations, we are beginning to automate various pieces of the environment and data setup/configuration – this should help us draw the time back down

Our goals were right, this helped us implement code into production that had a higher level of quality, however, I created a set of false expectations within the teams on the durations associated with this activity and they needed to give me feedback to reset my expectations and identify the correct timeline within the overall project activity. 

If you'd like more information on my background: LinkedIn Profile

Estimates - The Buck Stops with You!

Why are people so uncomfortable making a decision?  I admit I scratch my head frequently at some of the issues that land on my plate because someone was uncomfortable making a decision – or because they feel they haven’t accumulated enough information.

This is not a phenomenon unique to my current organization.  I’ve seen it across most organizations that I’ve worked in.  Managers have engrained into people that they are not allowed to make decisions and that they must review everything prior to anything getting done.  YIKES!  And then we wonder why people fail!  Better yet, we wonder why people leave us and move on to a new job – leaving us to spread the workload around to others on the team.

Look, I won’t claim my teams are exempt from this pattern.  But I do try to encourage people to think through problems they are facing and if they are unsure of a direction, at least bring options into the conversation when we sit down to have a discussion.  This at least allows me to see that they have thought through the problem and are attempting to address the issue.  Usually through conversation they’ll come to some type of a conclusion, thank me for the discussion and head back out.

I think where we struggle the most within my current teams is getting people to make estimates on project activity.  This is a personal decision by team members measuring how much work they will be responsible for against requested project activity.  When we first introduced this concept, there was a lot of push-back.  And to be frank, there were a lot of errors in the initial estimates.  But over time, people have grown more confident and are estimates are aligning better with the actuals that we measure over the life of a given project.

We go through regular training with people that are new on our teams.  It really doesn’t matter if they’ve been in development or whether they are new to the ropes.  Most developers have never been trained on how to produce reliable estimates.

So how do you pull together an estimate – well, for one thing, it’s different depending on what stage of the project you’re in: 

1. Discovery Phase: Usually estimates at this stage are done by someone within the management chain along with senior level technology resources.  How?  Well, the truth of the matter is the project scope is looked at in comparison to other projects that have been completed that smell like or look like they have similar impacts.
2. Requirements Phase: Typically this is a minor refinement of the estimate that was given in the Discovery Phase.  Any changes to scope are accounted for and typically there are additional details highlighted that point to previously unconsidered impacts.
3. Technical Design Phase: Now we are putting some meat on the bone.  At this point, technical resources have engaged and identified impacts across current systems/application silos and have also identified where new functionality must be created.  Along with the technical estimates business teams have assessed impacts to current and new processes.  Between the two paths, the full project estimate is considerably more accurate than anything that was put together in the first 2 phases of the lifecycle.
4. Development Phase: Some organizations perform the final detail design in this phase.  If so, this is the last chance to update those estimates both from a technical and business perspective.  This should allow the resources that are going to do the actual work to review the technical designs, identify the specific chunks of code or the business processes that are going to change the final say in what amount of time is necessary to complete project activity.

Rules of the road when it comes to performing estimates:

1. If you’re implementing new technology into your current technology stack – maybe you also need to set aside time to install and integrate the technology.
2. Don’t forget process changes.  Sometimes these changes can be just as critical to the project timeline as the technology changes.  Don’t underestimate the impacts these can have across your teams.
3. Don’t forget testing – and I’m not just talking about QA.  I’m talking about Unit Testing, Integration Testing, Parallel Testing, Performance Testing, Regression Testing, New Feature/Function Testing, and UAT Testing.
4. Don’t forget implementation planning and actual implementation activity.  While the universe may have been created with a big bang – in most instances you are going to see implementation activity spread out over a period of time.  Some implementation activity begins before the code touches production, and some activity happens months after the code is in production – you need to account for all of the activity.
5. Don’t forget to provide a buffer for 3^rd party activity.  I have yet to see a 3^rd party engagement go off without some type of change that has to happen.  The initial timeline provided for the engagement never usually is achieved.

Items for consideration when building the estimates:

• How many screen modifications need to be made?
• How many new screens are there in the project?
• How many web services need to be modified?
• How many new web services are there in the project?
• How many reports/data extracts need to be modified?
• How many new reports/data extracts need to be created.
• How many tables/stored procedures need to be modified?
• How many new tables/stored procedures need to be created?
• How many 3^rd party apps need to be touched?
• How many new 3^rd party integration activities are needed?
• How many processes need to be modified?
• How many new processes need to be created?
• What infrastructure changes are needed – servers, server types, routers, switches, security devices?
• What new infrastructure pieces need to be installed – servers, routers, switches, security devices?

Realize that you’re first efforts are going to be wildly off base.  Don’t be afraid to tell your team that you’re giving them time to get better at creating estimates.  This means when they’re wrong – it’s a conversation not a point against them in their annual review.  Use each estimate as a coaching opportunity when the final numbers come in – what they did right and what didn’t look so good and how do we make it better.

This by no means covers it all, but points you in the right direction.  Now I’ve heard some people say that in an Agile process you don’t need to worry about this.  Yeah, right.  Trust me, someone in the organization is telling the CFO what this thing your building is going to cost – how many 2 week iterations it’s going to take to get full functionality.  If you’re not responsible for making those conversations happen today – you will be at some point.  Better to start figuring this out know, before you’re asked to provide that first estimate.

Realize that you’re first efforts are going to be wildly off base.  Don’t be afraid to tell your team that you’re giving them time to get better at creating estimates.  This means when they’re wrong – it’s a conversation not a point against them in their annual review.  Use each estimate as a coaching opportunity when the final numbers come in – what they did right and what didn’t look so good and how do we make it better.

If you'd like more information on my background: LinkedIn Profile