Recapturing our Youth!

This has been an interesting period in my life. My wife and I recently moved our youngest son out of the house into his dorm, so that he could begin his college experience. It was a really incredible day, seeing his excitement and watching him and all the other kids in the dorm move in. Yes, we were busy setting up the room and getting all of his stuff tucked away. Yet, while all this was occurring kids were wandering the halls beginning to meet each other. The excitement was visible, everywhere!

Over the last couple of weeks, he has made it a mission to visit each of the floors in his dorm, using a Frisbee to start up conversations and get to know the people he now lives with. He has wandered into other dorms in an effort to get introduced to more people and build his social network. He and his friends have introduced themselves to people tailgating the football games and expanded their social circles. He seems to be on a mission to get to know as many people as he possibly can – yes, he thrives on social experiences.

Looking back at my own experience as I made the transition from home to college, I remember the fear, the excitement and the fun of leaving home, meeting new people and having new experiences. It got me thinking about how change is viewed in the office, let’s just say it is not necessarily a positive within our teams.

Let me walk thru a recent experience – over the last year we have been introducing the use of Kanbans as a tool to visualize enterprise project activity across the organization and within my teams using team stand-ups to talk thru the tactical daily promises. Our daily stand-up meetings are focused on simple communication – what did you promise yesterday/did you deliver, what roadblocks are you encountering and who needs to jump in to help remove the roadblocks and what are you promising for tomorrow. Can we move the card on the Kanban? Yes, we are crawling before we walk, walking before we run and running before we try and leap. These are things that many folks within the organization and within my teams have advocated over the last several years.

We are not fully Agile, not sure if we will ever get there, but we are trying to use some of the tactics within projects where it makes sense for us.

What I find interesting is the pushback we have received along the way. I was expecting some of this, but was surprised when some of the very people promoting Agile techniques began to resist the actual implementation of some of these techniques. Some of it came down to – well if you aren’t going to implement and adhere to the entire Agile principles, than how do you expect me to get onboard. Other times it came down to, well I’m not really comfortable trying this in-front of other people.

I get it, we hate change. But when did that shift occur? Again, I look back to my sons as they grew up. They constantly were experiencing change, each year asking to take on new responsibilities. Tackling new topics at school, meeting new teachers, being a part of school sports, taking on roles within the student council. Outside of school, trying various sports when they were younger and then adjusting to changes as they progressed with their athletic talents. Sometimes taking on leadership roles, other times acting in support roles. They grasped for these opportunities, not expecting failure and looking for the next challenge.

Yet, somewhere in our path to adulthood, we start to resist change. We like the stability of knowing what it is we are doing, what comes next and who does what. The funny thing is we talk about change, until it happens to us.

Ultimately, we need to be agents of change for the companies we work for today. In the past, there were periods where businesses did not need to worry about rapid change. That dynamic is no longer true, businesses must continually evaluate the products and services that they are providing, the competitive pressure of local and international competitors. If you’re not constantly ensuring that you are serving your customers, your customers will walk away.

I encourage you to recapture that spirit of youth where you wanted change. Where you weren’t afraid to accept change. If our businesses are going to succeed, we need to continually change to adjust to the needs of our customers.

If you'd like more information on my background: LinkedIn Profile

Don’t Be Afraid … Change!

Over the last several years, my teams have made significant changes to our project management, development, testing and implementation processes and paradigms. We’ve had some bumps along the way and in no way do I consider this evolution over. I will continue to challenge my team members to find a better way to build the mouse trap that drives our software development lifecycle.

Let’s look at some of the successes the teams have had introducing change into the organization:

1. Our project management process used to be split into separate and distinct teams: 1) project management for business and operational teams; 2) project management for technology teams. This has been consolidated into one project management team managing all project related activity across the organization. This has streamlined activity and improved communication across all teams involved in the process.
2. Our lifecycle was revamped as we consolidated the project management organization. During the revamp – we created RACI documentation that clearly outlined 1 owner for each key task/milestone within the overall project activity, along with those that would act in the roles of responsible, consultative and informed parties. This has provided clarity to who owns each activity and what the key tasks are that are included within every project.
3. Recently we made additional changes to the lifecycle – completely eliminating 1 phase and adjusting the tasks and milestones associated with both our scoping and design phases. This has allowed us to streamline the process and give clarity to when project activity actually starts within the organization.
4. We have challenged all of our team members to review the overall standard tasks/milestones within any given project, and challenge the project manager and sponsor to eliminate tasks and standard artifacts when it is justified and where there is minimal risk associated with utilizing an altered process.  This decision then is documented as part of the project and communicated to the team to ensure that down the road, people can review the effort and understand why certain documents or tasks might not have been done. This should allow us to be more nimble and find the best path for each project to take thru the organization.
5. A couple of years ago, it took up to 3 weeks to complete regression testing associated with key functionality within our application silos. This was a manual effort that typically was handled by multiple individuals. The teams have worked hard to automate the regression tests associated with these applications. Today, the execution of these tests can be done in less than 17 hrs. The labor savings moving forward will more than pay for this effort. This also allows us to provide quicker feedback to the development teams and ensures that our regression testing is more consistent across all projects.
6. Recently, the teams also completed the resurrection of standardized test data to be used across our test and development environments. This will be further enhanced as our automated tests are enhanced to create/modify/cleanse and reset data needed for a given test. This will reduce the time spent creating test data for each project and will ensure downstream processes/applications have the data that is needed to execute their regression tests.
7. Over the last several years, our implementation plans have seen significant improvement. Our business demands that our systems be available 24x7x365. When you want to buy a tank of gas at 3AM or 3PM, you want the transaction to go through so that you can continue on your way. You would not find it acceptable for us to deny the transaction because we are updating our systems.  Our implementation processes have been templated and include all implementation activity, across all application and infrastructure silos, commands, scripts, validation and fallback steps in the order they will be executed.  Recently, our external auditors informed me that our implementation plans are among the best that they have ever reviewed. This has hardened our implementation activity – reducing the overall time needed to implement changes into our production environment, standardizing the validation activity during and after implementation activity has occurred and most importantly having fallback steps at the ready when any validation activity shows that there may be an issue.
8. Our development teams have changed their processes to include code reviews of all changes prior to those changes being introduced into our production environment. This is hardening our code base and reducing the risk of making changes within our production environment.
9. Our development teams have changed their processes to ensure that unit tests are created for all code being touched within a project.  These unit tests are then kept in a repository and used in future projects. This, again, has hardened our applications and reduced the overall number of bugs slipping through the process into our quality assurance testing or into our production environment.
10. Our development teams are now doing nightly builds – with reports going back to each team on build failures, unit test execution failures, code coverage and other statistics. This is giving immediate feedback to our development team and ensures that issues are addressed within 24 hours.

These are just some of the changes that have been implemented by the teams over the last several years. And we’re not done! Moving forward, we will move the regression testing into our nightly build process. We will be evaluating further changes to our development lifecycle to reduce the amount of time between idea generation and product introduction into our production environment. We will continue to improve upon our implantation planning. We will introduce key metrics across the development lifecycle. And where it makes sense, we will introduce Agile techniques into the development process.

I talk to college and high school students on a regular basis to discuss what it means to be in the technology field (specifically software development). The one constant that I tell students is that being in this field means you will constantly have to learn. They will learn new languages, they will drive change into the organization, they will experience multiple different development paradigms. All of this is good – it means the organization is alive, is learning and is adapting to its competition. 

Don’t settle for how it works today or with someone telling you, “well, that’s the way we’ve done it for decades!” Your job, no matter what role you play in the organization, is to find a better way, review your suggestion with those that can help implement, trial those changes and then roll them out across the team and the organization. Change is good – don’t settle for the status quo! 

If you'd like more information on my background: LinkedIn Profile

Off-Shore Development - Plan it right or Fail!

Off-shore development continues to be a discussion point within many organizations.  I've managed teams across the US and had the occasion where I've used off-shore resources.  Let me share some of the experiences in the hopes that it helps others that struggle with the model or are questioning whether to proceed with the use of off-shore resources.  I won't claim this represents the model to run off-shore development, ultimately, you need to make the decisions that will work within your organization.  Hopefully, this will shine a different light on the topic and allow you to gain insight that may help you make the decision to use off-shore development resources or to keep the development on-shore.

First, it is not easy.  If you think this will solve issues within your current development lifecycle - don't bet on it.  If you're going to seriously begin to use off-shore resources, you better plan appropriately, or you'll pay the price with development timelines that get extended and budgets that creep beyond your expectations.  In the worst case scenario, you'll end up rolling product into production that has not been properly tested and vetted.

Some of the breakdowns that my teams encountered in using off-shore resources are as follows:

1. Communication - eliminating the natural lag in communications as teams work shifts that do not align.
2. Design Aberrations - ensuring the product developed fully conforms to the original design/requirements.
3. Coding Standards - putting processes in place to ensure that the off-shore development team followed the development coding standards of our internal teams.
4. Quality of Code - ensuring that the code had been properly tested all the way through integration testing with other application silos/subsystems.
5. Quality of the Team - ensuring the consistency of development resources applied to projects.
6. Product Knowledge - ensuring that knowledge of team was retained from project to project.

The above list covers the major challenges that we encountered.  Let me go in to some level of detail so that you can get the flavor of what was experienced and how we handled the longer term needs of the organizations use of off-shore resources.

Communication: Each and every day, without your knowledge dozens and dozens of project decisions are made as developers lean across the walls and talk to each other or see each other walking around the office.  My teams have been no different, requirements and designs could change based on a hurdle that needed to be solved and a quick conversation in the hall.  Suddenly you have a team separated by multiple time zones - in my case, they had left the office for the day by the time we arrived in the office.  If the remote team had a question, or hit a roadblock and needed direction, there was an automatic lag of at least 1 work day.  If it was a serious issue, the team might loose several days as communication threads worked their way through email.  In extreme instances, we might have some of the remote team stay late or some of the local team come in early to establish conference calls and work out the issues.

Design Aberrations: The standard methodology we used to develop project requirements within the organization wasn't sufficient to support the use of outside resources.  Didn't matter if those resources were on-shore or off-shore, our teams were comfortable dealing with each other and could abbreviate how the requirements were documented.  There were a lot of things assumed within the requirements that didn't need further explanation because the teams had years of experience working together.  This continued to feed through the process as design documents were generated by the team.  Suddenly we introduced a team of developers that did not have that tribal knowledge.  The results were obvious in hindsight, the off-shore resources built the product to match the documented requirements and designs.  The only problem - those requirements and design documents didn't contain the unspoken elements known by the internal teams.

Coding Standards: Some of the products that we chose to develop off-shore ended up needing to be supported by local teams.  We quickly found out that the coding standards we had in place were not documented thoroughly and left many items up for consideration.  Translation, the remote team was producing code that looked like none of the other code that was used internally.

Quality of Code: The internal teams had many years of experience working together, touching each others code and making the necessary revisions to the code base.  Suddenly we had bugs that had been solved being reintroduced into the code base later in the development cycle.  Worse yet, we would find chunks of code that were never being executed, yet were loading up the code base.

Quality of the Team: Just as we were getting used to working with one set of developers, suddenly they would be reassigned to a different project and we would end up having to reacquaint ourselves with a new team.  To say that this slowed down development and introduced inconsistencies into the code base would be an understatement.

Product Knowledge: As we completed development with one project and started up the next project, we were not assured that the team we had gotten used to would continue on and do development on the next project.

None of this is serious and can be resolved, but you need to think through the engagement up front and take the necessary measures to protect your organization.  Here are some of the tactics we began to utilize to minimize the risk in the use of off-shore development resources:

Communication: We required the off-shore company to place one of their resources on-site with our local teams.  They became the primary communication point between our teams and the off-shore resources.  They sat in on the meetings locally and could catch the small things that needed to be communicated to the off-shore team.  They could do the impromptu meetings in the hallway and get that information promptly to the remote teams.  Our project timelines included buffers to accommodate the communication lag between the teams.

Design Aberrations: We ended up beefing up our Business Analyst and Architecture teams to create more thorough documentation.  Our documentation became much more thorough and attempted to accommodate the things that people took for granted.  The architectural diagrams became much  more sophisticated outlining all servers touched, all databases touched and were color coded to identify new, changed or removed elements.  Design documentation began to reference not only the objects that would be touched, but could go down to the individual methods and parameters when needed.

Coding Standards: We ended up going through our coding standards and documenting much more thoroughly how code was to be structured, how variables names were to be created, what security precautions were to be taken when creating the code, etc. 

Quality of the Code: Code created by off-shore resources was required to go through the same code review process that we used for internal teams.  This ensured that our senior developers were reviewing the code - exactly as would be done for internally written code and that they could catch issues and feed them back to the remote team.

Quality of the Team: After some experience with team members shifting, we ended up negotiating with the off-shore company to ensure that we had a core set of developers that would not be reassigned off of the project.  This allowed us to maintain some level of consistency within the off-shore team and addressed quality of code, coding standards and communication issues.

Product Knowledge: Similar to the Quality of the Team issue - by keeping a core team of developers assigned to our projects, the off-shore teams knowledge base of our products began to increase and they could begin to anticipate some of the issues ahead of time.

I want to state again, these were experiences that my team had in working with off-shore resources.  Other organizations may have different experiences.  Hopefully, some of these issues and the ways in which we resolved them will assist you with determining whether you can support the use of off-shore development resources.

I typically do not use off-shore resources unless there is a need to temporarily augment our internal resources to accomplish specific development/project goals.  That's a choice that I feel comfortable with based on the particular business needs within the organization.  In previous roles, I've had the opportunity to work extensively with off-shore resources.  If you're going to use off-shore resources, I highly recommend you pick a couple of small projects and trial a couple of different vendors to establish which one will work better with your current development paradigm.  Don't be afraid to switch  vendors if you don't feel you're being taken care of - ultimately, it's you that will be held accountable for the delivered product.

If you'd like more information on my background: LinkedIn Profile

Implementations - When Things Go Bump In The Night!

Sometimes bad things happen.  The question - how fast are you able to respond and put things back on track?

Technical teams need to plan and test their ability to rollback changes that are negatively impacting users.  This needs to be a formal part of your process - sometimes that means falling forward, fixing the broken functionality in production; sometimes that means falling back and reverting to a known state where the application was functioning properly.

Let's be honest - none of us are perfect all of the time.  Occasionally, we are going to miss something and move code into the production environment that ends up breaking things and making life miserable for the folks actually using the application.  

What have you done to prepare for this moment?

1. Does your implementation plan include steps to proactively monitor the system as changes are moved into production?
2. Does your implementation plan allow time for you to look at patterns of application activity to validate that you are not seeing patterns that would indicate that something is broken within the application?
3. Does your implementation plan include validation steps to ensure that the application is acting accordingly and that the new/changed functionality is working?
4. Does your implementation plan identify risk points and fallback strategies?
5. Have you tested your implementation plan?
6. Have you tested your fallback plan?

Every environment is different - but we mostly face the same types of challenges.  Breaking things that users expect to work is not what we want to do, but occasionally, it happens!

Yes, planning your implementations takes time.  Yes, planning validation steps and inserting them into the overall implementation plan takes additional time.  Yes, testing your implementation plan chews up more time.   And, yes, testing your fallback plan takes even more time.  That all being true, failing to do any of this means that your operating by the seat of your pants and your recovery time when things go bad will hurt you worse.

1. When your application fails - the reputation with your customers takes a hit.
2. The longer the outage increases the risk that your customers will walk away from you and begin to use a competing application?
3. Will an outage impact your revenue stream?

Now, all that said, sometimes even with all of the best planning, you're going to get caught.  It's bad enough when it's a web based application/service where you can make immediate changes or perform and immediate rollback.  However, what if it's software that you're sending out to customers - that they install on their systems?  How do you think Microsoft feels when they send out one of their regular patches only to find out that it causes the blue screen of death.  This can affect millions of computers around the globe.

Not to hit Microsoft too hard - but, in April they released a patch to Windows 8.1 that was required before any other updates could be applied.  This impacted retail customers as well as corporate customers and was not fixed until early May.  I'm sure that they ran their standard quality processes against the code prior to the initial release in April - but something obviously got by the testers, and it wasn't a minor issue.  It took time for Microsoft to hit the reset button, but in the meantime their customers were in an uproar.  

I'm going to walk softly here because as a developer it could happen to me or the teams that I lead.  I'm not trying to single out Microsoft - anyone that writes software has had to recover from a misstep when loading code into production or sending out a new release.  What I'm advocating for is that you implement solid processes that minimize the risk to your customers, your team and your organization.

What are you doing to mitigate the risk of an implementation/software release going bad?

Tags: Development; Programming; Programming Languages; Change; Decisions; Decision Making; Project Management; SDLC; Lifecycle;

For more information on David L. Collison: LinkedIn Profile

Back In The Saddle Again ...

I have been distracted with several items over the last few months and apologize for my absence.  That said, let's dig into our discussion and see if we can do some sharing.

Sharing my thoughts on Microsoft ...

My early career was built on Microsoft.  I graduated high school in the early 80's just as the IBM PC started to be sold.  Yep, aging myself.  Anyhow, after learning BASIC and Fortran on a DEC 1170 - I migrated to programming on an IBM PC using BASIC and then learning PASCAL and Assembler.  Then I learned dBase III and Clipper - learning to program against databases.  All on a PC with floppy drives - yep, no hard drives at that time, just dual floppies where you needed to swap disks to manage all of the data.  None of this would have been possible without the OS that Microsoft built and licensed to IBM.

Then in marched the IBM compatibles - Compaq the most successful in the early days.

Not only was I programming, but I also ended up selling IBM and IBM compatible equipment.  The company I worked for let me lug around a Compaq portable while I worked for them.  I had access to Microsoft compilers for Basic and Assembler, and used Borlands Pascal.  The company I worked for also had me teaching programming courses.

Along the way we transitioned to Windows and I learned how to program using MS Visual Basic.  This became my bread and butter for quite some time - allowing 2 buddies and I to eventually open our own consulting firm.  We built a healthy practice, hiring additional developers and working for several local and national companies.

I was able to enter the field of software development right as Microsoft was transitioning the industry.  Moving us all from monolithic computers operating in the back room to a vision of a computer on everyone's desk.  I knew the first time that I saw the IBM PC - that it was my future.  I didn't look back - I wanted nothing to do with the mainframe class machines, I wanted to figure out how these small (by the standards at that time) boxes could be used in business.  And oh how the world turns as I know manage development activity on mainframe class machines and Unix/Linux clusters.

For a long time - Microsoft owned the development community.  They created tools that allowed us to transform the industry, thereby further fueling their growth.

Then Microsoft lost it's way - it became more focused on protecting it's cash cow and locking developers into their environment.  Companies attempted to displace Microsoft and turn us onto their platform, but most failed.  Then along came the browser and smartphones - to say Microsoft was not ready would be an understatement.  They nearly lost the browser wars and only woke up at the last minute to use their monopoly power of the OS to win the browser war.  The smartphone is a different story.  Windows Phone is an also ran and I'm not sure Microsoft will ever have the chance to be a driving platform within the phone space.  Oh, they'll stay in, but in my opinion, they won't truly compete with Google or Apple.  They missed the boat.

On the desktop Win8 was a colossal fail - yes, they can brag about the numbers sold, but most companies are using the license terms to downgrade to Win7.  I personally hate Win8 - it's a mindless combination of OS modes that make no sense on the desktop.  Win8 may be pretty on a tablet and phone, but on the desktop it only irritates.  Where I used to regularly recommend Windows equipped laptops and desktops to friends and family, I've now moved to pushing Apple's ecosystem.  Many people only need an iPad, and for those that need more, the MacBook Air,  MacBook Pro, iMacs and Mac mini's are the choice.

And this is a stunning development - ask my family and friends, for a long, long time I viewed Apple skeptically and always thought they were too expensive.  Well, you may pay more for a Mac, but if that allows me to walk away from the problems that I experience in the Windows environment and mess that they call Win8, it's worth the money.

Where the Apple ecosystem just works, the Microsoft ecosystem confuses - mindlessly flipping between metro mode and desktop mode, sometimes for no apparent reason.

Of all the expert articles written about this farce of an OS and what Microsoft needs to do to fix it, I vote for Windows Red as documented by InfoWorld.

Can they come back, who knows?  They'll never go away, they're too big and have too much money in the bank.  They would really need to screw up to disappear, not saying it can't happen, but the chances are remote.  They have become the new IBM - too big to fail.  

All that taken into context, the best move they could have made was to ditch Balmer and replace him with Satya Nadella.  I don't think Balmer would ever allow Office on the iPad, and while the package introduced by Nadella is handicapped, it's at least a start.  Microsoft appears to be making some steps to stop the ship from sinking - I wish them well, but really wish they'd kill the FrankenOS on the desktop.

I know this has nothing to do with Project Management or specifically with software development, but I thought I needed to do a mind dump on this topic.

Tags: IBM; Microsoft; Windows 7; Windows 8; Win7; Win8; Change;

If you'd like more information on my background: LinkedIn Profile

Languages, Languages, Languages ...

I speak regularly with High School and College level students promoting careers in technology - specifically careers with Applications Development, Quality Assurance and Project Management.  I enjoy interacting with the students and telling them what they may experience as they transition from the role of a student into the professional workforce.  Along the way, I also ask students to think about interning - either with my organization or some other organization in the area.  This is a fantastic opportunity for students to get real-world experience, become visible to an employer, and begin to understand some of the concepts taught in the classroom as they apply them to tasks in the workplace.

Many times, students ask me why we use the languages that we use and why we don't take advantage of the latest scripting or language fad.

My answer - why should I?  They seem to be stumped by my response.

Well, really, why should I?  Why should my team be disrupted on a regular basis just to use the latest language fad?  Why should I continually experience a productivity loss as team members shift between languages?  Why should I take the risk of bugs being introduced into production systems because the developers are not experienced with the language/tools that they are using?  Why should I risk customers being subjected to performance issues because the developers haven't figured out how to optimize the programs they're creating?  Why should I make the on-boarding experience for new developers more difficult?

Look, there are a handful of languages that we've committed to within our organization.  Four languages are used to generate all of our production level systems.  A couple of additional scripting languages have been used to create internal tools that assist us in production setup, as well as assisting us in ripping through logs to find specific information when we are researching issues in our environment.

Programming languages are programming languages - if you're a good programmer, you can pickup any language.  If you're a good developer, you understand that once you understand the various constructs of a language - learning a new language simply means translating the specific syntax of the language you want to learn to the languages you already understand.  It's translating how to use an if statement, a loop, a case statement, etc.  

Let's say it take you a month to become familiar with the new language - assume that your productivity drops by 50% and for 6 months after your initial learning period, you're running at only 75% productivity.  That means that over a 7 month period - you're productivity has cost the organization 8 weeks of development time.  Now multiply that across the entire team.  Assume their are 10 members on the team - that's a total of 80 weeks of lost development effort.  Assuming this same 10 member team and an average of 2 weeks vacation time for each team member - you have a total of 500 weeks that you can allocate to development efforts.  The end result is that you've stripped the team of 16% of the overall capacity of the team for the year.

Now, the numbers discussed above don't even take into account the impacts down the line as we move through test, implementation and post-implementation support.  How much more time will the Development Team need to create and execute unit tests?  How much time will the Development Team need to perform integration and regression testing?  How much time will the Development Team need to perform load testing?  How much extra time will the Quality Assurance Team need to test the application?  What additional issues will be experienced during the implementation of the code into production?  Can you quantify the number of additional issues that will be identified post-implementation that will need to be fixed and how much additional time it will take to address those issues?
 
The impact on productivity across the organization is significant and material - that is real time that prevents projects from being initiated or completed.  That's time that allows your competitors to either close the gap or get further ahead.

I don't mind taking on a new language when there is a need, or when there is a specific purpose.  Over the last couple of years, we've actually brought in specific tools that have allowed us to move in to the mobile market.  At this point, a select number of developers within the overall team have been exposed to these new tools.  Over time additional developers will be brought in to the loop.  Here, there is a specific need - our tools at the time did not lend themselves to creating cutting edge mobile apps.

Take a step back and look at the decisions being made within your organization.  There's usually a reason that organizations aren't moving as quickly from one language to another as developers would like.  To the developers that would like to get an organization to change - don't just go in and say that you need to be able to use a new language.  Document what you feel the benefits are, and just as importantly recognize the impact to the organization and craft your discussion points to address the costs that the organization will incur.


Tags: Development; Programming; Programming Languages; Change; Decisions; Decision Making;

For more information on David L. Collison: LinkedIn Profile