Follow The Data ... Software Design, Development and Testing

‘Follow the data!’ I have preached this for years. If you truly want to be a great developer, you learn the data flows!

As a software engineer, we want to be recognized as exceptional in delivering code that functionally delivers against the user needs and doesn’t introduce performance issues or defects into the production environment. We want our code to elegantly solve the problem that has been described and delights the user. If the code that we develop is a smaller slice of an overall application or platform, we want to ensure that our integration points are rock solid, that we are doing nothing to impact the ability of the larger application or platform to successfully process and perform the function as it was designed.

It is critical that as a software development professional, we teach ourselves how data flows thru the system. How does the data flow into the application function that we are developing, how does the data get manipulated by the function we are creating and what happens to the data when it leaves our function. This means breaking the data down into its discreet elements and understanding how they are manipulated ahead of us, what we are going to do with them as we execute the new or changed functionality and what happens after it leaves us.

Let’s take a small scenarios (and I’m oversimplifying the scenario to make the point) … a Cardholder walks up to an ATM and inserts their card into the machine to request a $100 withdrawl. Behind the scenes the machine captures information off of the card to send to the financial institutions Processor. The Processor receives the information off the card and checks the balance in the bank account and spits out the $100 to the Cardholder.  Mission accomplished – right? Nope!

The ATM in the above scenario is not owned by the financial institution where the Cardholder has their money deposited. So, somehow we have to settle the money between the two financial institutions. The withdrawl is one distinct component of the overall transaction. We need to follow the data from the original transaction to the end of the day when the money actually moves between the financial institution that dispensed the $100 from the ATM they owned and the financial institution that holds the Cardholders funds. Whew, done! Not exactly.

At the end of the month, the Processor must bill for the services provided to the financial institution that holds the Cardholders funds. There are specific transaction fees that get charged based on volume.
Now, let’s think about the software engineer that works for the Processor. Not only do they have to see the data in real-time and make authorization decisions, but they need to move the money at the end of the day and ensure that the financial institution is billed properly at the end of the month.

To follow the data, the software engineer needs to understand what data is read by the card reader at the ATM and is transmitted into the data center for authorization. They must understand how to match the card data up to the financial institution cardholders account to check balances, they must understand the authorization cycle and how the individual data elements in the card data are used to detect fraud and either accept or decline the transaction, they must understand how that data moves thru the system and is stored to ensure that the money moves between financial institutions at the end of the day, and they must understand how the data is used at the end of the month to properly bill the financial institution based on the type of transaction and other factors.

The software engineer cannot be sure that the functionality that they changed is working properly until they’ve tested the data all the way thru the transactions lifecycle to ensure that it settled at the end of the day and that the financial institution was billed properly at the end of the month. Testing should be designed with the full data flow in mind, not just the partial discreet transaction where the money was dispersed to the Cardholder. Again, I’ve oversimplified, but this scenario does describe the breadth of understanding that a developer must have before they can claim success.

Now add in some side scenarios – let’s say the data is used in web and mobile applications where the Cardholder can view transaction history data and initiate fraud claims. Those data usage scenarios introduce additional data flows that the software engineer must understand and validate before they can claim success.

Understanding how the data flows, where it is stored, how it gets manipulated prior to, during and after the function you are working on, is critical to understanding whether you actually can design, develop, test and deliver the functionality correctly. Knowing how to code is the simple part, understanding how the data moves and is manipulated is the hard part.

If you'd like more information on my background: LinkedIn Profile

Code Reviews - A Must For Any Team!

As professional coders, we should be concerned about the cost to remediate ‘bad requirements’ or ‘defects introduced into the code’ thru the development process. Over the years studies have shown that there is a significant increase in cost to remediate these code issues the deeper into the development lifecycle that you go. One of the most recent published studies completed by NASA broke it down as follows:

Phase	Cost to Remediate
Requirements	1x
Design	5x - 7x
Build	10x - 26x
Testing	50x - 177x
Production	100x - 1000x

In other words, if a bad requirement is not caught before design, by the time it makes it all the way into the production environment, there is a cost of 100x to 1000x the original cost to remediate the requirement. Yikes! Now, if a developer makes an error in building the feature/function described in the requirements or introduces a defect into the code unrelated to the feature/function being built, the cost to remediate that when found in production would be 10x to 100x if it was caught and remediated during the build process.

From a pride perspective, no coder wants to be seen as the one that introduced the bad code or broke current functionality. Beyond the pride factor, though, there are real hard costs being spent by organizations to remediate these issues, as well as the reputational impact with customers.

There are things that developers can do to lower the risk of introducing defects into the code base and to limit the exposure of the additional costs needed to remediate these bugs. Code Reviews are a proven paradigm that developers can use to catch these issues early and to remediate them during the build process.

Code Reviews: The intentional and systemic check of code by other developers to identify and remediate mistakes or security threats. The practice has repeatedly shown that it can accelerate and streamline the software development process and reduce the number of defects that reach production. In its simplest terms, a code review simply means that the person developing the code shares that code with other developers who review the code to:

• -        Ensure that team coding standards are followed
• -        Look for errors that can be remediated.
• -        Look for ways to ensure the integrity and security stance of the application
• -        Look for ways to simplify algorithms
• -        Look for ways to remove code that is not needed

As much as we like to think we aren’t the ones that introduce defects into the code base, we are humans and therefore the code will contain errors – those errors may be small, or they could be significant show stoppers. As professionals, we should acknowledge that fact and look for ways to mitigate these issues early. And that, my friends, is why Code Reviews became a thing!

When Code Reviews are done properly, they will address all of the above items, they will reduce time needed in later stages of the lifecycle, they will deliver hardened code into the production environment, they will reduce ongoing maintenance costs, they will foster communications within the development team, they will provide a means for more experienced engineers to mentor those engineers not as familiar with the code and they will ultimately increase the confidence and trust of the customers using the application. With all the positive impacts that code reviews can have for the developer, the team, the organization and our customers, why wouldn’t we use this well-defined paradigm to improve the quality of what we deliver downstream and ultimately to our customers.

Let’s quick review how this process looks:

1. Code Submitter (the software engineer changing the code) checks the code into the repository and makes the Code Reviewer(s) aware of the code requesting a review take place.
2. Code Reviewer opens up the code and begins to check the code:
3. Does the code do what it is actually supposed to do based on the requirements/user stories?
4. Looking for obvious defects in the code that might produce errant results.
5. Does the code adhere to the established coding guidelines that the team uses (there are tools that can automate this)
1. Code Formatting
2. Variable Naming Conventions
6. Is there Automated Test Coverage for the code being reviewed?
1. Have the tests been written in a way that is easily understood and provides clear results?
2. Have the tests been written to provide red route/green route testing – is the sample of tests broad enough to fully test the functionality?
7. Does the code adhere to the security/regulatory guidelines used by the team (OWASP for web)?
8. Does the code fit into the overall architecture of the application?
9. Does the code introduce potential performance issues that need to be addressed.
10. Is the code in the right place, are the services and methods grouped into the right places to easily be found by other developers?
11. Did the code introduce ‘duplicate functionality’ that should be removed?
12. Has the code been written in a way that is easily understood?
13. When the new code fails, do the exception messages provide clear direction on what failed and why?
14. Reviewing complex algorithms to see if they can be simplified.
15. Identifying chunks of code that are not needed anymore that should be removed.
16. Code Reviewer identifies recommended changes to the code that was submitted and documents the changes for the Code Submitter.
17. Code Reviewer discusses the recommended changes to the code with the Code Submitter in a respectful manner.
18. Code Submitter makes agreed upon changes to the Code.
19. Code Submitter circles back to Step 1.

That’s a lot of stuff! But the benefits are huge if done correctly. The beauty, I’ve seen interns and new engineers perform successful code reviews and give feedback to seasoned engineers, just as I’ve seen our most experienced engineers give guidance to our newest team members. This process in itself fosters communication within the team and opens up doors that allows the team as a whole to raise the bar and create high quality code. This becomes the expectation within the team, nobody wants to be seen as the coder that introduced a bug, they all want to be seen as experts in their chosen profession. Additionally, it gets developers talking to each other and opens up opportunities not seen in the code review process where they will begin to discuss challenges in real-time as the code is being created and not wait for the code review process to catch the issue.

I’m not saying that this will catch everything, again, we’re all human. But by using code reviews properly, we enhance our ability to catch issues early and remediate them before they impact our user community and potentially damage our reputation.

If you'd like more information on my background: LinkedIn Profile

Students - Getting Noticed By Hiring Managers!

One of the great parts of my role here at SHAZAM is the time that I’m able to take to go out and visit with many of the local middle school, high school and collegiate students interested in careers in technology. My being able to talk about the ins and outs of this industry, how it has kept me engaged over my career and to get feedback from these students lifts me up. I enjoy talking to them about the opportunities that they will have and the changes that they will be able to make along the way. 

Invariably, during these discussions, someone will ask, “but, how do I get noticed?”

Great question, how do you get noticed among the many students graduating and entering the work force. Let me walk thru what I tell students, and by the way, I’ve given this same advice to my youngest son, as he is pursuing a career in technology. This will be focused on software development and test automation.

First Focus on Your Grades: The grades you earn are important for that first professional job as you enter the working world. It is an indicator that you paid attention and took the time to understand the material presented in the classroom and that you were able to translate that into working knowledge on individual and group assignments. While a low grade will not be a disqualifier in the process, it is a barrier that you’ll need to explain. Show me where you went above and beyond on classroom assignments. Did you ever take on an extra project and share it with the class or the instructor?

Programming Experience: Doing your homework is the bare minimum. You’re going to need to explain group projects, your role in those projects, how you overcame conflict in the group and how you got people to work together. If you want a step up on other candidates that your competing with, you should look to do projects outside the classroom. Let me give you some examples:

• Expand your knowledge. Find a technology that you want to play with and start to learn it. You may not become an expert on it, but it is going to teach you knew concepts and new techniques. Are you interested in big data – show me something that backs up that statement? What about machine learning – instead of just saying you want to learn it, have you taught yourself Python and PyTorch. Even if you haven’t successfully gotten your first engine up and running, what have you learned, what can you show me, what challenges have you encountered?
• Recreate a game you like. Bring that with you to the interview and talk about the choices you had to make, what you learned from the experience and how it made you better as a developer. My son, one day announced that he was going to recreate the game ‘2048’ on his own and to one up the normal game, he was going to create an AI behind the game that would play it for him. Mind you, this was during his first semester at college, he was taking general education classes and wanted something technical to play with.
• Build a mobile app. We are all carrying around smart phones. The cost of entry for development is $0 – you can download the software development kits off the internet. A couple of years ago, we were looking to hire some new team members. I remember the hiring manager walking into my office after one of the interviews talking about how they had just completed an interview with a candidate. He walked into the interview pulled out his smart phone and started talking about an app that he had just created. So, this candidate showed he was passionate about coding, that he geeked out along with the best of them and that he could learn new technology. Winner!
• Stretch Goal – Contribute to an Open Source Project. Find an open source project that you care about, study the code and start submitting changes to correct features that aren’t working correctly. Your changes might not get accepted at first, but you will learn a lot along the way and maybe one day you will have a change submitted that gets accepted.
• Stretch Goal - Contract Program. Are you good enough that you can actually go out and take on small contracts? Maybe build a web site or a mobile app and get paid for it? This is not for everyone – you better be able to back up your claim as a developer and be able to deliver within the constraints given by the company that your taking on the effort for. There are plenty of web sites out there that allow you to bid on contract jobs.
• Stretch Goal - Volunteer Programming. Find a non-profit that needs help. These small non-profits in your community typically need help to track donors, track the details of programs they are running. Can you help them out

Use Git: As a student, you have the ability to use Git Education for free and get a bundle of software/tools that will help you as a developer. If you don’t know, Git is a versioning system that allows you to manage changes to your code and allows you to recover code when you break something. Load your assignments into Git, load the projects you created for yourself into Git. Do not share code you created under contract – that is owned by the company that paid you. Use this code repository as a portfolio that you can make available to hiring managers to show them your capabilities. Be prepared to talk about these projects in the interview. Which ones were assigned projects, which ones did you take on just to learn and explore? What choices did you face along the way? How did you resolve issues? How did you test?

Understand Testing: Whether you work in an Agile or Waterfall environment, you need to understand the components of software development and the responsibilities on the person creating the code. I’m over simplifying here, but you should be able to talk thru how you design, how you plan for testing, how you code and how you test. If you don’t understand why you should figure out how you’ll test code before you actually write the code, repeat this step as often as needed. Truly gifted developers understand the need to plan their test strategy and write their tests before they code.

Join your schools Computer Club: Take on an active role in the club. Become an officer, help run events. Has the club sponsored a hacking event? Can the club sponsor a hacking event? Can the club take part in competitions at other schools? Be able to talk about your experiences in the club, what you did with the club how you helped the club meet its mission.

Establish Relationships with your Professors/Instructors: Everyone is going to need to have a set of references when they apply. As an applicant for a job, your ability to identify instructors that you’ve had along the way who will speak positively about your participation in class and with the assigned work is a net positive for you. Remember, people like myself have relationships with many of the local instructors and we’ll ask them about your participation in class, how you worked in collaboration with other students on team projects. Many times, I’ll have instructors send me a note or stop me after I’ve met with their students and they’ll tell me students that I should get in touch with, who are the ones going above and beyond on their assignments, who works well in class and is always paying attention and helping others out.

Research Companies: As a Freshman, start figuring out what kinds of companies you want to work for after you get out of college. Do you want to work for large multi-national companies? Do you want to work for a smaller company? Do you want to live in the country? Do you want to live in a large metro area? Do you want the opportunity to telecommute? Start creating a list of companies that you’re interested in and make sure you look for opportunities to connect with those companies. Follow them on LinkedIn. Regularly visit their web site to see what the company is doing. Follow them in news articles to understand what they are doing. If they have a recruiting department – connect with their recruiters or with people in HR via LinkedIn. Find out thru your research what skills and experiences that they want students to have. As an example, one of the companies that my son eventually wants to work for leans towards hiring students with collegiate research experience.

Attend All Job Fairs: Even as a Freshman in college, you should be going to job fairs to introduce yourself to potential employers. Let them know upfront that you’re a Freshman and understand that you’re probably not eligible for internships, but that you’d like to learn about their company, what they are looking for and ask them how to stand out as an applicant. Connect with corporate recruiters from these job fairs via LinkedIn. Send them a note after the job fair telling them what you heard about their company and what they are looking for and then let the recruiter know that you’ll keep them updated on your progress. Then actually stay in touch as you move thru your educational career.

Use LinkedIn: Do not wait – as a Freshman, create a LinkedIn account. Begin following companies you are interested in. Begin following people at the companies you are interested in. Specifically, follow the companies you are interested in, find managers that might potentially hire you someday, find the recruiters in the company and connect with people in HR. Build these connections so that you can see what they are posting. When you see something posted that you can contribute to in a meaningful way, jump into the conversation. Don’t reply to a post just to reply, be thoughtful and add to the discussion.

I’m not saying every student should hit every one of the above items, but if you are just going to class, doing the bare minimum to get buy on homework, that is not how you’ll get noticed by employers and find internship opportunities, let alone job opportunities when you graduate. If this is going to be your career, take it seriously and invest the time needed to not only tackle your education, but to prepare yourself for when you want to get hired!

If you'd like more information on my background: LinkedIn Profile