Back to Basics - Data

Two things happened recently that brought me to the point of writing this post …

1. I was meeting with a candidate for an open role inside my organization and the individual began discussing how he tackles design and development. My ears perked up when I heard him mention that he creates a data map to understand what needs to happen to individual pieces of information as they move thru a proposed solution. Only when he understands the data does he feel he can make a judgement on whether the proposed solution will work or feel that he can properly define a solution if one doesn’t exist.
2. I’ve been playing with a small code concept with a couple of buddies outside of work – we are all doing this just to explore some concepts and because at heart we still like to geek out. Amazing what happens when you're a coder, but you can’t touch the keys during your day job! Anyhow, we were having a chat about our little app concept and immediately all 3 of us began tracking the data elements we would need and marrying these data elements up to potential screen flows. The discussion always came back to the data, what data would be available at any given point and what manipulation of the data would need to take place between different visual aspects that the user would have access to in the app.

As many of you have read in previous columns, I’m a firm believer in understanding data. In my opinion, if you don’t understand the most discrete data elements being used by an application – where they came from, how they are being manipulated, what the contract is between functional elements of code and how they are presented to the user – you will never have a true grasp of the problem you are attempting to solve. And if you don’ t understand the problem, how do you test for and claim success! So how do we, as professionals, ensure that we understand these discreet element, how can we use that information to harden our applications and ensure that we are producing rock solid systems?

Data elements can be broken into 2 distinct types that have meaning to a developer – stored (permanent) data elements and calculated (transient) data elements. Stored data elements are those pieces of information that are physically stored somewhere that the program can read from at any time and can write to as needed to update the value. Calculated data elements are transient, at some point in the program the values are identified (calculated), used and then discarded.

Stored Data Elements (Examples)

1. Application Configuration Information: Service API Locations, Authentication Keys, Application Rules
2. Multi-Tenant Configuration Information: Images, Application Rule Overrides (Customizations), Colors
3. User Configuration Information: Colors, Alerts
4. Data Retained in a Database: Customer Tables, Invoice Tables, Inventory Tables, Indexes, Relationships, Views

Calculated Data Elements (examples)

1. Days until loan payment due – calculated from stored Date Due
2. Forcing a string value to upper case for comparison and/or display – uses stored values but does not need to be stored itself
3. Calculating the payoff amount of an open loan if paid today – uses stored values but does not need to be stored anywhere

As a developer, it is our role to understand how each distinct data element is received, what manipulation needs to take place before we display the value, store the value back, pass the data on to the next function or used in another calculation. You need to understand the boundaries of what is acceptable and you need to plan for exceptions when the data does not fit within the defined boundaries, does not exist or causes the system to respond in an unplanned way.

For my purposes, I take data mapping beyond the typical definition used in development. Most people will refer to data mapping as the exercise needed to understand the underlying data schema that an application utilizes – being very simplistic the database(s), table(s) and field(s) needed by the application. I have always extended this definition. To me data mapping includes the following information:

Data Schema – the database(s), table(s) and column(s)/field(s) touched by the application. It will include key information needed by the technical team – architects, leads, developers, quality analysts and test automation:

1. Database configuration information: Server Location, Port Information, Access Authentication Information
2. Table configuration information: Table Information, Relationship Information, Index Information, Column(s)/Field(s) Information (including constraints, defaults, boundaries, lists)
3. Views needed to support the display and manipulation of data within the application

Application Configuration Information – this is the stuff that allows the application to ‘prep’ itself, make all the connections it needs to various databases, services, file systems so that it can run. Typically this information is stored in the file systems – Config, Parameter, JSON, YAML, XML, etc.

1. Where is the email server and the associated connection information that you need so your app can send emails? Server names, ip address, authentication information.
2. Where is the message service and the associated connection information that you need so your app can send SMS messages? Service address, authentication information.
3. Where is the logging service and the associated connection information that you need so that your app can log errors? Server names, ip address, authentication information.
4. What are the default rules used by the application? Sometimes instead of hardcoding logic of how a piece of data will be manipulated, you define rules that the application will read and then use to process the data.
5. What are the internal server addresses and connection information that the application needs to provide all planned functionality. Server names, ip address, authentication information.

Multi-Tenant Configuration Information – this is the stuff that allows the application to understand what ‘organization’ the user is accessing so that data is segmented properly and that the correct UX Schema and Functionality is made available. This information can be stored in the database or alternatively can be stored in the file systems – Config, Parameter, JSON, YAML, XML, etc. My personal preference is to store this stuff in the file system, not in the database, but each developer has their own feel for how they will store and manipulate this data.

1. What are the default color schemes used in the UI? If a ‘tenant’ wants to override the colors, what colors do they want?
2. Where are the default images used in the UI? If a ‘tenant’ wants to override them with their own images, where are they located?
3. Where are the tenant application logging files going to be kept?
4. Is there going to be a difference in what gets logged for specific ‘tenants’
5. Can an administrator turn on/off functionality for a specific ‘tenant’ – if so, this needs to get recorded somewhere so the system knows how to behave.
6. What are the overrides for the Application Configuration Information? File Store changes, authentication keys, database specific configs, default rule overrides.

User Specific Configuration Information – this is the stuff that allow us to specifically configure the user experience, menu items, colors. Most people keep this stuff in the database, but it is possible to keep this in the file system – just remember, you need to be able to scale for the number of users you expect to access the system.

1. Can the user override color schemes?
2. Does the system allow the user or an administrator to customize the menu options available to the user?

Calculated Elements

1. Need to understand where in the app that calculated values will be used.
2. Document the calculated values that the system will need.
3. Example: Current Loan Payoff Value – the system stores the following information: Current Loan Outstanding Amount, Loan Interest Rate. The system will calculate the following information: Daily Interest, Number of Days Interest In Current Period, Current Interest Due, Current Loan Payoff Amount
4. Example: Average Sales for Current Month – the system stores the following information: Date of Invoice, Invoice Total Amount Due. The system will calculate the following information: Sum of Invoice Total Amount Due, Count of Invoices For Current Month, Average Sale for Current Month

What I’ve discussed above is at the 50k foot view. Specifically on the database design stuff – as you begin to utilize apps out there that allow you to describe your databases, tables, columns, relationships – you’ll see that there are a lot of additional descriptive elements that these programs allow you to document. You’ll need to decide how much of that functionality you take advantage of, what value it brings to the team and for future support purposes. Remember, think thru how someone will maintain your design and code 18 months from now and what appropriate information that they will need to make their lives easier.

As you are sketching out your thoughts on your application, this is the stuff that will ensure that you get it right. That you understand what is happening as the data moves thru your system and then how you can test the application to ensure that you’re handling the data properly. If you are the total team – then you can keep track of this stuff with open source tools and/or spreadsheets. If you’re working with a team – get agreement upfront on the tools you’re going to use to manage and share this information – whether they are proprietary tools or open source tools.

All of the above are living elements that need to be continuously updated throughout the life of the development cycle and the application lifecycle. As you actually get into the application design and layout the specific logic flows associated with user interactions, you will most likely end up changing information at all of the layers discussed – modifying and changing tables/columns as well as potentially adding new tables/columns. Your configuration specific data and rules data will also change – keep the documentation up to date so that when you have to go back and support it months or years after it was originally developed, you’ll understand what all of the discreet data elements are, why they are there, how they are used, where they are used and what calculations are used to manipulate the data.

As you work thru identifying the above information, you’ll naturally begin to identify the class definitions you’ll need and a skeleton of the properties and methods that will make up the class. Document as much of this as possible up front, it will save you countless hours as you not only create the app initially, but more importantly as you have to come  back and maintain the app.

If you'd like more information on my background: LinkedIn Profile

Software Development - Attracting The Next Generation

I continue to enjoy my career - having worked for, and been mentored by some very talented people along the way! Today, I’m am lucky to work with an incredibly gifted group of professionals. From the team members that work directly within my organization to the team members that we support and interact with every day, this is a group of people committed to making a difference for our the financial institutions, merchants, fintechs that we do business with daily and the card holders trying to manage and spend their money. On a daily basis, we all get to see the impact we have as we help our customers manage their business and assist consumers making purchases across the world.

I am also lucky enough that I’m able to spend time with students across the Central Iowa area, promoting careers in STEM (in particular, careers related to software development and quality assurance). This ranges from kids as young as 10 to students getting ready to graduate from college. This is the next generation that will drive innovation – data analytics, artificial intelligence, serverless technologies, containers, virtual reality and more that I/we haven’t thought of yet.

Sometimes I get direct 1-on-1 interaction with students and at other times it is speaking to groups of 50 students. I enjoy the opportunity to talk about what make the industry tick and what the individual opportunities are within these fields. There are things these young minds need to know:

1. Software Development is an opportunity to have incredible impact on the organizations and the consumers of the technology you build! As the cost of entry to the technology has come down over the years, people have access to systems and technology that are fundamentally changing the way that people live their lives. The most recent example is the Apple Watch 4 and the built in ECG capabilities that are already saving lives.
2. Software Development provides opportunities every single day! This is not a job where you do the same things day after day. If you want variety, this is the role for you. Being a developer gives you the ability to creatively solve problems every day and those problems will be different. You will not be sitting in an office looking at the same types of reports and data every day and making similar decisions, you will be solving unique problems and challenging yourself.
3. Software Development provides unique opportunities to grow and advance within an organization! As a developer, you will be exposed to and have the opportunity to drive change within the organizations where you work. These experiences give you the unique ability to understand the business and to contribute in a way that others may not be able to see. Your ability to see ‘connection points’ that might be lost on others gives you the ability to recommend and make changes that drive efficiency, improve the customer experience and increase the profitability of the organization.
4. Software Development means you will be in demand! There is a current shortage of software developers, this is a trend that will get worse as the baby boom generation continues to retire and leave positions open. This means companies need you! If you need any proof of this, go out and look at all the openings for software developers – they are literally everywhere. According to the Bureau of Labor Statistics, there will be a 24% growth in the overall number of software development jobs thru 2026.
5. Software Development gives you flexibility! More and more companies are allowing their software development team members to work remotely – if not full time for at least part of their regular work schedule. If you can prove you have the ability to work independently and still drive to milestones and hold yourself accountable, companies will provide you more flexibility. In fact, traditional ‘coastal’ technology companies have recently announced initiatives to train and hire workers in rural Iowa. This trend will continue and developers will find opportunities to work wherever they choose to live.
6. Software Development pays well! As a software developer, you will need to establish your bonafides, once you have proven you are capable, you will be rewarded. Software development is one of the higher paying professions and will give you the ability to live comfortably. Current average national salary for entry level software developers is $68K. Yes, depending on location, that number will be higher or lower, but it is nothing to sneeze at when comparing to other jobs. And, software developer are ranked as the #1 in the top 100 jobs nationwide, as well as the #1 STEM Job.

To be a really good software engineer, means to understand the above and then choose to passionately pursue this career choice. Being a really good developer means that you are eager to explore new technologies, learn new development techniques, invent new ways of doing things and to be creative. It means taking all of that, and using it to work on different projects – no day will ever be the same. It means dedicating yourself to making the time to work on smaller side projects to explore and hone your skills.

We should be encouraging students as early as possible to explore software development. Kids in grade school can start with on-line tools like Scratch from MIT (https://scratch.mit.edu/). If they get comfortable with Scratch, then they can move up to Python, Ruby or even JavaScript. With the technology available today, there is nothing to stop them from learning. Online manuals and tutorials, online forums where they can ask questions and be mentored.

This generation has access to so much information and it’s all available at their finger tips thru their smart phones, laptops and computers. Let’s encourage them to explore opportunities where they can have fun, make a difference doing something they love and make a nice salary along the way!

If you'd like more information on my background: LinkedIn Profile

The Education of Software Developers

When I first entered the ‘professional workforce’ years ago as a software developer, it was not a requirement to have a 4 year degree to be able to get a job. Several things have changed along the way that have made it more difficult for high school students to get ‘formal educational’ training and to then be able to go out and successfully apply for and land a software development job. I don’t believe all of these were good things.

1. Many high schools that used to teach computer science (programming courses) stopped doing so.
2. State educational departments made rules that disallowed many teachers who had been teaching programming from teaching those classes at high schools
3. State educational departments made rules that disallowed many working professionals from teaching computer science at community colleges

Combined, these rules limited the ability for students to explore software development as a profession and limited the ways in which students could get formal education on how to become a software developer.

On the plus side, several things have happened that have made it easier for young students to explore software development. If only these things would have been around when I was a kid:

1. The internet … boom!
2. Open source development tools – languages, compilers, frameworks, editors, IDEs, test automation frameworks
3. On-Line tutorials
4. On-Line college level programming courses
5. On-Line help forums
6. GIT – online source control and sharing of code
7. Bootcamps teaching programming (primarily Web and Java)
8. Scratch online programming educational tool for students in K-8 – developed by MIT Media Lab
9. Lego Mindstorm competitions

The problem, we don’t talk about the learning opportunities early enough in school to attract students to the idea that they can become software developers. In many school districts, students have no insight into any of this until they graduate high school. That is too late. We need to be exposing kids to the idea of software development very early in their educational career – think grade school. Then we need to reinforce those opportunities with additional educational opportunities thru middle school and high school. The goal should be to allow those interested in pursuing this to potentially get out of high school and find entry level software development roles. Not every student would be able to do this, some will need additional education, or may want to pursue additional education to perform within specialized areas – artificial intelligence, data analytics, etc.

I would also argue that we need to rearrange graduation requirements at the high school level – computer science classes should be counted as part of the core curriculum, either as language classes replacing Spanish, French, German, etc., or as math and science classes. Additionally, every high school should be offering computer science classes – either locally or thru remote class arrangements with the nearest community college. I know of high schools teaching Java, C++, C#, and networking (providing CISCO certifications at the end of the class). These kids are old enough to understand these concepts and get a jump start on their post high school education or prepare themselves for direct entry into the workforce. There is no reason not to provide these learning opportunities and have students work ready for entry level programming positions by the end of high school.

Can it be done? Yes, even with the constraints that were in place when I was a kid – no internet, no open source software, no PC’s, I was able to learn enough that I was doing contract programming by the time I was in high school. I personally have hired kids that were graduating from high school that showed they had the chops and the passion to be a developer. Does it take a little extra hand holding? Yep – it sure does, but it’s worth it.

I’m not trying to devalue a college education. There are reasons to pursue a 4 year degree, but we also must be cognizant of the challenges we face today as employers:

1. There is currently a shortage of skilled software developers that will only get worse as the baby boomers retire – some sites are reporting a shortage of at least 220,000 software developers in the US.
2. Currently colleges are graduating approximately 30,000 computer science majors each year.
3. Unemployment for folks in the software development role is currently running around 1.9% (mid 2018 stat)

We need to think outside the box on what we are doing and how we are making this career path accessible to young students. The big boys on the block are starting to change their hiring methods – no longer requiring degrees. 

•         IBM, Amazon and Microsoft Use Apprenticeships
•         Idaho Offering Code to Career Apprenticeships
•         Silicon Valley Looks To Iowa

Be nimble, look for ways to help out your local schools, find ways to let kids explore opportunities in software development. Encourage your kids to go online and find coding learning opportunities. If we don’t figure it out, we are going to be at a disadvantage as we attempt to find bodies to fill the jobs that we need to be successful.

If you'd like more information on my background: LinkedIn Profile

Our Future in Software Development

My profession has changed significantly over the years. I entered this profession back when I was a teen – doing contract jobs for companies near where I lived. Mind you, during that time, the internet did not exist and contracting meant going on-site to the company that needed my services and actually working on their equipment. There was no PC being used in business back then – the Apple II and the TRS-80 were state of the art for enthusiasts, but rarely used in business. Even if a company did have an Apple II, it was limited to single station processing as there was no such thing as a network, wired or wireless. I personally never witnessed anyone using a TRS-80 for business. I’m not saying it didn’t happen, I just never witnessed it.

Yep, we were carving symbols out on rocks!

I still remember the day a friend of mine got an Apple II computer. We were stunned that he had this in his house and was able to create programs in BASIC and store them to a floppy disk. Not only that, but he create a program that he could run to actually open and close the shades in his bedroom. How cool! Another buddy of mine had a TRS-80 and the storage system on his computer was a cassette drive – for those that don’t remember, this was a painfully slow way to load data/programs into memory. Additionally, the TRS-80 regularly lost information on the cassette tapes. It was not uncommon that you had to store something on 3 or more different cassettes to actually ensure that you had a copy that worked.

Today, I wear more computer power on my wrist then what was available on the mini-computers that I was programming back in the day. Seriously, my Apple Watch 3 has 768MB of RAM and 16GB of storage. The S2 processor on the Apple Watch 3 runs at 780 Mhz. As a reference the IBM 5110, which was the first computer that I professionally programmed on, had 32K RAM (could be expanded to 64K), with two 1.2Mb floppy drives and a processor that ran at 1.9Mhz. The IBM 5110 had a retail price of $10-20k depending on options, it had a 5 inch black and white screen (16 lines of 64 characters) and weighed over 40 pounds. And yet, somehow we made things work!

My how times have changed!

One of the first professional programs that I wrote managed a customer list for a business using the IBM 5110. The customer information was spread out over multiple 8” floppy disks (side note: I still have copies of the original programs and data on the 8” floppies – not sure I could even find the equipment to read them at this point). During processing of the list, the computer would prompt you to change the floppy disks so that it could continue processing the data. It was clumsy to say the least and meant that someone had to babysit the programs when they were running, changing disks back and forth as needed. And to complicate matters further, there was no such thing as a database – your data was written to fixed length ASCII files and you had to manually read/write and manipulate the data off the floppy disks. All those fancy things that SQL does for you in retrieving, manipulating, sorting, summarizing and managing data, had to be done by hand. I remember spending huge chunks of time writing generic BASIC PROCs to handle common functions that can now be done with a single line of SQL.

Creating the programs was interesting to say the least – there was no internet to look things up or to turn to when you needed help. You pulled out the programming manuals from IBM and you were on your own to plow thru the manuals and piece information together. The IBM 5110 could be programmed either in APL or BASIC – you literally had to flip a switch on the front of the computer to tell it whether the programs you were running were written in APL or BASIC. I wrote the programs in BASIC and they were not structured – lots of GOTO and load PROC statements. I think back on how I created those programs now and shivers run up and down my spine. But, that was just the way it was done. That was cutting edge back then – remember, the IBM 5110 was the precursor to the IBM PC.

If you want a blast from the past, check out the IBM Reference Manual for creating BASIC programs on the IBM 5110: http://www.bitsavers.org/pdf/ibm/5110/SA21-9308-0_IBM_5110_BASIC_Reference_Manual_Jan78.pdf

Today, I don’t program during the day. My role is different, and that’s ok. I’m leading 120+ team members, ranging from our Enterprise Architect and Tech Leads, to Software Engineers, to Quality Assurance Analysts and Test Automation Teams. So my coding is limited to my playing around at night to keep my skillset relevant and to ensure that I can still engage in technical conversations with my teams. Occasionally, I will do some volunteer work – coding where I can make a difference for a non-profit or helping out when a business owned within my family needs some coding done.

Lately, I’ve been creating an application using Ruby on Rails with the backend database running on MySQL. What a leap from the old days! Ruby, used correctly, is a dynamic, interpreted, object oriented programming language. The addition of Rails provides a full Model-View-Controller framework that can be used to quickly build sophisticated web applications. It provides the underlying framework for accessing various databases, providing the web services layer and generating the web pages for the application. What used to take me hours or days to accomplish writing out programs on the IBM 5110 I can now do in a matter of minutes. I spend a few minutes describing an underlying table, then I tell Rails to generate the scaffolding around that table to create a base controller and view that I can then tweak that controls all of the standard Create-Read-Update-Delete (CRUD) activity. The screens generated are not pretty, but they get the job done and I can then go back in and tweak them to give them the look and feel needed by the app. I can also tweak the controller to add additional business logic where it is required to meet the needs of the business.

I’ve also been playing around with a personal application I’m writing for the iPhone and Apple Watch. I’m using Swift for that particular effort. I’m conflicted here, I love Swift, but I absolutely detest the Apple tool to create the UI/UX. The tool included in Apples development environment is not intuitive and is over complicated. I consistently get that feedback from other developers that I talk to. They love Swift, but can’t stand the way that you have to create the UI/UX for an app. In fact, I have to say that my distaste for the UI/UX development interface has caused me to walk away from the app several times, I do eventually come back to it because I really want to make this app work, but I’m looking forward to third party tools that may allow me to replace the Apple UI/UX builder. (Yesterday, Google announced that Vesion 1.0 of Flutter is now available, providing a write once deploy to both Android/iOS tool – this may be my option, although the Watch interface will still most likely have to be done in Swift.)

Think about all of this – the first programs I wrote back in the day were massive, thousands of lines of code written in a messy BASIC with undecipherable GOTO or LOAD proc commands sprinkled liberally thru the code. Additionally, variable names were limited in size and were generally cryptic – where today, you can have variable names that are very descriptive and don’t leave the next person touching the code guessing as to what a variable is being used for in the program.

Time has allowed us to mature how we approach software development. We now have languages, frameworks, tools and integrated development environments that allow us to create code that is easier to read, is more elegant in the way that it handles data and processing and also is easier for the next person to pick up, understand and modify. We have developed tools to share code that is being developed and that allows groups of distributed developers to easily understand what has changed since the last time they pulled a copy of the code – I remember sharing code by exchanging floppy disks, yikes! We have backend data storage SQL/NoSQL that allow developers to store millions of records and quickly access, manipulate and change the data. We have developed new languages that allow us to do things with a few lines of code that used to take pages of code to accomplish. The code is now written using Object Oriented models that allow us to more accurately describe the things that an application is manipulating and gives developers a better way to create and manipulate those things. We have moved to development paradigms that make us more productive – test driven development, WAGILE (Agile-Waterfall Hybrid), Agile. We have introduced tools that allow developers in disparate locations to interact and solve problems in real time – SLACK anyone?

As a profession we need to continue to challenge ourselves in how we create software. Ultimately, it is about solving real problems for people. I don’t necessarily care how sexy a new tool/language/framework is, if it doesn’t allow me to streamline how I design, code, test and deliver the solution to the customer, I have a tough time justifying using it either personally or within the teams I am lucky enough to lead.

The current state of software development has advanced significantly from the late 1970’s and early 1980’s. That said, we can’t afford to stand still, we must understand the new waves that are heading our way. How will machine learning and artificial intelligence allow us to change the way we design, develop, test and deliver software. Will AI augment or replace roles within our teams? What pressures will the cloud continue to put on us? Today you can spin up a company without having to purchase the underlying hardware to deliver your services – you can use AWS and do a pay as you go based on the actual usage of your application. Where does it make sense to put infrastructure in the cloud? Has your organization begun using containers to deliver applications and services? How is your organization dealing with the pressure to deliver features faster – sprints anyone?

Last year, over Christmas, my son wanted to explore Artificial Intelligence. So my wife and I purchased several books on the topic. After opening these on Christmas, he holed up in his room one day and came down proclaiming at the end of the evening that he had built his first AI engine to recognize numbers in images. Uhm, what? Over the summer, before entering his first year of college, he took on-line courses to further his knowledge of machine learning. He as able to do all of this by reading some books, going on-line to do research and get questions answered and then taking some on-line courses to push himself. He is much further along the curve than I was, and has access to tools that I could only dream of. His generation will be the ones leading the wave of new paradigms into the companies we all work for today. What change will they bring with them and are we preparing ourselves?

I’m excited to see where the future takes us as a profession and how it will change what we do as developers. One of my challenges will continue to be to identify how we integrate these new ‘things’ into the legacy applications we support (and, yes, that means all that fancy code you are writing right now will be legacy code that has to be retrofit tomorrow)! 

Welcome to the future …

If you'd like more information on my background: LinkedIn Profile

Follow The Data ... Software Design, Development and Testing

‘Follow the data!’ I have preached this for years. If you truly want to be a great developer, you learn the data flows!

As a software engineer, we want to be recognized as exceptional in delivering code that functionally delivers against the user needs and doesn’t introduce performance issues or defects into the production environment. We want our code to elegantly solve the problem that has been described and delights the user. If the code that we develop is a smaller slice of an overall application or platform, we want to ensure that our integration points are rock solid, that we are doing nothing to impact the ability of the larger application or platform to successfully process and perform the function as it was designed.

It is critical that as a software development professional, we teach ourselves how data flows thru the system. How does the data flow into the application function that we are developing, how does the data get manipulated by the function we are creating and what happens to the data when it leaves our function. This means breaking the data down into its discreet elements and understanding how they are manipulated ahead of us, what we are going to do with them as we execute the new or changed functionality and what happens after it leaves us.

Let’s take a small scenarios (and I’m oversimplifying the scenario to make the point) … a Cardholder walks up to an ATM and inserts their card into the machine to request a $100 withdrawl. Behind the scenes the machine captures information off of the card to send to the financial institutions Processor. The Processor receives the information off the card and checks the balance in the bank account and spits out the $100 to the Cardholder.  Mission accomplished – right? Nope!

The ATM in the above scenario is not owned by the financial institution where the Cardholder has their money deposited. So, somehow we have to settle the money between the two financial institutions. The withdrawl is one distinct component of the overall transaction. We need to follow the data from the original transaction to the end of the day when the money actually moves between the financial institution that dispensed the $100 from the ATM they owned and the financial institution that holds the Cardholders funds. Whew, done! Not exactly.

At the end of the month, the Processor must bill for the services provided to the financial institution that holds the Cardholders funds. There are specific transaction fees that get charged based on volume.
Now, let’s think about the software engineer that works for the Processor. Not only do they have to see the data in real-time and make authorization decisions, but they need to move the money at the end of the day and ensure that the financial institution is billed properly at the end of the month.

To follow the data, the software engineer needs to understand what data is read by the card reader at the ATM and is transmitted into the data center for authorization. They must understand how to match the card data up to the financial institution cardholders account to check balances, they must understand the authorization cycle and how the individual data elements in the card data are used to detect fraud and either accept or decline the transaction, they must understand how that data moves thru the system and is stored to ensure that the money moves between financial institutions at the end of the day, and they must understand how the data is used at the end of the month to properly bill the financial institution based on the type of transaction and other factors.

The software engineer cannot be sure that the functionality that they changed is working properly until they’ve tested the data all the way thru the transactions lifecycle to ensure that it settled at the end of the day and that the financial institution was billed properly at the end of the month. Testing should be designed with the full data flow in mind, not just the partial discreet transaction where the money was dispersed to the Cardholder. Again, I’ve oversimplified, but this scenario does describe the breadth of understanding that a developer must have before they can claim success.

Now add in some side scenarios – let’s say the data is used in web and mobile applications where the Cardholder can view transaction history data and initiate fraud claims. Those data usage scenarios introduce additional data flows that the software engineer must understand and validate before they can claim success.

Understanding how the data flows, where it is stored, how it gets manipulated prior to, during and after the function you are working on, is critical to understanding whether you actually can design, develop, test and deliver the functionality correctly. Knowing how to code is the simple part, understanding how the data moves and is manipulated is the hard part.

If you'd like more information on my background: LinkedIn Profile

Code Reviews - A Must For Any Team!

As professional coders, we should be concerned about the cost to remediate ‘bad requirements’ or ‘defects introduced into the code’ thru the development process. Over the years studies have shown that there is a significant increase in cost to remediate these code issues the deeper into the development lifecycle that you go. One of the most recent published studies completed by NASA broke it down as follows:

Phase	Cost to Remediate
Requirements	1x
Design	5x - 7x
Build	10x - 26x
Testing	50x - 177x
Production	100x - 1000x

In other words, if a bad requirement is not caught before design, by the time it makes it all the way into the production environment, there is a cost of 100x to 1000x the original cost to remediate the requirement. Yikes! Now, if a developer makes an error in building the feature/function described in the requirements or introduces a defect into the code unrelated to the feature/function being built, the cost to remediate that when found in production would be 10x to 100x if it was caught and remediated during the build process.

From a pride perspective, no coder wants to be seen as the one that introduced the bad code or broke current functionality. Beyond the pride factor, though, there are real hard costs being spent by organizations to remediate these issues, as well as the reputational impact with customers.

There are things that developers can do to lower the risk of introducing defects into the code base and to limit the exposure of the additional costs needed to remediate these bugs. Code Reviews are a proven paradigm that developers can use to catch these issues early and to remediate them during the build process.

Code Reviews: The intentional and systemic check of code by other developers to identify and remediate mistakes or security threats. The practice has repeatedly shown that it can accelerate and streamline the software development process and reduce the number of defects that reach production. In its simplest terms, a code review simply means that the person developing the code shares that code with other developers who review the code to:

• -        Ensure that team coding standards are followed
• -        Look for errors that can be remediated.
• -        Look for ways to ensure the integrity and security stance of the application
• -        Look for ways to simplify algorithms
• -        Look for ways to remove code that is not needed

As much as we like to think we aren’t the ones that introduce defects into the code base, we are humans and therefore the code will contain errors – those errors may be small, or they could be significant show stoppers. As professionals, we should acknowledge that fact and look for ways to mitigate these issues early. And that, my friends, is why Code Reviews became a thing!

When Code Reviews are done properly, they will address all of the above items, they will reduce time needed in later stages of the lifecycle, they will deliver hardened code into the production environment, they will reduce ongoing maintenance costs, they will foster communications within the development team, they will provide a means for more experienced engineers to mentor those engineers not as familiar with the code and they will ultimately increase the confidence and trust of the customers using the application. With all the positive impacts that code reviews can have for the developer, the team, the organization and our customers, why wouldn’t we use this well-defined paradigm to improve the quality of what we deliver downstream and ultimately to our customers.

Let’s quick review how this process looks:

1. Code Submitter (the software engineer changing the code) checks the code into the repository and makes the Code Reviewer(s) aware of the code requesting a review take place.
2. Code Reviewer opens up the code and begins to check the code:
3. Does the code do what it is actually supposed to do based on the requirements/user stories?
4. Looking for obvious defects in the code that might produce errant results.
5. Does the code adhere to the established coding guidelines that the team uses (there are tools that can automate this)
1. Code Formatting
2. Variable Naming Conventions
6. Is there Automated Test Coverage for the code being reviewed?
1. Have the tests been written in a way that is easily understood and provides clear results?
2. Have the tests been written to provide red route/green route testing – is the sample of tests broad enough to fully test the functionality?
7. Does the code adhere to the security/regulatory guidelines used by the team (OWASP for web)?
8. Does the code fit into the overall architecture of the application?
9. Does the code introduce potential performance issues that need to be addressed.
10. Is the code in the right place, are the services and methods grouped into the right places to easily be found by other developers?
11. Did the code introduce ‘duplicate functionality’ that should be removed?
12. Has the code been written in a way that is easily understood?
13. When the new code fails, do the exception messages provide clear direction on what failed and why?
14. Reviewing complex algorithms to see if they can be simplified.
15. Identifying chunks of code that are not needed anymore that should be removed.
16. Code Reviewer identifies recommended changes to the code that was submitted and documents the changes for the Code Submitter.
17. Code Reviewer discusses the recommended changes to the code with the Code Submitter in a respectful manner.
18. Code Submitter makes agreed upon changes to the Code.
19. Code Submitter circles back to Step 1.

That’s a lot of stuff! But the benefits are huge if done correctly. The beauty, I’ve seen interns and new engineers perform successful code reviews and give feedback to seasoned engineers, just as I’ve seen our most experienced engineers give guidance to our newest team members. This process in itself fosters communication within the team and opens up doors that allows the team as a whole to raise the bar and create high quality code. This becomes the expectation within the team, nobody wants to be seen as the coder that introduced a bug, they all want to be seen as experts in their chosen profession. Additionally, it gets developers talking to each other and opens up opportunities not seen in the code review process where they will begin to discuss challenges in real-time as the code is being created and not wait for the code review process to catch the issue.

I’m not saying that this will catch everything, again, we’re all human. But by using code reviews properly, we enhance our ability to catch issues early and remediate them before they impact our user community and potentially damage our reputation.

If you'd like more information on my background: LinkedIn Profile

Students - Getting Noticed By Hiring Managers!

One of the great parts of my role here at SHAZAM is the time that I’m able to take to go out and visit with many of the local middle school, high school and collegiate students interested in careers in technology. My being able to talk about the ins and outs of this industry, how it has kept me engaged over my career and to get feedback from these students lifts me up. I enjoy talking to them about the opportunities that they will have and the changes that they will be able to make along the way. 

Invariably, during these discussions, someone will ask, “but, how do I get noticed?”

Great question, how do you get noticed among the many students graduating and entering the work force. Let me walk thru what I tell students, and by the way, I’ve given this same advice to my youngest son, as he is pursuing a career in technology. This will be focused on software development and test automation.

First Focus on Your Grades: The grades you earn are important for that first professional job as you enter the working world. It is an indicator that you paid attention and took the time to understand the material presented in the classroom and that you were able to translate that into working knowledge on individual and group assignments. While a low grade will not be a disqualifier in the process, it is a barrier that you’ll need to explain. Show me where you went above and beyond on classroom assignments. Did you ever take on an extra project and share it with the class or the instructor?

Programming Experience: Doing your homework is the bare minimum. You’re going to need to explain group projects, your role in those projects, how you overcame conflict in the group and how you got people to work together. If you want a step up on other candidates that your competing with, you should look to do projects outside the classroom. Let me give you some examples:

• Expand your knowledge. Find a technology that you want to play with and start to learn it. You may not become an expert on it, but it is going to teach you knew concepts and new techniques. Are you interested in big data – show me something that backs up that statement? What about machine learning – instead of just saying you want to learn it, have you taught yourself Python and PyTorch. Even if you haven’t successfully gotten your first engine up and running, what have you learned, what can you show me, what challenges have you encountered?
• Recreate a game you like. Bring that with you to the interview and talk about the choices you had to make, what you learned from the experience and how it made you better as a developer. My son, one day announced that he was going to recreate the game ‘2048’ on his own and to one up the normal game, he was going to create an AI behind the game that would play it for him. Mind you, this was during his first semester at college, he was taking general education classes and wanted something technical to play with.
• Build a mobile app. We are all carrying around smart phones. The cost of entry for development is $0 – you can download the software development kits off the internet. A couple of years ago, we were looking to hire some new team members. I remember the hiring manager walking into my office after one of the interviews talking about how they had just completed an interview with a candidate. He walked into the interview pulled out his smart phone and started talking about an app that he had just created. So, this candidate showed he was passionate about coding, that he geeked out along with the best of them and that he could learn new technology. Winner!
• Stretch Goal – Contribute to an Open Source Project. Find an open source project that you care about, study the code and start submitting changes to correct features that aren’t working correctly. Your changes might not get accepted at first, but you will learn a lot along the way and maybe one day you will have a change submitted that gets accepted.
• Stretch Goal - Contract Program. Are you good enough that you can actually go out and take on small contracts? Maybe build a web site or a mobile app and get paid for it? This is not for everyone – you better be able to back up your claim as a developer and be able to deliver within the constraints given by the company that your taking on the effort for. There are plenty of web sites out there that allow you to bid on contract jobs.
• Stretch Goal - Volunteer Programming. Find a non-profit that needs help. These small non-profits in your community typically need help to track donors, track the details of programs they are running. Can you help them out

Use Git: As a student, you have the ability to use Git Education for free and get a bundle of software/tools that will help you as a developer. If you don’t know, Git is a versioning system that allows you to manage changes to your code and allows you to recover code when you break something. Load your assignments into Git, load the projects you created for yourself into Git. Do not share code you created under contract – that is owned by the company that paid you. Use this code repository as a portfolio that you can make available to hiring managers to show them your capabilities. Be prepared to talk about these projects in the interview. Which ones were assigned projects, which ones did you take on just to learn and explore? What choices did you face along the way? How did you resolve issues? How did you test?

Understand Testing: Whether you work in an Agile or Waterfall environment, you need to understand the components of software development and the responsibilities on the person creating the code. I’m over simplifying here, but you should be able to talk thru how you design, how you plan for testing, how you code and how you test. If you don’t understand why you should figure out how you’ll test code before you actually write the code, repeat this step as often as needed. Truly gifted developers understand the need to plan their test strategy and write their tests before they code.

Join your schools Computer Club: Take on an active role in the club. Become an officer, help run events. Has the club sponsored a hacking event? Can the club sponsor a hacking event? Can the club take part in competitions at other schools? Be able to talk about your experiences in the club, what you did with the club how you helped the club meet its mission.

Establish Relationships with your Professors/Instructors: Everyone is going to need to have a set of references when they apply. As an applicant for a job, your ability to identify instructors that you’ve had along the way who will speak positively about your participation in class and with the assigned work is a net positive for you. Remember, people like myself have relationships with many of the local instructors and we’ll ask them about your participation in class, how you worked in collaboration with other students on team projects. Many times, I’ll have instructors send me a note or stop me after I’ve met with their students and they’ll tell me students that I should get in touch with, who are the ones going above and beyond on their assignments, who works well in class and is always paying attention and helping others out.

Research Companies: As a Freshman, start figuring out what kinds of companies you want to work for after you get out of college. Do you want to work for large multi-national companies? Do you want to work for a smaller company? Do you want to live in the country? Do you want to live in a large metro area? Do you want the opportunity to telecommute? Start creating a list of companies that you’re interested in and make sure you look for opportunities to connect with those companies. Follow them on LinkedIn. Regularly visit their web site to see what the company is doing. Follow them in news articles to understand what they are doing. If they have a recruiting department – connect with their recruiters or with people in HR via LinkedIn. Find out thru your research what skills and experiences that they want students to have. As an example, one of the companies that my son eventually wants to work for leans towards hiring students with collegiate research experience.

Attend All Job Fairs: Even as a Freshman in college, you should be going to job fairs to introduce yourself to potential employers. Let them know upfront that you’re a Freshman and understand that you’re probably not eligible for internships, but that you’d like to learn about their company, what they are looking for and ask them how to stand out as an applicant. Connect with corporate recruiters from these job fairs via LinkedIn. Send them a note after the job fair telling them what you heard about their company and what they are looking for and then let the recruiter know that you’ll keep them updated on your progress. Then actually stay in touch as you move thru your educational career.

Use LinkedIn: Do not wait – as a Freshman, create a LinkedIn account. Begin following companies you are interested in. Begin following people at the companies you are interested in. Specifically, follow the companies you are interested in, find managers that might potentially hire you someday, find the recruiters in the company and connect with people in HR. Build these connections so that you can see what they are posting. When you see something posted that you can contribute to in a meaningful way, jump into the conversation. Don’t reply to a post just to reply, be thoughtful and add to the discussion.

I’m not saying every student should hit every one of the above items, but if you are just going to class, doing the bare minimum to get buy on homework, that is not how you’ll get noticed by employers and find internship opportunities, let alone job opportunities when you graduate. If this is going to be your career, take it seriously and invest the time needed to not only tackle your education, but to prepare yourself for when you want to get hired!

If you'd like more information on my background: LinkedIn Profile