Laissez Faire: My Story of Kayak and American Airlines

I consider myself a reasonably intelligent person, not a novice adult by any means, but I got burned by Kayak and American Airlines to the tune of over $500 dollars. I hope this post will help others avoid such situations, and avoid taking a loss like this.

For a little back-story, my parents after more than 40 years of marriage, filed for divorce this past summer and put my childhood home up for sale. Losing your childhood home is something almost everyone goes thru, and I've been preparing myself for a while. Saying goodbye every time I left after a visit, and being thankful for the trip 'home'. The divorce of their parents on the other hand is not something that middle aged adult often goes thru, but sadly it is becoming a more common occurrence. Through other circumstances my mother was very isolated and still living in the house, but the house had finally sold and she needed to move out. I live half the country away from her but volunteered to fly home to help with the final move to her new apartment. This is where my story begins.

Dreading this whole experience to begin with I had to make last minute plans to fly home. I decided to use Kayak, a site my household has used many times successfully to book flights. So, I went to Kayak's site which is where blunder #1 happened. Be absolutely sure when dealing with Kayak, or others apps broker transactions with other sites, that you are logged in with an account before you start a transaction. I learned the hard way, that if the transaction has any issues, Kayak definitely won't have any record of the transaction if you are not logged into Kayak.com.

Because of time frame, where I live and where I was flying to, my options are limited for airlines and flights. I found something acceptable in price and time with American Airlines and started the booking process. At the completion of the booking process thru Kayak and American Airlines, I briefly received a screen with confirmation of a booking request, not a confirmation of the booking itself, before the page crashed and I lost that book request confirmation number. Thirty six hours after attempting this booking, and not finding any other communication from either Kayak or American Airlines I decided to do some investigation. So I went to Kayak to see if I could find record of my transaction, and this is where blunders #2 and #3 happened. Before you decided that an online transaction has not gone thru, check all intermediaries and the primary. This includes in my case, my credit card company, Kayak, and American Airlines. I only checked with Kayak, and Kayak had no record of my name or credit card. So I booked a flight again with Delta, at a higher rate and an even less convenient time.

So now it is less than 24 hours before I leave, I check in to the booked flight I know about, and a few minutes later I receive my first email about my original booking with American Airlines. An important lesson, even if you've never had a problem with emails being placed in junk mail, always check your junk mail, you just never know. So I call my credit card company to check and dispute this charge.

I doubt my dispute will be resolved in my favor, I've since talked with both my credit card company and American Airlines. So this is a story of 'buyer beware' especially online; you have to be IT help desk, travel agent and your own customer service if a website or app error occurs. I'm more angry with myself at this point, the result will be that I will be doing things differently from now on regarding booking flights. I will be looking into a travel agency or some other human contact way of booking flights, the process of acquiring tickets for flights is not a free and competitive market, the ever changing price, availability, and timing make direct comparison difficult and allows airlines to obfuscate choice, to squash pressure from buyers that would drive down pricing. If you could say I want to fly to Walla Walla on Thursday between 8am and 9am, and there be actual choices in flights, that weren't wildly different in price, duration of travel and return options I would say that the situation is manageable. But we all know, like utilities, there is no real choice and that we all must really beware when dealing with companies who deal in a business where competition is not healthy. Though I'm sure that some trolls and advocates for Oligarchs that are salivating to tell me I'm a fool and communist.

TheWorkNumber, Equifax, and My Story Part I

I'm currently in the process of purchasing a home. Part of the process for applying for the mortgage is acquiring a verification of employment. Many employers use a third party service, called TheWorkNumber, to handle employment verification. This third party employment verifier is a division of Equifax, one of the three credit bureaus. Currently TheWorkNumber is refusing to verify my employment. For the past several weeks my lender has repeatedly contacted TheWorkNumber requesting a verification of employment. The only message my lender is provided is the following.

Ask the employee (applicant) to call 1-866-604-6570 and choose Option #2 if he or she thinks there is a problem with our records.

I have called this number. The person greeting me at the other end never verified who I am, took any information to verify my identity, and didn't seem to understand or comprehend why I had called. Until prompted by someone who was monitoring our conversation, they then provided the number 1-800-367-5690. I provided this number to my lender. This was two weeks ago, and I was contacted by my lender this past Friday, to let me know that they still have not been able to verify my employment through TheWorkNumber.

At this stage I am "approved" for the mortgage, and am under contract on a home. I have spent more than $2,000 of my own money on inspections, with an additional $1,000 of due diligence money provided to the seller as payment to take the house off the market to perform the inspections. But all of this may be for naught if I cannot get my employment verified by my lender.

I go to work Monday through Friday, I get a paycheck every week, and have pay stubs to corroborate the fact that I am employed. Unfortunately this is not enough for a lender to verify employment. But the employment verification process is simple. Usually only 3 to 4 questions, and most lenders will gladly speak with a hiring manager or any HR employee to perform the verification. But here is where the rub is. I have reason to believe, via statements by my employer, that TheWorkNumber as part of the terms of their verification of employment service, contractually forbid employers from verifying an employee's work status directly. All verifications of employment must go through TheWorkNumber.

So what is going on here? While I can't speak of the internal motivations or processes of TheWorkNumber, lets back up for a minute and revisit some information. I will also provide, what I thought at the time was an anecdotal detail, but may have more meaning to this story. The verification of employment provider TheWorkNumber is a division of Equifax, one of the three credit bureaus. As part of the pre-approval process, the lender will run a credit check, and when my credit check was run. A collection from Sprint was found, I paid that collection in the full amount the same day I found out about it. It seems that when I canceled my mobile phone service with Sprint last year. After Sprint had sent my mobile phone bill to the correct address for more than 5 years. Somehow the final bill, with the early termination fee, was sent to my previous address from 5 years earlier. I never received that final bill from Sprint, or any of the subsequent notices, or the collection notices from the bill collector. Only when my mortgage lender checked my credit report through, yes you guessed it Equifax, was the outstanding collection found. This correlation is a little too fantastic to be just a coincidence.

I started researching TheWorkNumber and Equifax. It seems I am not the only person to run into issues with employment verification through TheWorkNumber. With Equifax not being a Better Business Bureau registered company there are very few documented complaints against TheWorkNumber. But plenty of anecdotal ones can be found here. With regard to Equifax, the company has been the subject of two major class action suits, resulting in unprecedented judgments against Equifax. One in 2013 regarding Oregon woman who was repeatedly denied credit because of incorrect credit report data, and one in 2015 where a man who was the subject of identify theft found out that Equifax was providing his confidential information to the thief.

I've posted this story here so that others who find themselves in a similar situation will know that they are not alone. Ultimately the loser in this situation, should I not get the mortgage, will be my lender. We are a dual income household, and a mortgage will allow us to purchase a home now, rather than wait to purchase our home with cash a few years from now. This means that while I will be out of $3,500 or so dollars, should this contract fall through, my lender will be out of $80,000 in interest they would have acquired through me during the life of the mortgage. Instead I'll pay another $40,000 in rent while I save the money for the cash purchase of a home. But during that time I will not be paying property taxes and upkeep on a property, so long term this situation could benefit me greatly.

My long term plan is to let my credit rating drop to 0, as I stop using credit all together. But this current situation with TheWorkNumber may precipitate that drop. Your credit rating is a measure of your ability to finance profit to lenders, not a measure of your financial solvency, and once you stop paying finance charges. It's amazing how quickly you can save money. We buy our cars with cash, pay all our bills and expenses through direct bank drafts or cash. We have also paid off all our student loans over the last 8 years, nearly a $100,000 in total, and keep a zero sum budget every month. While in principle I'm upset over this current situation, and it is an inconvenience. It by no means, will prevent me from purchasing a home. I'll update more as this story progresses.

Update: 6/24/2016

I was able to get my employment verification. After my lender made repeated phone calls, a secondary set of instructions provided by my employer was discovered explaining how to setup a PIN for income verification. I did not need income verification, my lender did not ask for income verification, and my employer never explicitly mentioned income verification. But alas it appears that they way The Work Number has their system setup, a PIN for income verification has to be created by the employee, before a lender can verify employment. Now that makes sense doesn't?

Anyway I hope this blog helps someone else who may be needlessly struggling with something as simple as employment verification. But problems always seem to arise when Equifax gets involved.

Ruby's OpenSSL::Cipher, How Does Padding Work & What's Your State?!

I had occasion to learn how ruby's openssl gem handles encrypted data padding when I was given a requirement that credentials be read from an AES encrypted file on disk. The CBC symmetric mode would be used with a 128 bit key. There would be no initialization vector used in the encryption. The encrypted credentials reading would evening be implemented in both ruby and python. But ruby was the current need.

Being familiar with the python PyCrypto package I found the ruby syntax disorienting at first, in particular the example below.

 cipher.update(ciphertext) + cipher.final  

Where the addition operator appears to be overridden to perform chaining of execution to a reference. Reading ruby's OpenSSL::Cipher documentation gives this explanation of #final.

ECB (which should not be used) and CBC are both block-based modes. This means that unlike for the other streaming-based modes, they operate on fixed-size blocks of data, and therefore they require a "finalization" step to produce or correctly decrypt the last block of data by appropriately handling some form of padding. Therefore it is essential to add the output of #final to your encryption/decryption buffer or you will end up with decryption errors or truncated data.

I was still left wondering. How is OpenSSL::Cipher handling the padding? Looking through the documentation for the 'update' method I found no mention of padding. I would return to this question and the OpenSSL::Cipher documentation later, learning two important lessons.

After completing the ruby package, I turned my attention to exploring the creation of a python package with the same functionality. It then occurred to me while working with PyCrypto, that the details of how padding where handled between PyCrypto and OpenSSL::Cipher where going to be important to understand. I decided to write two scripts, one in python and one in ruby. Each would alternately encrypt and write to file, or read from file and decrypt data to and from each other using a common 128 bit key. I would use the lines "This is from python! This is also from python. Happy reading!" and "This is from ruby! This is also from ruby. Happy reading!" as the data to encrypt. The first string is 61 characters long, assuming one byte per character, the string is also 61 bytes of data. While the second string is 57 characters and 57 bytes.

Reading and decrypting the ruby encrypted text in python I saw the following in my terminal.

 64 
 This is from ruby! This is also from ruby. Happy reading!^G^G^G^G^G^G^G 

The '64' is a print(len(plaintext)) before printing the plaintext. The encryption succeeded, but I was not sure how ruby indicates the padding used (but if you are quicker than I you might see it).

My initial attempt at reading the python encrypted file with ruby did not go well. I was greeted with the following error.

 64 
 ruby-data.rb:29:in 'final': bad decrypt (OpenSLL::Cipher::CipherError) 
 from ruby-data.rb:29:in '<main>'

The 64 was the byte length of the encrypted data as expected. With some experimentation I was able to get the error to go away by omitting the '+ cipher.final' from the 'cipher.update(ciphertext)' call. What I saw was the following.

 64 
 48 
 This is from python! This is also from python. H

The encrypted text from the file is 64 bytes long, the plaintext decrypted is 48 bytes long, meaning the entire last block is truncated, and the truncation confirmed in the print out.

After experimentation with changing the length of the data encrypted by both python and ruby I noticed a pattern. When I changed the string encrypted by ruby to "This is from ruby! This is also from ruby. Happy reading!1234567" and printed the decrypted results from python I saw.

 88 
 This is from ruby! This is also from ruby. Happy reading!1234567^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P

That is when the lightbulb lit up. The previous glyph printed for the padding on the encrypted data from ruby decrypted by python was ^G and the padding was 7 bytes. The current encrypted data from ruby and decrypted by python is ^P and the padding is 16 bytes. The glyphs printed at my terminal correspond to the chars who's ordinals are 7 and 16 respectively. OpenSSL::Cipher is using the ordinal value of the last byte in the file as a char, to encode the padding used. Because my new string encoded in ruby is exactly 64 bytes long, and padded with 16 bytes. It also shows that OpenSSL::Cipher always padded my data with at least 1 byte of padding but now more than 16 bytes. Ensuring that no data is truncated by mistakenly interpreting the last byte of data read from the file as padding.

This is a clever scheme for not having to use an extra byte to encode the padding length used in the encrypted data. But not a common paradigm, and the lack of configuration of the 'update' method for padding had me concerned that OpenSSL::Cipher might be capably of reading encrypted data from other tools that don't use this clever padding scheme. This is where I learned another important lesson about ruby, and added another item to my list of things I don't like.

Stepping back for a moment and talking about interfaces. A common scenario with interfaces is the breaking of backwards compatibility when the interface changes. Specifically when the signature of an interface member changes. For example take the following interface function.

 def method_foo(arg1) 

A change of the following in many languages would break backward compatibility.

 def method_foo(arg1, arg2) 

Causing code written to the old version of this interface to break. A better way to add the new argument to the method signature is as a keyword argument. As long as the new argument is optional. This preserves backwards compatibility with code written to the old interface.

 def method_foo(arg1, kwarg1='default') 

Keeping invocations of the following valid.

 method_foo('bar') 

While also allowing new development to make use of the new argument option.

 method_foo('foo', kwarg1='fooy') 

If the new argument is not optional then changing the method signature and breaking old code at the invocation of the method that has changed is problematic but potentially easy to find and fix. Keep that in mind.

Now back to OpenSSL::Cipher and ruby. Another approach when dealing with changes to an interface is to expand the interface when changes are made, and leave the existing members and there signature unchanged. This is the approach that OpenSSL::Cipher used with its interface design. An engineer on the team more familiar with ruby clued me in this approach, and I should have seen the pattern coming. When a new cipher instance is created in OpenSSL::Cipher before it can be used to either encrypt or decrypt data either 'encrypt' or 'decrypt' must be called.

 cipher = OpenSSL::Cipher.new('AES-128-CBC')
cipher.decrypt
cipher.key
cipher.update(ciphertext) + cipher.final

This pattern struck me as odd. But not being able to see the forest for the trees in ruby, I took note of it and moved on. The piece I was missing with OpenSSL was an addition to this pattern.

 cipher = OpenSSL::Cipher.new('AES-128-CBC')
cipher.decrypt
cipher.key
cipher.padding = 0
cipher.update(ciphertext)

Allowing the omission of the "... + cipher.final" and changing the OpenSSL::Cipher instances behavior when the 'update' method is called. The encrypted text will be decrypted and passed returned padding and all. If instead cipher.encrypt was called, the plaintext passed to 'update' must be a multiple of 16 bytes is size or an error will be raised.

While the use of methods like 'decrypt', 'encrypt', 'padding' and 'update' may seem like a novel way to sidestep problems with changes to an interface. The result is the behavior of an OpenSSL::Cipher instance is not immediately apparent without first examining the state of the OpenSSL::Cipher instance. An ill defined context becomes the lay of the land. I have an OpenSSL::Cipher instance, does it decrypt or encrypt when 'update' is called? Do I have to handle padding myself?

I point to the 'update' method as a first bad smell. What does 'update' or updating a cipher mean? A cipher can decrypt or encrypt data so methods 'encrypt' and 'decrypt' make sense. But on an OpenSSL::Cipher instance, 'encrypt' and 'decrypt' don't do what their verb would indicate. Instead 'encrypt' or 'decrypt' change the state of the OpenSSL::Cipher instance, and the behavior of 'update'. I also take issue with 'key' which makes the key used by the cipher part of its state. I would be less critical of making the key part of the instances context if the key was required by the initializer, the same goes for configuration of the cipher to encrypt or decrypt. But my experience with ruby so far is that the use of constructors to create new instances of types in stable well defined states is not common.

Why I Don't Like Rails and Why You Shouldn't Either

Via some interesting events on my current contract I find myself working in Ruby on Rails. I have not worked with Ruby before and viewed this as an opportunity to expand my horizons. For most people coming to Rails from another technology the first thing you notice is that Rails is more of a platform with its own domain language, rather than a web development framework for Ruby. In Rails there is a lot of overriding of object behavior and syntax to add 'useful' functionality. This is the first bad smell. What I next realized is that ActiveRecord provides default initialize methods for constructing your types. Not unlike many other ORM technologies, but what struck me was that ActiveRecord does so by overriding the Ruby Foo.new and Foo.create syntax and object behavior. To me this is a pollution of the class name space, and instead of the objects being first Ruby classes, they are ActiveRecord's perversion of Ruby classes. Now things are smelling even worse. We have effectively been cut off from Ruby's very complete and well thought out type definition capabilities. Most notably the ability to encapsulate and abstract out instance creation logic.

My belief in stable state instantiation of instances is well documented here, and all these arguments still apply to my current context with Rails. While all ORMs break some part of the language, it is usually polymorphism, but with Ruby on Rails you effectively cannot encapsulate your stable state instantiation logic. ActiveRecord has broken a fundamental piece of Ruby's object oriented nature. Often in ORM mapped classes there is a lot of messy logic for creating associations, especially with nested 'foo has many bar' instances. The simplest way to hide this messy implementation is to place it inside constructors. Giving every developer who works with these types a well defined set of methods, either a base constructor or factory methods, that will return stable instances or raise an error. In Rails this is not the case, you are on your own to wrestle with the default constructors to create your stable instances. Leaving traces of this logic scattered around your code base.

Now I will get to my most controversial point, Rails is an anti-pattern imposed onto Ruby. If you have ever tried to write complex models in Rails and encapsulate that logic you have seen the effects of the Rails anti pattern in your code base. While the allure of the Rails tutorials show all the 'sugar' and 'magic' of creating trite little web apps, the sheen soon fades away when you try to do anything more complicated with Rails. This is why I don't like Rails, and why you shouldn't either.

I See Stupid 10 Times a Day

You have to be smart before you are capable of really stupid actions. One of the dumbest things I've ever seen is at my first engineering job. We where a Java shop that had hosted development environments provided by our IT department. We shared these hosted development environments with an informatics group that was using Ruby to do genome analysis. An issue arrived where Ruby jobs where kicked off in these hosted development environments that would use up all available memory on the server. With the blessing of the centers director, IT implemented memory quotas on all the hosted development environments, limiting the amount of memory any one process/user could acquire on a given server. These quotas did go into effect until a new shell process was started. Many of us used screen to keep shell processes running, while being able to disconnect form the server.

Over the next week, for various reasons, existing shell processes were terminated and new shell processes were started. All the developers in my group by weeks end were unable to start a JVM. Many service tickets where filed with IT. But there was no response other than, "we are working on it." Finally on Monday of the following week, the directory of our engineering group went into the IT directors office, closed the door behind him and just started screaming. Nearly an entire weeks efforts had been stalled because of our inability to work.

The cause of the issue was the memory quotas. The JVM when starting does a 'check' of the available memory on the system. This check was blocked by the memory quota, resulting in the inability to start a JVM for any reason. Unfortunately this is not the end of the tale of stupid. The solution IT came up with was to allow developers in my group to have terminal level access to the cluster where our production processes ran...

This question is not, if we are on a ship of fools. But rather when the fools on our ships will decide to run the ship aground. I find myself in a similar situation now. In a effort to improve security, security scans are being run on servers in development spaces. Resulting in erroneous shut downs of development environments because they are development environments and don't meet production security standards. I'm sure to someone at some point this all seemed like a really good idea. I unfortunately just have to count this as yet another instance of stupid I encounter in my professional life.

So instead of continuing to work on the tasks for completing our next goals. I'm scrambling to find a new way to actually do the work because my development environments are disappearing and I'm having to spend my time learning how to mock up a full environment with docker.

Paris Under Attack

What is happening this evening in Paris is tragedy. Lives are ending, people are suffering and it will be hard to hold back the emotions from taking us over. But we must, we must resist giving in to anger and hatred. Because the violence that was unleashed on Paris was rooted in emotions getting out of hand, rooted in hatred and anger, and forgetting that there is another way forward. A way forward without violence.

We cannot control what happens to us. But we do have the ability to try and control our responses. While sometimes if may seem that lashing out and seeking some form of control is the right thing to do. Be that 'justice' or something much less civilized. The best way forward is often to seek the path that puts compassion and forgiveness first.

Choosing the path of compassion and forgiveness does not mean foregoing justice. Often justice can only be achieved by putting emotion behind us and seeking a resolution that is the most peaceful.

Apprentice, Journeyman, Master: What it takes to progress as a software Engineer

I truly believe that software engineering is a trade, unlike other engineering disciplines there is a unprecedented need for software engineers in all parts of society, and software engineers are finding themselves working in every business sector from healthcare to telecom, education to cupcake shops. Like carpenters, electricians, plumbers and auto mechanics. Many diverse institutions desire our services. Unfortunately like all the other trades I've listed there is a mistrust because few outside our trade understand what we do. Professional misconduct, whether intentional or accidental, has given some good reason to be wary.

My father is an electrician, he can tell stories all day about some of the crazy things he has seen. A surprising number of those stories are not about home owners who have done their own work but of other electricians work. There is an element of subjectivity but there are widely held standards, and in the case of the electrical trade there are national and international groups that endorse a set of standards that can be applied with a level of objectivity. For software engineering there are similar standards but nothing as widely held as building code, but I would argue that software engineering is still in the early stages of creating a set standards. Much of the reason for that is how rapidly the field is changing, and will probably continue to. But something that does concern me is how those standards and practices will be taught, maintained and spread. The other trades have apprenticeships that often last years, where someone has to work with a master tradesman while they complete a number of hours in addition to course work. Once the course work and number of hours as an apprentice have been completed an exam is administered and the apprentice if they pass can become a journeyman. A journeyman is someone who can work independently, but who must be overseen by a master who inspects and certifies their work. Then after many years the journeyman can apply for the exam to obtain their master status in their trade.

I don't know if such a system is appropriate or even implementable in software engineering with all the regulations and standards, but I do believe that the years of mentorship are necessary. I truly hope the age of the lone hacker is passing, the lone developer is of no real use to anyone except for the simplest of tasks. I have worked on too many projects where the body of code was written by a single individual with a grand vision but not the skills to bring it to fruition. All of us need to over site of master because of the sheer complexity of what we do as software developers. It is very rare that one individual will have the experience and knowledge to cover the entire scope of even a moderately complex project.

I place responsibility on both the engineers and their patrons. What often happens is a company, research group or institution needs software written and some knows "a guy." Very similarly to your car needs a new transmission and your brother-in-law knows a guy who can do it for half the price of G.M. Goodwrench's quote. Not that every shade-tree mechanic is a rip-off but you need to use your better judgment. Don't bring the car that you need to drive 50 miles a day to work someplace where you will have no warranty for the work. Your car is how you get to and from your job, but the weekend golf car, or the spare vehicle might now be a bad idea. The same goes for software, don't have some guy code your mission critical application for a major initiative. But if you need a simple web application for your project to spread and gather some information to prospective customers and participants. You can probably go to your sister-in-law's friend who would love the addition to her portfolio of work. Use your better judgment, if they solution is nontrivial you will need the security of experience along with the support of more than one or two experienced engineerings. A good rule of thumb is if you are not delegating the task because it is something you could do but don't have the time, then you should probably seek reputable professional advice. Changing a light switch is much different than wiring a new addition.