I am going to recount a recent learning experience I encountered while working on a Rails Project. The ideas I came away with are not Rails specific, however, and are applicable to all software. The learning experienced involved me leaning on third party libraries. This lean later turned into a fall and I realized that third party libraries are to be used for a specific case and only for that case. Dependencies are for what you yourself can not do on your own in a reasonable time period. I learned that we must be careful with the libraries we make use of since their behavior is outside of our control even if we believe to have them pinned down in test.
ActionMailer is a gem most Rails programmers are familiar with. For a story I was completing I needed to email notifications and I knew I was not going to implement a mailing system in a reasonable time period. I then introduced ActionMailer and wrapped the functionality. I had the system under test and I felt pretty confident with what I was using ActionMailer for. My tests were green. The code then went live and I quickly learned that I had made a big mistake. I started receiving ActionMailer generated exceptions. What had happend? I was green when I had committed!
Well, I was green and it was a false positive. ActionMailer has a different set of rules for test and for production. In my case I leaned on ActionMailer and got burnt. I took an array of email addresses and joined them by commas to produce a string to be used as the recipient list of my generated mail. When there were no array entries this produced an empty string. When I sent an empty string to ActionMailer in test it essentially disregards it. Great, I thought, ActionMailer handles my empty email list case! WRONG!
In production an empty string as a recipient list with ActionMailer produces an exception. When the operation is fairly common it produces a lot of exceptions. It was my fault for not narrowing my usage of ActionMailer to the specific cases in which it was actually needed. I instead used it for my empty string case. The moral of story, external dependencies are for specific use cases and nothing more.
Showing posts with label Rails. Show all posts
Showing posts with label Rails. Show all posts
Sunday, January 29, 2012
Sunday, September 18, 2011
Exposing Edge Cases Through Test
We recently had an interesting situation with our test suite while working on a Rails project. When our entire test suite was run (rspec -> cucumber -> jasmine) we would have all tests pass, however, when cucumber was run on its own it would fail. Upon inspecting our stack trace we saw that a followed redirect in cucumber was not rendering correctly. This meant that the environments differed between running the entire suite and running the acceptance tests in isolation. How could this be, we all thought, an app is an app and the environment is test no matter how it is run.
Upon inspection we were able to follow the problem to a 'singleton' we had within the system. We had a mailing list object that was meant to have a single instance to represent the single mailing list we are keeping. Within our seeds.rb file we had a check to see if an instance exists, and if not to then create an instance. Our class looked like this:
All developers on the project understood that mailing list was to be called by mailing_list.instance and in our heads we thought of it as a singleton. In production this was working. Thankfully, we had tests to expose the problem with our logic. When rspec ran it created it's first MailingList instance at id 1, which made instance work throughout the test suite and explained why our tests passed when the entire suite was ran. However, cucumber deleted its instances of MailingList after a scenario had finished. This was a problem because the mysql backend kept issuing ids in order. Therefore, the above code would return a nil object when we asked for the instance.
Our revised singleton looks a little better:
We now give instance more importance. We were able to drop the singleton creation out of our seeds file since calling instance always ensures a singleton. When we use create it gives an informative error and leads the developer to use instance, as intended. Moral of the story, tests are more important than you think.
Upon inspection we were able to follow the problem to a 'singleton' we had within the system. We had a mailing list object that was meant to have a single instance to represent the single mailing list we are keeping. Within our seeds.rb file we had a check to see if an instance exists, and if not to then create an instance. Our class looked like this:
All developers on the project understood that mailing list was to be called by mailing_list.instance and in our heads we thought of it as a singleton. In production this was working. Thankfully, we had tests to expose the problem with our logic. When rspec ran it created it's first MailingList instance at id 1, which made instance work throughout the test suite and explained why our tests passed when the entire suite was ran. However, cucumber deleted its instances of MailingList after a scenario had finished. This was a problem because the mysql backend kept issuing ids in order. Therefore, the above code would return a nil object when we asked for the instance.
Our revised singleton looks a little better:
We now give instance more importance. We were able to drop the singleton creation out of our seeds file since calling instance always ensures a singleton. When we use create it gives an informative error and leads the developer to use instance, as intended. Moral of the story, tests are more important than you think.
Subscribe to:
Posts (Atom)