Archaic Positives

Adventurer, Traveler, Rubyist

Integrating a AWS Proxy Into a Flask App

I ran into an issue recently when I was working on Percolate’s Hello application, which serves as Percolate’s intranet. We have API documentation that is generated by our application codebase, and pushed up to AWS S3 for storage and hosting purposes. It wasn’t immediately clear to me how to proxy the documentation effectively. Do I download the documentation files from AWS, and render each file as a template? Turns out it’s a lot simpler than that.

What is Hello?

Hello is a Flask application that we use as an intranet at Percolate. Flask is a great Python framework that allows us to build small, flexible applications. In our case, we’ve used it to host documentation, serve an API that interfaces with Namely, and also index information specific to Percolate.

The Initial Problem

So the first issue that I ran into was trying to figure out how to properly proxy documentation hosted on AWS S3 through the Hello application. Due to some initial confusion that I had, I didn’t know if AWS S3 would support relative links if they’re proxied on Hello. For example, if an HTML file is proxied, would the relative links on the AWS bucket (for example, CSS/JS files in the HTML file), would those relative links be loaded? I experimented first by downloading the files from S3 to the local filesystem for Hello.

Fail - Other Options?

It turned out to be a failure, because downloading them into the application’s local filesystem was breaking the relative links in all of the HTML templates. I would either have to host the asset files internally from within the application, or find another option forward. It turns out, there’s a method called open_read in boto’s Key library that will allow us to render the HTML template in the application, even though it’s being hosted on AWS S3. So here’s how we set up the /docs endpoint URL in Hello:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
@app.route("/docs/<path:aws_key_name>", methods=['GET'])
def s3_key(aws_key_name):
    if aws_key_name.endswith('/'):
        aws_pathname = aws_pathname + 'index.html'

    s3_doc = HelloS3.find_doc_in_bucket(aws_pathname)
    if s3_doc:
        try:
            s3_doc.open_read()
        except S3ResponseError as e:
            return Response(e.body, status=e.status,
                            headers=s3_doc.resp.get_headers())

       headers = dict(s3_doc.resp.get_headers())
       return Response(s3_doc, headers=headers)

    return abort(404)

So to recap, here’s what we’re doing with the logic above. Remember that the documentation we’re hosting on AWS has an index.html template stored in teh documentation’s directory. For example, documentation for the animals API is going to be rendered via a index.html template, so the path for that documentation would be /penguins/index.html. So we have to account for that in our logic. Whenever someone types in a request, let’s say /docs/example_key/, it sets aws_key_name = example_key/. If the key name ends with /, then it will automatically append index.html to the end. Then we have a module called HelloS3 that will search for that key in the AWS S3 bucket. If it exists, It will proxy that file through our application. If it can’t find a file, then it will return a 404 page.

Lesson Learned

Instead of trying to build out everything yourself, check to see if there’s a simpler solution right in front of you. There was one in the boto package, and I missed it. I hope this helps someone else out!

You’re Not an Impostor: How to Manage Self-Expectations as a New Developer

Managing Uncertain Expectations

When I started programming over a year ago, I kept second guessing myself every step of the way because I believed that I truly didn’t know how to solve a problem or implement code correctly. I also believed that I was a burden on others, and the best strategy was to keep my banal questions to myself. Looking back, I wasted so much time and effort tricking myself into believing that I wasn’t good enough, and that I never would reach a state in which I was content with my abilities. I had no idea that I wasn’t alone.

To self-criticize is Human

Humans are very critical, sometimes overbearing, creatures by nature, and we tend to come down hard on our own inabilities to overcome small obstacles, often attributing them to some inherent flaw in our mental make-up. This humanistic trait essentially makes it even more difficult when it comes to learning something new. We fault ourselves for not being able to get it right away, and we give up either out of desperation or because we fear how much we don’t know. As a result, we’re mentally hard-wired to set ourselves up for failure.

Perspective is Everything

As an Instructor for the Web Development curriculum at the Flatiron School, the responsibilities I typically have each day range from bug hotfixes to in-depth explanations of how the Rails framework handles controller logic. But another, less obvious part of my job is constantly helping others keep their expectations in check. From the start, we tell our students that it’s important to keep their skill level in perspective—that the path to becoming a better developer is a slow upward trajectory. We tell them, ”you will never feel as dumb as you did today.”

So what exactly is impostor syndrome? Simply put, it is the feeling that “we are frauds and we do not deserve the success we have achieved. Proof of success is dismissed as luck, timing, or as a result of deceiving others into thinking that they are more intelligent and competent than they believe themselves to be.” Even more simply put, it is an incongruous perspective — the act of people with actual ability underestimating their relative competence.

At some point in their career, a developer will experience impostor syndrome in some way, shape, or form. How they manage those feelings of insecurity depends on what strategies or techniques they decide to use.

Impostor Epidemic

It wasn’t until I started at my first programming job that I really started to feel inadequate, and I kept trying to convince myself that I couldn’t fulfill the project assignments I was given each week. I spent a lot of time trying to justify why I wasn’t qualified, and less time on praising the progress I had made since I made the decision to change careers into programming. I essentially spent a lot of time trying to attribute my success in my first few months to sheer luck or the efforts of others.

Simply put, you will never completely eradicate the presence of impostor syndrome. But you can manage it effectively through positive reinforcement and open dialogue. I was very fortunate to work with a group that held a feedback session every Friday, and on one Friday, I brought up my feelings with my team. I was surprised to find that my feelings were widely shared—even among the senior developers that I worked with.

Dealing with Imposter Syndrome

Coming into programming from an accelerated learning route (via Flatiron School), this is something that our students run into firsthand. Going through the process of learning the nuances of Ruby, Sinatra, Rails, Javascript, and ultimately how to build a functioning web application can be extremely daunting. Even more so is the fact that they slowly have to remove their hands from the guardrails in order to stand on your own as a developer. This was the most challenging part about my own experience, and this is something our students face every semester.

Stay positive. It’s important to keep in perspective how far you’ve come in such a short amount of time. Play up your successes, and think about your failures, and how you can learn from them. It doesn’t matter how many times you fall — what ultimately defines you as a developer is that you continue to get back up, and recognize that there will always be moments of uncertainty and discomfort.

Talk about it. The realization that everyone felt the way I did was huge. With the help of others, I started using several techniques to keep my doubts about my abilities in check — like positive reinforcement from friends and co-workers, and journal entries. I started participating in weekly code talks called Code Newbie that my friend, the amazingly talented and wonderful Saron Yitbarek, started and currently maintains. She spoke about her experience at RailsConf 2014 and in this post.

Be OK with not knowing everything. I’ve had to figure out how to be OK with (excited, even!) to not know everything — it’s an opportunity to learn new things and refine my current skills. If you’re a new developer, it’s fine to acknowledge the long path ahead of you. But know that you owe it to yourself to enjoy the process of learning along the way. Accept the fact that you will never know everything there is to know about programming, and that is fine. It should be about shared experiences and delayed gratification. Take pride in new skills, incremental returns, and be excited for the programmer you’ll become.

Learn with others and let them teach you. The best piece of advice I can give is that learning to become a programmer should not be an isolated, but rather a shared, experience. Through others, particularly a mentor, you can receive the validation and gentle push necessary to see you through the more challenging obstacles. Go to Meetups. Reach out to a developer that you like or admire. Emulate their workflow through observation. Build a web application on the side with a friend. Great software developers are the sum of their aggregated experiences over time. I guarantee you that they’ve experienced more failures than they have successes. You never know what may come of your efforts unless you try.

And most of all, believe in yourself. No one can predict the future. It is full of unknowns, and there are bound to be curve balls thrown here and there. It’s very easy to convince yourself that you are a tiny fragment of what you really are. Despite the dark storm clouds of negativity or belittlement that may gather in your mind, believe in yourself. Temper your lofty expectations. Adapt. Empower. Share. As Bruce Feiler says, “take a walk with a turtle. And behold the world in pause.”

Working With Rack Sessions

I spent the last few days building a Flatiron School lab that specifically focused on setting sessions within a small Sinatra app. The idea was that this lab would help students better understand the concept of a session, and how they allow web applications to maintain state. By going through several RSpec tests and making them pass, the students would iteratively see what the session variable values would be in each step, and would understand the process from start to finish. The problem that I ran into was that I was not able to directly access the session variable in Sinatra.

The Root Problem?

The fundamental issue was that in Rack (on top of which Sinatra is built) does not allow you to directly access the sessions variable. I could not access it when I was writing acceptance tests, and so I set out to resolve that issue.

Troubleshoot, take 1

The first step that I took to troubleshoot this was to read through this StackOverflow post to see how I could directly access and manipulate session data. So one underlying thorn in the side was that it’s hard to directly access sessions for acceptance testing purposes. I found this gem that supposedly makes it easier for a developer to access and manipulate sessions variables.

I went down that road, and quickly realized that for the purposes of this lab, I was overcomplicating things a little bit. What the Rack Session Access gem does is that it provides middleware for your Sinatra app that allows you to set and obtain data for your application’s current session. I didn’t need to set anything - I just needed to be able to access the sessions hash and confirm that the values met the specifications set forth in my RSpec tests.

Next option.

Troubleshoot, take 2

The second step I took was to read this post. This gave a better in depth explanation of the issue I was having, which was that I couldn’t directly access the sessions hash method in Rack middleware. To remedy this, the top-voted answer suggested that I try to access the sessions hash through the env hash. It was through this post that I realized what I was doing incorrectly all along. I didn’t have to implement the solution that was suggested in this post, but I had an idea what I could do to resolve the problem.

Let’s dig a little further.

Identifying the Solution

So how did I address this? So remember in the previous section that you can access the sessions hash through env. Before I address the solution, let’s briefly run through the anatomy of a Rack sessions hash. When a new session is created, it creates a new Rack::Request object, as the example below shows.

1
2
3
def call(env)
  request = Rack::Request.new(env)
end

The request object, which is assigned to a new instance of Rack::Request, has a method called session. When you call this method on the request object, it will return the sessions hash.

Now, let’s apply this in terms of acceptance testing. Using the rack/test module, I wrote the following method in my spec_helper.rb file:

1
2
3
def session
  last_request.env['rack.session']
end

The last_request attribute is actually a Rack::MockRequest object that is used to generate a request, such as a get '/' request method. You can access the sessions hash of last_request by accessing the rack.session key inside of the env method. Once I was able to access this, then I was able to write tests that could be used to test the values inside my sessions variable.

It’s a very simple way to resolve the original problem, but it definitely threw me for a loop while I was in the process of problem solving. Isn’t programming fun?

Intersection Between Tolstoy and Agile Programming

After I got back from the gym tonight, I decided in a spur of the moment decision to watch a quick TEDTalks episode. I ended up choosing one that discussed “Agile Programming”, and the talk was given by a man named Bruce Feiler. In his talk, he referenced the opening line of Leo Tolstoy’s book Anna Karenina. I’ll come back to that.

Bruce Feiler’s talk was something of a revelation for me. He spoke about the concept of agile programming, which was a organizational concept created by a programmer named Jeff Sutherland to address the inefficiencies of the top-down approach to software development. The idea is that if a company’s management comes up with an idea to develop into a software product, the creative process is too limited and does not involve a lot of feedback. It will result in a product that is poor, insufficient, and outdated. Agile programming focuses on bottom-up development, and there is emphasis on a few core tenets: “individuals and interactions over processes and tools, working software over comprehensive documentation, customer collaboration over contract negotiation, and responding to change over following a plan.”

Feiler took the concept of agile programming and refocused it into three core tenets that could be applied to any one entity, whether it’s an individual person or a collective organization. First, always keep adapting to change. Second, take opportunities to empower yourselves. Finally, never stop telling your story. The last concept sounds a little silly, but in the Internet age where information and content is at every turn, a sense of identity is more important than ever. It’s important to know what exactly your values are, what your goals are, and to find that button that will make you click. As Feiler said in the TEDTalk, “adaptability is fine, but we also need bedrock.”

Feiler finished off his talk with a story about Leo Tolstoy, pictured above. In the opening paragraph of Anna Karenina, Tolstoy opens with a very profound, yet thought provoking statement: “All happy families are alike. Each unhappy family is unhappy in its own way.” It sounds like a cliche sentence, but Tolstoy chose this line because it would come to define the entire narrative of Anna Karenina. Applying this statement to our individual lives, we have the building blocks to make ourselves stronger, better, and happier. We utilize these building blocks by applying the three tenets: adapt, empower, and share.

When Tolstoy was five years old, his brother Nikolai came to him and said that he had engraved the secret to universal happiness on a stick, which Nikolai had buried in a ravine somewhere on the Tolstoy family estate in Russia. ”If the stick were ever found,” Nikolai said, “all humankind would be happy.” Tolstoy apparently was consumed with trying to find that stick, but never did find it. On his deathbed, Tolstoy asked to be buried in the ravine where he believed the stick was buried.

The point is, we hold the key to our universal happiness. Adapt, empower, and share. Feiler closed his discussion with this: “Happiness is not something that we find. It’s something that we make. What’s the secret to happiness in general? Try.”

Bruce Feiler TedTalk

RSpec Testing for a JSON API

I am in the process of building out a JSON API for a registry application, that, when pinged by Salesforce, will fire off emails to prospective Flatiron students that notify them of their acceptance into the Flatiron School program. One thing that I noticed is that there aren’t many resources out there for setting up a testing environment and eventually building out a JSON API via Ruby on Rails. Hopefully this resource will shed a little more light on how to accomplish that.

The first step is to integrate a Rails serializer in order to encapsulate the JSON serialization of objects. We first install the active_model_serializers gem into the Gemfile, and then bundle. Now, for each model that we want to serialize for JSON serialization, we need to create a serializer: rails g serializer student.

Inside of the serializer file that has just been generated, we need to add in student attributes that will be defined and visible in the JSON API.

app/serializers/student_serializer.rb
1
2
3
class StudentSerializer < ActiveModel::Serializer
  attributes :id, :first_name, :last_name, :email
end

For example, this is how the JSON data will be represented on the API tree, based upon the order of attributes in the student_serializer.rb file above:

JSON data for student
1
2
3
4
5
6
7
8
9
  students:
    [
      {
        id: 4,
        first_name: "Doctor",
        last_name: "Who",
        email: "doctor_who@whoville.com"
      }
    ]

There is a key called students, and inside of its value store, it holds a collection of student objects. Inside of the array collection, there is 1 student with an id of 4, first name of “Doctor”, last name of “Who”, and an email “doctor_who@whoville.com”. This is a demonstrative example of how the JSON object data will be rendered when the API is pinged.

In the app/controllers directory, I created a new folder system that allows for semantic versioning of the API. Currently my directory looks like this: app/controllers/api/v1. Inside of that folder structure, I have one file: students_controller.rb. We will come back to these files once we start building out our controller actions.

Next, I set up the routes for the students resource. My config/routes.rb file looks like this currently:

config/routes.rb
1
2
3
4
5
namespace :api do
  namespace :v1 do
    resources :students, only: [:index, :show, :create]
  end
end

This is the basic setup for the API itself. At this point in the API development, we’re only concerned with the index, show, and create actions for the API Students Controller. Now, we’ll go ahead and set up the RSpec tests for the controller.

spec/controllers/api/v1/students_controller_spec.rb
1
2
3
4
5
6
7
8
9
10
11
12
require 'spec_helper'

describe API::V1::StudentsController do
  describe "GET 'index' " do
    it "returns a successful 200 response" do
      pending
    end

    it "returns all the students" do
      pending
    end
  end

This describe block refers to the index action of the controller. In this block, we are testing for two expectations: for the JSON response for the index action to return a 200 status code, and for the JSON response to return the correct number of students that exist in the test database.

spec/controllers/api/v1/students_controller_spec.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
require 'spec_helper'

describe API::V1::StudentsController do
  describe "GET 'index' " do
    it "returns a successful 200 response" do
       get :index, format: :json
      expect(response).to be_success
    end

    it "returns all the students" do
      FactoryGirl.create_list(:student, 5)
      get :index, format: :json
      parsed_response = JSON.parse(response.body)
      expect(parsed_response['students'].length).to eq(5)
    end
  end

In order to get these tests passing, we first need to submit a get request for the index action in a JSON format. We first expect the response to be successful, or more specifically, to result in a 200 status code.

We also need to test whether or not it returns all of the existing students in the test database. I’ve used the FactoryGirl gem to mock out the student list create_list(:student, 5). I’ve also set up the parsed_response variable, which will translate the response body in JSON format into a more readable format. Then I’ve set the expectation that the length of the parsed_response[students] should be equal to 5, as specified in the FactoryGirl.create_list(:student, 5) line.

spec/controllers/api/v1/students_controller_spec.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
  describe "GET 'show' " do
    it "returns a successful 200 response" do
      pending
    end

    it "returns data of an single student" do
      pending
    end

    it "returns an error if the student does not exist" do
      pending
    end
  end

This block refers to the show action. I am testing for three specific expectations: a successful JSON response to return a 200 status code, a successful response to return the correct student JSON object, and for the JSON response to return an error message for a student JSON object that doesn’t exist.

spec/controllers/api/v1/students_controller_spec.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
  describe "GET 'show' " do
    let(:student) { create(:student) }

    it "returns a successful 200 response" do
      get :student, id: student, format: :json
      expect(response).to be_success
    end

    it "returns data of an single student" do
      get :student, id: student, format: :json
      parsed_response = JSON.parse(response.body)
      expect(parsed_response['student']).to_not be_nil
    end

    it "returns an error if the student does not exist" do
      get :student, id: 10 , format: :json
      parsed_response = JSON.parse(response.body)
      expect(parsed_response['error']).to eq("Student does not exist")
      expect(response).to be_not_found
    end
  end

We’ve built out a student mock using FactoryGirl, and will be using this to test whether or not the show method returns the correct student from the test database based on the student’s ID. For each test, we are submitting a get request for the student we mocked out earlier, and we should expect a 200 response for the first test, and for parsed_response['student'] to essentially be a valid student object returned from the test database. For the third test, we are asking to return a student with an ID of 10, which doesn’t exist in our database. We should expect an error messaged from parsed_response['error'], and we should also expect the response to return a message saying that the object was not found.

spec/controllers/api/v1/students_controller_spec.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
  describe "POST 'create' " do
    context "correct email format" do
      it "returns a successful json string with success message" do
        pending
      end
    end

    context "incorrect email format" do
      it "returns an error if an incorrect email format is submitted" do
        pending
      end
    end
  end
end

This block is describing the create method, which will take in an email address parameter. If the email address is valid, then it will fire off an email to that email address, and then fire off a success message within a JSON response. If the email address is invalid, then it won’t fire off the email, and will render an invalid message within a JSON response.

spec/controllers/api/v1/students_controller_spec.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
  describe "POST 'create' " do
    context "correct email format" do
      it "returns a successful json string with success message" do
        post :create, { email: "newstudent@example.com" }
        expect(response).to be_success
        parsed_response = JSON.parse(response.body)
        expect(parsed_response['success']).to eq("Accepted email format.")
      end
    end

    context "incorrect email format" do
      it "returns an error if an incorrect email format is submitted" do
        post :create, { email: "new@studentexample" }
        parsed_response = JSON.parse(response.body)
        expect(response).to be_bad_request
        expect(parsed_response['invalid']).to eq("Invalid email format.")
      end
    end
  end
end

In the first test, we’re passing in a valid email address inside of the post request for the create action. If it’s valid, then we expect the JSON response to have a 200 status code, and we also expect parsed_response to have a success message as well. The second test passes in an invalid email address. We expect the JSON response to return a bad request status, more specifically a 400 status code, as well as a invalid message inside of the parsed_response.

In order to make all of these tests pass, here’s how the corresponding app/controllers/api/v1/students_controller.rb file looks:

controllers/api/v1/students_controller.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
module API::V1
  class StudentsController < ApplicationController
    before_action :find_student, only: [:student]

    def index
      @students = Student.all
      render json: @students
    end

    def show
      render json: @student
    end

    def create
      if valid_email?(params[:email])
        send_acceptance_email(params[:email])
        render json: { success: "Accepted email format." }
      else
        render json: { invalid: "Invalid email format." }, status: :bad_request
      end
    end

    private

    def find_student
      @student = Student.find(params[:id])
      rescue ActiveRecord::RecordNotFound
        render json: { error: "Student does not exist" }, status: :not_found
    end

    def valid_email?(email_address)
      !!(email_address =~ /.+\@.+\..+/)
    end

    def send_acceptance_email(email)
      NewStudentMailer.acceptance_email(email).deliver
    end
  end
end

The index, show, and create methods should be pretty straightforward, but perhaps I should elaborate more on the private methods. Within the context of building an API, we only need to focus on two: find_student and valid_email?(email_address).

The find_student method will query the Student model and its corresponding ActiveRecord database in order to find the student object with the ID attribute specified in params. In the event that it cannot find that corresponding student and Rails throws a ActiveRecord::RecordNotFound error, then it will execute a rescue clause that will render a JSON response with two components: the message “Student does not exist” and a 404 status code (“Not Found”).

The ‘valid_email?(email_address)’ method is simply a regex that will parse a parameter passed in, and determine whether or not it is a valid email address. If it is valid, it will fire off an email in the send_acceptance_email(email) method, but if it is not valid, then it will render a JSON response with an invalid format error message.

Strong Params in Rails

Recently, I ran into issues trying to pass a new variable into my params hash after integrating Devise into my app, so I decided to do a refresher of strong params. Strong params was implemented in Rails so that users could not maliciously manipulate form submissions via form fields.

Strong parameters was implemented in Rails 3 via whitelisting, which is the act of permitting specific attributes that can be passed into the params hash via the model. In Rails 4, the responsibility of whitelisting has now been passed to the controller.

Typically, we have a private method inside of the controller that delegates the whitelisting that should take place.

For example:

UsersController
1
2
3
4
5
private

def resource_params
  params.require(:user).permit(:name, :age, :email)
end

The require statement inside the resource_params method performs the parameter validation for the user parameter. If the user parameter exists, then it will go on and validate each of the attributes. If the user parameter does not exist, then it will throw an ActionController::ParameterMissing error and return a 400 status code response.

Additionally, the permit method will strip out any attributes that do not belong inside of the params. For example, if we tried to include a :admin attribute inside of the permit method, it will not be passed into the params.

Simple Authorizations With Devise

We are in the process of building an application that requires several levels of permissions for a User model. Because we do not quite fully understand the scope of what permissions will entail in future iterations of the application, we wanted to leave open the opportunity to expand the permissions scheme further. We made the decision to use CanCan and integrate with Devise. CanCan is an authorization library that defines permissions for different resources. In addition, Devise is a large authentication gem that’s used for Rails, and has modules that allow for a large degree of customization.

We added in the CanCan gem, and, while building out our login features, quickly ran into problems associated with the strong params component of Rails. For whatever reason, Rails was not able to properly create a new object because the authorization level integer (via bitmask attribute) was not able to pass through strong params. I quickly found out that we were not properly overriding the Devise params method that whitelists each of the attributes.

As a result, we decided to do away with CanCan and integrate our own permissions/authorization layer for our app.

Here’s how it is set up currently in our User model:

User Model
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
ROLES = {
   10 => 'super_teacher',
   7 => 'teacher',
   5 => 'student',
   0 => 'user' #default role set on creation
  }

  def self.roles
    ROLES
  end

  def super_teacher?
    self.role == 10
  end

  def teacher?
    self.role == 7
  end

  def student?
    self.role == 5
  end

  def user?
    self.role == 0
  end

We’ve also set up our Registrations controller to override the default that’s provided by Devise:

Registrations Controller
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
class RegistrationsController < Devise::RegistrationsController

  def new
    super
  end

  def create
    user = User.create!(resource_params)
    sign_in(user)
    redirect_to root_path
  end

  def update
    super
  end

  def resource_params
    params.require(:user).permit(:role, :email, :password)
  end
end

As a result, we can scope the view throughout our app by referencing the role attribute of each User. The app is very young at this point, but already a major piece of our app has been implemented. Looking forward to seeing where this will be going.

Rails Caching

Caching in Rails

What is Caching?

It is a tool that allows you to avoid extensive database querying on a page by storing elements of a page in memory and retrieving that memory store each time that page is visited. It enables faster page loading on refresh, and saves resources.

Cache Types in Rails

Page Caching

This is a Rails mechanism that allows a request for a generated page to be fulfilled by the webserver. This has limited uses, and can’t be used with pages that have before filters (such as pages that require authentication). It also requires that cache expiration be set explicitly. Since Rails 4, the page caching feature has been removed into a separate gem called actionpack-page.

Action Caching

Action caching is used where page caching can’t be utilized – such as with pages that require authentication. It is very similar to page caching, except that the incoming request hits the Rails stack so that before filters can be executed before the cache is served. Since Rails 4, the action caching feature has been removed into a separate gem called actionpack-action.

Fragment Caching

Fragment caching allows a fragment of view logic to be wrapped inside of a cache block and served out of the cache store when the next page request comes calling. Basically, the cache block is wrapped around logic inside of your view, and that cached view logic will be served to the page view until it expires. Then the cache process will start over again.

Cache Setup

Configuration Settings

You can set up your app’s default cache store by calling config.cache_store= inside config/application.rb or inside of your environment files in config/environments/*.rb.

Cache::Store

This is the foundation for interacting with the Rails cache. The class in Rails is provided via ActiveSupport::Cache::Store. There are four primary methods: read, write, delete, exist?, and fetch. The fetch method takes a block and will return an existing cache, or it will evaluate the block and write the result to the cache if a cache doesn’t exist previously.

There are four options that can be passed in to the config.cache_store= configuration. They are: - :namespace - Option is used to create a namespace within the cache store (useful when cache is shared with other applications). Default is the application name and Rails environment. - :compress - Used to indicate that compression should be used in the cache (useful for transferring large caches) - :compress_threshold - Used int conjunction with :compress to indicate a threshold under which caches should not be compressed (default is 16 kilobytes) - :expires_in - Sets an expiration time in seconds - :race_condition_ttl - Used in conjunction with :expires_in option to prevent race conditions when a cache expires (basically prevents multiple processes from regenerating cache entries simultaneously)

Cache::MemoryStore

This stores cache entries in memory. This has a size limit specified by the :size option (default size is 32 megabytes). When the cache exceeds the size limit, a cleanup will occur. This is not ideal for large app deployments, and typically works best for small, low traffic sites.

This has to be specified in configurations via: - config.cache_store = :memory_store, { size: 128.megabytes }

Heroku Cache Configuration Setup

In order to enable caching with Heroku, it works best with Memcachier, which is a Heroku add-on. Memcachier essentially manages and scales clusters of memcache servers for Heroku apps. See the link below for setup instructions.

Controller Testing via RSpec

I’ve been testing controllers pretty often over the last few weeks, and wanted to write a few reminders to my future self. I’ve been testing an app at work with RSpec, and have gleaned a few best practices to integrate into my unit tests for controllers.

First off, the purpose of controller testing is to test the controller actions directly and see what they do. For example, if we are testing a redirect, we want to make sure that the controller action is redirecting to the right path. It’s important to follow the AAA pattern: Arrange, Act, and Assert. You set up the test by arranging a before(:each) statement, and setting up the data needed to execute a test via let blocks. Then you act upon the arranged data by manipulating it to the test’s specifications. Then you assert that the arranged data should result in some specific action or result. Arrange, act, and assert.

Another tip to keep in mind. Avoid before(:all) statements whenever possible, because you hardly will ever need it, and it sort of goes against Sandi Metz’s Single Responsibility Principle (SRP). If you ever actually do need it, you are most likely dealing with a very extraordinary circumstance. The before(:all) block can affect test-data stability due to unwanted side effects.

For BDD/Rspec language, prefer to use active language instead of passive language. Of course the system should do something. Let’s write tests where the language asserts definitively that it does or does not do something, thereby documenting not what our system should do but what it does do. It’s important to be as clear and succinct in our language about what our expectations are.

Finally, a small but important distinction between let() and let!(). An object defined in a let() statement is lazily evaluated, meaning that it won’t instantiate that object until it has been called in a RSpec test. However, an object defined in a let!() statement is forcefully evaluated, and the object will be instantiated once the statement has been invoked.

Hopefully these thoughts and insights will prove useful to someone, as they will for me in the near future.

Foray Into Cucumber Feature Testing

For the past 2 weeks, I have been getting more involved in Cucumber feature testing as a result of my day-to-day job. When I first started, I found the prospect of Rails feature testing daunting, because it seemed like a cumbersome task to implement on top of building actual application features.

We had built out some features for a client project that I’m currently working on, and we had decided to implement testing for features via Cucumber, rather than solely relying on RSpec testing. There was a reason for taking this approach: when designing a feature, we didn’t want to overtest our feature to the point where we were overemphasizing testing of what could be a very basic feature. Cucumber allows for all-around integration testing without being overly verbose and cumbersome.

The first feature test I wrote was for a “Forgotten Password” feature. I had to test a User that had forgotten his/her password, clicked on the “Forgot Password?” option at the login screen, and expect to recieve an email with instructions to reset the password. So you would set up the feature test as follows:

Feature Story
1
2
3
4
Feature: Partner Password Reset
  As a Partner who forgot the password
  I need to be able to reset the password
  So that I can log in

The feature description describes the storyline in very simple, straightforward terms. You need three things when describing a feature: who, what, and why. As in English literature, those three questions are essential to building up any kind of story. Then I would describe the first scenario:

Feature Scenario
1
2
3
4
5
6
Scenario: Send Reset Password Email
  Given I am a Partner who has forgotten the password
  When I am at the 'Forgot your password?' page
  And I type in my admin email address
  Then it should send me a 'Reset password instructions' email
  And it should come from the default administrator email address

The scenario describes step-by-step how the feature is going to be tested, what the conditions are, and how the expectations should be met. You describe scenarios with three major keywords: Given, When, and Then. Given implies an implicit condition that is required for the scenario. When specifies an action or verb that should take place. Then explicitly states the result that should occur as a result of the action that took place under the When clause.

Once you run the cucumber command in Terminal, the Terminal will generate a list of steps, or tests, that correspond to the scenario that you have written out. The following is a such output of the reset password scenario discussed earlier.

Reset Password Scenario Steps
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
Given(/^I am a Partner who has forgotten the password$/) do
  #pending 
end

When(/^I am at the 'Forgot your password\?' page$/) do
  #pending
end

When(/^I type in my admin email address$/) do
  #pending
end

Then(/^it should send me a 'Reset password instructions' email$/) do
  #pending
end

Then(/^it should come from the default NYTM address$/) do
  #pending
end

Then(/^it should result in a 'not found' error message$/) do
  #pending
end

Once the Cucumber test steps are generated and incorporated into a step definition script, then you can start filling in the test code as it corresponds to your application. In this situation, I need to generate a user object (via FactoryGirl-Rails gem that has been included in the Gemfile), manipulate the user interface via Capybara methods. The finalized step definitions are shown below.

Reset Password Scenario Steps
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
Given(/^I am a Partner who has forgotten the password$/) do
  @user = FactoryGirl.create(:admin_user)
end

When(/^I am at the 'Forgot your password\?' page$/) do
  visit new_admin_user_password_path
end

When(/^I type in my admin email address$/) do
  fill_in "Email", with: @user.email
  click_on 'Reset My Password'
end

Then(/^it should send me a 'Reset password instructions' email$/) do
  @email = ActionMailer::Base.deliveries.first
  expect(@email.to.first).to eq(@user.email)
  expect(@email.subject).to eq('Reset password instructions')
end

Then(/^it should come from the default mailing address$/) do
  expect(@email.from.first).to eq('original@mailing.com')
end

Given(/^I am not a Partner$/) do
  @user = FactoryGirl.build(:admin_user)
end

When(/^I type in a non\-admin email address$/) do
  fill_in "Email", with: @user.email
  click_on 'Reset My Password'
end

Then(/^it should result in a 'not found' error message$/) do
  expect(page).to have_css('.inline-errors', :text => "not found")
end

Hopefully this gives you a general idea of how Cucumber feature tests are implemented. Each scenario that you generate must be short, succinct, and straightforward in regards to the feature that you’re testing. It took me a while to figure out best practices for implementing such tests, but I was able to get the hang of it over the course of two weeks. Just keep practicing, and see if the feature story makes sense to you.