Nomad is coming!

By Samantha Quiñones on by

I’ve been in the software engineering business for a long time, but until I was introduced to the PHP community, I never really cared to engage in the social aspect of this profession. I really feel that the PHP community is something unique. It’s rare to find a community that stands on the firm foundation of long-established friendships that is also so incredibly welcoming to newcomers.

I am still staggered by the friendship and support that has been given to me by people in the PHP world. The commitment that people have in the PHP community to raise one another up and to share their triumphs and frustrations, should be a model towards which every tech community should aspire.

One great way to get involved is to attend Nomad PHP, a monthly on-line miniconf where devs come together to share their knowledge and experience. I am immeasurably proud to have the opportunity to team up with Nomad PHP this month and share something that I’m very passionate about. Please join me April 17th at 8pm CDT to talk about the REST API, where it came from and where it’s going, and how you can build better ones.

REST is more than just a buzzword in a recruiting email. It describes a pattern for organizing and delivering content that is tremendously appealing because of of its relative simplicity and low overhead. Understanding where the idea of REST comes from, and how established and emerging standards are used to construct REST APIs, will give you the edge you need to build highly effective APIs that are a breeze to scale, test, and maintain. You definitely don’t want to miss it!

About Samantha Quiñones

Samantha is a development team lead at Politico.com. The opinions expressed on this site are hers and hers alone.

Project Laravel — Session 1: First impressions

By Samantha Quiñones on by

I was super excited when I found a couple of free hours to finally sit down with Laravel and kick off my little blog project. That two hour time box turned out to be pretty important to this effort. I hit up puphpet.com to get a quick-and-dirty little debian development server running and loaded up the Laravel quick start. Unfortunately I glossed over the very clear instructions about the necessary permissions for the app/storage path and lost about 10 minutes trying to figure out why I was white screening, but from the moment I sat down to the point where I had a functioning skeleton installed totaled maybe 20 or 25 minutes. I started in on my first goal, a simple database-backed authentication system.

I won’t go in to details on a topic as well-trod as that, but I think it speaks extremely highly of Laravel’s approachability that I was able to build out a reasonably rich authentication system in a little more than an hour, including command-line tools to provision and manage users. I’ve worked in a number of PHP frameworks large and small, from Codeigniter, Fuel, and Yii to Zend and Symfony, but I have never felt as immediately productive in any of them as I did sitting down with Laravel.  That alone goes a huge way to explaining how this framework has managed to enchant such a huge segment of the greater PHP community. The prospect of the drop-in structure of a framework without the steep learning curve is incredibly alluring.

Almost alluring enough to offset some of the things that I wasn’t quite as comfortable with. I want to say, I went in to this determined to give facades a fair shot but they didn’t make a great impression. I am the kind of person who likes to know what’s going on under the hood and facades seemed determined to get in my way. I don’t like feeling frustrated trying to find how something is implemented, and I found myself in that position a handful of times during this first exposure and it left a definite bad taste in my mouth. Maybe it’s just my lack of familiarity with the framework, or maybe as a python dev, I just have a bias towards the explicit over the implicit, but if it weren’t for the fact that facades are optional, I think my experiment might have ended right then and there.

So, the takeaway from my first brush with Laravel is that it’s a very welcoming piece of technology that invites you to step in and immediately start creating stuff, which is awesome. I’d love for the non-facade style to have a more prominent place in the docs. The idea is explained very succinctly by Taylor Otwell himself on his personal blog, but I think it would do a lot to quell some of the criticism– not to mention, make the framework a bit more welcoming to people, like me, who just reflexively distrust “magic”–  if the DIC itself were more visibly in the docs.

Next session I’ll be building my content management back-end and wiring up mongo. I’m really looking forward to doing some real work with Laravel at last!

About Samantha Quiñones

Samantha is a development team lead at Politico.com. The opinions expressed on this site are hers and hers alone.

Jamming with the Feds

By Samantha Quiñones on by
Getting to work at the White House

Getting to work at the White House

“What on Earth is a Data Jam?” is precisely the question I asked myself yesterday afternoon when an email inviting me to participate in a White House Data Jam on STEM Workforce Quality, Flow, and Diversity landed in my inbox. Reading through some of the supplied documents made things only a little bit clearer. Despite the confusion and misgivings, I figured being asked if I want to come to a White House anything is a good enough reason to apply the Jim Halpert maxim and simply reply, “absolutely, I do.”

So that’s why at a shockingly early and frigid hour this morning, I found myself trudging down a quiet street in downtown Washington on my way to the Eisenhower Executive Office Building. After clearing security, I made my way upstairs to a large conference room, poured myself a cup of coffee (sweetened with splenda from yellow paper packets emblazoned with the Seal of the President of the United states!), and took a seat among a sea of a bunch of very, very smart people.

It was through the introduction that the point of all this finally became clear. I was aware of the Obama Administration’s support for government open data initiatives, which I praised in a very detached, “hey, open data’s cool” kinda way, but I was not at all in tune with the very real ways in which the Federal government is trying to put this stuff in to practice.

One of the big problems, as any open data advocate can tell you, is that governments amass just gigantic quantities of data in the course of their day-to-day… governing, but there aren’t often clear ways to use it or even consistently access it. This is data, of course, that’s held by the government in trust for the benefit of the the people as a whole, but it really doesn’t do a lot of good to anyone if it’s locked away in some database that no one’s ever heard of.

And so this is where Data Jams come in. I think we’re all so conditioned to see the government as hopelessly bureaucratic and glacial that something like a Data Jam is actually a little bit shocking. It’s a simple concept: get a bunch of government stewards of various datasets, potential consumers of that data, entrepreneurs and other private sector people in closely related fields, and some software developers together in a room and come up with 5 or 10 really good, actionable ideas for making that data work for the American people.

Since I’m not so sure about the etiquette of these things, and some folks were already a little on edge having someone from POLITICO hanging around**, I won’t go too deeply in to the proceedings, but I have to say that I was amazed by what we accomplished in 6 short hours together. Brainstorming with a neuroscientist, a senior manager in the Federal service, an open data specialist, other developers, and human resources gurus really stretched my brain out in weird ways that I have to say I really enjoyed. And coming away from it, I feel like we actually did hit on some good, practical ideas that have the potential to make the Federal service, and the American workforce in general, just a tiny bit better.

I guess most importantly to me, it was a reminder that we can all serve society in our own ways. Not all of us are meant to be soldiers or politicians or diplomats. I’m just simple programmer who really likes building cool APIs, and if I can help, anyone can.

**I feel the need to make a strong disclaimer here that while I am employed by POLITICO, I’m in no way a journalist and I have no pretensions of being one. I have a tremendous amount of respect for my colleagues in the journalism organization at POLITICO and while I’m always proud to represent the company, my participation today was primarily as a member of the broader technology community.

About Samantha Quiñones

Samantha is a development team lead at Politico.com. The opinions expressed on this site are hers and hers alone.

Project Laravel

By Samantha Quiñones on by

I’ve always been pretty candid about the fact that I’m not a big fan of frameworks. I think they’re great for prototyping and for very simple applications. In my experience, though, whatever time you save starting your application up is lost when the inevitable quirky requirements and unexpected complexities force you to eventually extricate parts of your app from the framework you built it in.

That said, I have a number of friends who happen to be big Laravel fans and I’ve told most of them, at one point or another, that I’m willing to give it an honest chance before I write it off completely (like most other “full-stack” frameworks.) Patryk Kruk, one of my colleagues at POLITICO, is a particularly persistent voice in my ear both to give Laravel a shot and to actually update my blog on occasion, so I dedicate this project to him.

This site is sitting in a dusty corner of a shared hosting account that I’ve had sitting around for years. It’s well past time to modernize and move it. That’s my project: rebuild this personal site (in my spare time between work, personal projects, and other commitments) using Laravel and move it off this shared hosting account that I probably should have closed during the Bush administration.

The Parameters

  • Use all the “main” features of Laravel (i.e. Eloquent, facades, etc.) at least once.
  • Gotta have serviceable blogging functionality that isn’t a complete PITA to use.
  • Blog my progress at least weekly, even if it’s just to say I didn’t do anything.
  • Make the code freely available.

That’s all there is to it, really. At least I’ll finally be able to get the Laravel fans in my life off my back.

Or maybe I’ll end up joining them?

About Samantha Quiñones

Samantha is a development team lead at Politico.com. The opinions expressed on this site are hers and hers alone.

SkiPHP Talk Slides

By Samantha Quiñones on by

SkiPHP has been a blast. The organizers have done a fantastic job putting together an amazing conference!

If you’re interested in viewing the slides from my talks, they’re available on speakerdeck, so please check them out!

Demystifying the REST API
Designing Service Architectures for the Cloud

About Samantha Quiñones

Samantha is a development team lead at Politico.com. The opinions expressed on this site are hers and hers alone.

Juggle Chainsaws, Not Types

By Samantha Quiñones on by

No matter how popular an activity it is, I really don’t like to bash on PHP. Every language has its flaws when you look closely enough, and if PHP wears its idiosyncrasies a little closer to the surface than most, I think it makes up for it in other ways. PHP’s handling of types, however, is confusing at best and at worst completely deranged.

I’ve seen intercity rail schedules that can’t hold a candle to PHP’s type comparison tables. The bizarre and unexpected behaviors that can result from a non-strict comparison has all but rendered the equality operator (==) useless. The typical advice from PHP master to PHP neophyte is, always check for identicality (===) unless you’re very sure what you’re doing.

In fact, as part of our pre-screening process at Politico, we often ask PHP developer candidates to explain why the following expression evaluates as true.

("politico" == 0);

This question probably wont stump a PHP developer with any significant experience. If you manage to code in PHP for more than a few months without running face-first in to this issue, you might not be trying hard enough.

PHP is weakly typed in much the same way that water is slightly dry. When faced with the expression above, the interpreter tries to make sense of things by converting the string in to a number, following a single and actually quite well documented set of rules. To put it simply, if the string contains a ‘.’, or the letter ‘e’, PHP will attempt to turn it in to a float. Otherwise, it turns it in to an integer. If the string starts with “valid numeric data,” those numbers will be used as a value of the new integer (e.g. (int) “123foo” -> 123). If the string doesn’t start with a number, the value will be 0. The literal string “politico” starts with a ‘p’, which is not a number, and so the interpreter rewrites the above expression like so:

(0 == 0);

The obvious second half of that question is, how do you make PHP give you an answer that isn’t quite as… insane? You use the special identicality operator (===), more commonly called the triple-equal. This operator compares the types of the values in the expression, and if they don’t match, it doesn’t go through the whole circus act and just returns false.

Cute, right? Maybe until you realize that this behavior happens to catch a lot of unsuspecting programmers, especially ones who are new to the profession. Sometimes it creates subtle and hard-to-reproduce bugs, and sometimes it creates serious vulnerabilities.

Once in a while, it’ll even catch a more experienced programmer.

I was recently implementing a fairly simple class with a setter method. The method accepts one argument and checks to see if it is in a list of allowed arguments before setting a protected property. If the argument is not in the list of allowed arguments, an exception is thrown. These arguments are configuration options that are generally expressed through the use of pre-defined constants and our code is merely storing the value.

public function setFoo($foo)
{
    $validOptions = [VALID_CONSTANT_A, VALID_CONSTANT_B];
    if (!in_array($foo, $validOptions)) { 
        throw new InvalidArgumentException("foo must be one of: " . implode(", ", $validOptions)); 
    } 
    $this->foo = $foo; 
}

Even for a setter method, this is pretty unremarkable. Yet when we run our unit tests, we get an interesting failure. Our test that passes in an invalid argument expects to see an exception, but it doesn’t. In fact, this method accepts almost any string as valid. And despite my being way more familiar with PHP’s weird type handling than I really want to be, it still took me several minutes to figure out what was going on here.

Those constants are a form of abstraction, hiding an implementation detail of one of our class’s dependencies that we shouldn’t have to care about. If the dependency ever decides to change that implementation, using the constants means we don’t have to alter our code to keep up. But ignorance is often misguided bliss because VALID_CONSTANT_A is actually set to the integer 0, and by default, the in_array function doesn’t do a strict, type-safe comparison.

Let’s make a quick fix…

public function setFoo($foo)
{
    $validOptions = [VALID_CONSTANT_A, VALID_CONSTANT_B];
    if (!in_array($foo, $validOptions, true)) { 
        throw new InvalidArgumentException("foo must be one of: " . implode(", ", $validOptions)); 
    } 
    $this->foo = $foo; 
}

In case you didn’t notice the change, in_array accepts a third argument, a boolean which forces strict comparison when set to true. Now our unit tests pass and our crisis is averted.

Maybe there’s a parable here. It’s a pretty vivid illustration of why unit tests are worth our time and effort, if nothing else. Still, it nags at me. At the very best, silly things like this eat up valuable time. I don’t even want to consider the very worst.

About Samantha Quiñones

Samantha is a development team lead at Politico.com. The opinions expressed on this site are hers and hers alone.

Speaking at Ski PHP, Jan 17th & 18th, 2013

By Samantha Quiñones on by

I’ll be talking about Service Architectures and REST APIs. Should be exciting! Hope to see you there!

 

https://www.skiphp.com/attend/schedule

About Samantha Quiñones

Samantha is a development team lead at Politico.com. The opinions expressed on this site are hers and hers alone.

Pushing Bamboo Notifications to Datadog

By Samantha Quiñones on by

In my last post, I mentioned that where I work, we settled on Bamboo as our continuous integration solution. I love Bamboo, but its built-in notification features are pretty lacking. In fact, your only options out of the box are XMPP messaging (with no conferencing, so meh) or email (yech).

I <3 Datadog

Another tool we use at Politico is Datadog. If you’re not familiar with it, Datadog is an amazing hosted devops tool that combines rich visualization of metrics data with very nifty event streams and quasi-social groupwarish features. Datadog is changing how devs and system admins work in our environment, and we’re increasingly looking for ways that it can make our lives better.

Datadog is our central event management platform, and having our build events there cuts Bamboo off from the rest of our notification pipeline, and that’s just the pits. Over the past day or two, the Politico Tech Lab has been abuzz with increasingly Rube-Goldbergian ideas for how to solve what boils down to a very simple problem: how do we push a tiny blob of text across the Internet. Eventually we came to the idea of setting up an XMPP server that both Bamboo and our IRC bot could long in to, with the goal of teaching the IRC bot to forward messages to Datadog’s API. Yeah, silly.

Then, suddenly, the clouds parted and the heavenly chorus picked up and the solution presented itself as clear as crystal: python.

Python to the Rescue

Python’s standard library includes a simple, but very functional SMTP server implementation. Yeah, that’s right, we built a mail server.

The code’s very short, but we can take a quick tour.

import asyncore
import email
import socket
from dogapi import dog_http_api as api
from smtpd import SMTPServer

We’ll need the basic asynchronous socket server module, email (to tame Bamboo’s HTML mishmash), the socket module, the basic SMTPServer module from smtpd, and Datadog’s API, which is available from the cheese shop with a simple ‘pip install dogapi’.

class Server(SMTPServer):
    def __init__(self, dd_api_keys, host="127.0.0.1", port=4242):
        self.dd_api_keys = dd_api_keys
        asyncore.dispatcher.__init__(self)
        self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
        self.set_reuse_addr()
        self.bind((host, port))
        self.listen(5)

Now we just have to extend the SMTPServer class. We’ll need to take in some datadog api keys (as a tuple, in this case), as well as an address to bind to and a port to listen on. SMTPServer itself extends asyncore.dispatcher, so we need to init correctly before setting up a socket to listen on.

    def process_message(self, peer, sender, recipients, body):
        api_key, app_key = self.dd_api_keys

        message = email.message_from_string(data)
        title   = " ".join(self.message["subject"].split("\n"))
        text    = message.get_payload(0)

        api.event(title, text)

The method ‘process_message’ will be called any time a mail client delivers a message to our server for delivery. We’ll get information about the peer as a standard (addr,port) tuple, the sender’s address as a string, a list of recipient email addresses, and the message body also as a string.

At this point, we go ahead and unpack those api keys. The python email module will break up the raw email message enough to make things simpler. We can easily grab the subject to use as our event title. Bamboo sends multipart messages with the plaintext body in the first part, so we just use get_payload to pull out the 0th index.

Calling api.event will fire the event off to Datadog.

if __name__ == '__main__':
    apikey = 
    appkey = 
    server = Server((apikey, appkey))
    asyncore.loop()

And last, we instantiate our server and start our little server up.

Wrapping it Up

In our actual implementation, we ended up massaging this data a bit more to clean things up and make the events conform more to our own internal needs and desires. This code is what remains after that stuff has been chopped out, so NB, I haven’t even attempted to execute it and may not run as is. It should be enough to get you started on your own sneaky little SMTP server

 

Further Reading

About Samantha Quiñones

Samantha is a development team lead at Politico.com. The opinions expressed on this site are hers and hers alone.

Why you should consider Continuous Integration

By Samantha Quiñones on by

Last night at DCPHP, I gave a short “lightning” talk on Continuous Integration and Deployment in the context of PHP applications. I really like the lightning talk format as it forces you to focus on the meat of your topic– there’s no time to get distracted by details. As great as that is, though, I wanted to expand a little on the subject and touch on some of the details that I couldn’t include in my presentation.

What is Continuous Integration?

Continuous Integration is a grand-sounding name for a very simple concept. Every time you make a change to your application’s code (or, at least, on a very frequent and recurring basis), you build and test the entire application. Each change is integrated in to the application immediately.

Of course building and testing your entire application after each and every commit seems like a big task, and it is. Because we’re building so frequently, it’s essential that our building and testing happens automatically. Human intervention in the process has to be limited only to occasions when a build fails. When everything builds successfully, it should happen in the background like magic.

What’s Wrong With Traditional Build/Deployment Practices?

To answer that question, it’s important that we think about everything that usually goes in to building and deploying an application, specifically a web-facing PHP application. Very often there are configuration files that need to be modified. Bootstrap scripts and autoloaders need to be updated. Sometimes we find ourselves writing scripts to move files around, or update databases.

Hopefully part of the process involves running some kind of automated tests.

And once all of that is done, and it’s time to deploy, we find ourselves staring at an FTP client. Often we use our VCS as a deployment tool, which is great because typing ‘svn update’ is so simple, but how many of you have ever left a .git directory or some other VCS artifact hanging out there for the world to see?

Sometimes we’re stuck copying and editing files by hand in production, which is the kind of thing that gives me nightmares.

To put it bluntly, building and pushing by hand sucks. It’s extremely error-prone and however unintuitive it may sound, the more you do it, the more likely you are to screw something up. When it’s two in the morning and you’re tired and you’ve done it a million times before, your attention wanders. You forget a step. You push the wrong thing out to production or forget to run a database update script and now you have a fire on your hands. Several years ago I was manually moving a production application between two servers for routine maintenance. It was something I had done hundreds of times before. I forgot to scp a file over and brought the system up on a month-old copy of the database. It took two days to set things straight again. That was a lesson I don’t think I could ever forget.

If that’s not bad enough, doing it by hand encourages another kind of laziness. We’re developers and when we see problems, it’s in our nature to fix them. “It’s just a simple fix,” we’ll tell ourselves. Maybe no one’s noticed it’s broken yet and a quick edit to that one file will save the headache of a problem report in the morning. Maybe we’ll remember to commit the change to the repo as well, or maybe in six weeks that bug will show its ugly face again the next time someone deploys the unedited version of the code from git.

There is a Better Way

Continuous Integration is one of the practices of Extreme Programming and Agile Development. When you think about it, it’s a perfect fit. Agile development is all about short, tight iterations. User story, enhancement, build, release. Lather, rinse, repeat. Martin Fowler and Kent Beck first described CI back in the late 90s and Martin’s original article is still essential reading for any team looking to adopt the practice.

Before you can get started with CI, though, you need to prepare your codebase. If you don’t have automated tests, you’re going to need them. Going continuous relies on having confidence in the state of the application and the only way to build that confidence is through automated, repeatable tests.

You’re also going to need an environment that’s as similar to production as is reasonably possible. Today, thanks to the ease of setting up virtual machines (what up, vagrant and puphpet!?), there’s no excuse to develop and test in Windows and then deploy on Linux anymore.

Once your codebase is ready, it’s time to start automating your builds. There are tons of tools out there to handle build automation, from ye olde make(1) to rake and phing and ant and maven and so on. I’m personally a fan of phing (which is inspired by ant and php-focused), but feel free to use whatever you’re most comfortable with.

While you’re evaluating build automation tools, don’t forget about your databases! If you use an RDBMS and don’t use any database migration tools, start! Database migration tools are like version control for your database, and they provide a framework for deploying database changes in a sane way. Many major PHP frameworks include a migration tool, and there are plenty of standalone tools like dbdeploy, liquibase, and alembic (it’s not just for python projects!).

Now the Real Decisions Have to be Made

What does a good build look like? How much code needs to be covered by tests before you feel confident to deploy? Should a build fail because a developer failed to follow a coding standard or forgot to add auto-documentation tags to her comments? Should a build be considered successful even if the number of findings from your static analysis tool doubled since the last release?

These are questions that every team needs to answer for themselves, but when you think about it, having to consider what concrete, objective criteria need to be met before you deliver software is a great problem to have!

This is a good time to start looking at Continuous Integration servers. There are tons of them out there, both free and commercial, but they all perform essentially the same functions. The CI server is the tool that’s going to monitor your VCS repository for changes, run your build automation scripts, notify your team when things go poorly, and kick off your deployments.

In a future article, we’ll explore each of these stages in more detail, so stay tuned!

Further Reading

About Samantha Quiñones

Samantha is a development team lead at Politico.com. The opinions expressed on this site are hers and hers alone.