Monolithic docker containers for local development environments

This post has an companion github repo using wordpress as an example, feel free to take a look.

In agency work there isn’t the same liberty to be able to deploy our lovely isolated docker containers. Often those environments are the clients, and they just want the git repo and the mysql database. This does not excuse developers from doing everything possible to try and match the production environment in their local development environment.

Often what developers ends up with is a working version of a stack consisting of an apache, mysql and php on their machine. Trying to add extensions or debug someone’s stack is usually pretty difficult because everyone ends up doing it a different way. Couple this with the fact that working on one environment, with one set of binaries and configuration, often is not going to reflect production in every project. Often things work well enough so these shortcomings are ignored. Configurations to support multiple projects running out of sub-directories with one vhost and .htaccess hacks are often culprits of easily avoided bugs, too.

What is the solution? I think vagrant comes really close but it’s a little too heavy and doesn’t do enough to abstract things like the memory, storage, networking of the vm. Essentially most people just want a container with exposed ports, mounted volumes, and an isolated environment, and that’s docker. Docker advocates for splitting up your services across multiple containers and that makes a lot of sense. However, I think it might be overkill for these basic php projects that a lot of web agencies get. I think there is a use-case for docker and running everything in a single container that is vm-esque.

This single-container approach gives you a lot of advantages like tracking your Dockerfile (and any future changes to the docker image) in your git repo, being able to run Docker with your mounted project directory, and just an overall quick and snappy setup. I have an example repo if you’re curious about an implementation of how this would work. Ideally, you would use composer or a some other package manager to track the framework and its dependencies leaving only a Dockerfile, your manifest file declaring your packages, and your application code, in your repo.

Be careful with those primitives types

This is probably a refresher for most, but I was curious about how js is handling typing. After all we have String , Number , and Boolean  global objects that have these wonderful prototypes that get us some really handy functions. So we can do something like:

Neat, and because we have these global objects we can augment their prototypes to give us access to extra methods on every instance. For example:

Ember.js augments prototypes to make things a little easier and to quickly get access to methods without having to pass it into an Ember object. This is something that requires a lot of responsibility, as there is some overhead involved and generally you don’t want to surprise people who share the environment with you.

Augmenting prototypes is also useful to polyfill functionality that might not exist, like Array.map in older versions of IE.

Prototypes also help with defining inheritance in javascript. We have useful operators like instanceof and  typeof to help us make sense of these. Where things get tricky is when you have a primitive like "hello, I'm talking loudly" being a string primitive, but also having access to the String  prototype, like how I added the exclamation  method.

We would expect that since we are using a method on the String prototype, and that "hello, I'm talking loudly" was able to access it, that “hello, I’m talking loudly” instanceof String would equal true, but it doesn’t. Oddly enough, typeof “hello, I’m talking loudly” equals “string”, and new String(“hello, I’m talking loudly”) instanceof String equals true.

If all that seems a little confusing it did to me, too. Here is a quick summary of what we’re looking at:

What is happening is that when you use “quotes” or call the function without the operator  new you actually are working on the primitive. The primitive isn’t an Object and therefore can’t be an instance. When you aren’t using the new operator, the function is simply returning the primitive. As you might recall, using the new operator on a function uses that function as a constructor and creates an instance that inherits from the function’s prototype. Check out the MDN resource for a better explanation, but essentially we’re dealing with an object.

How are we able to access these method’s on a prototype then? There is something called “autoboxing”, or more commonly as “primitive wrapper types”, happening behind the scenes that wires up methods of a literal to its appropriate Function. You can also do things like this transparently where these “objects” are handled appropriately:

Interestingly, the typeof  for each of these ends up being 'number'

The way these work also affect comparison operators:

In general, everything works the way you would expect it to except when you’re dealing with matters of “type”. If what you have is a string, number, or boolean and it’s created with the new operator you need to check instanceof String, Number and Function. If it was created as a primitive you need to check typeof value returning ‘string’, ‘number’, ‘boolean’. You could check if either returns true in a helper function, too.

Check out this stackoverflow post for some good discussion about checking types of string, and how using new shouldn’t be used considering the confusion and how unnecessary it is. While it might be in bad style I think it illustrates some of the details behind javascript’s inner workings. Also, I remember reading old javascript examples in books that make use of the new operator to try to ease people in from OOP backgrounds, so you can’t really escape that this exists. There are some interested reads on autoboxing/”primitive wrapper types” to check out, too.

Like many parts of javascript there’s always little gotchas that keep things interesting. Luckily as standards move forward and as people create libraries/frameworks/polyfills to pave the cowpaths, we will end up with an easier way to write javascript. I hope this made sense and if I made any mistakes, or need some clarify anything, please let me know in the comments.

Raspberry Pi wifi sleep issue with Edimax EW-7811Un

Although I have been reasonably happy with my current airplay speaker setup I ran into some issues where I couldn’t find the airplay speakers listed. I also couldn’t  ssh in, and on the pi directly  ifconfig wasn’t reporting an IP address either. I found myself having to continually run  ifconfig wlan0 up.

Turns out the Edimax EW-7811Un wifi adapter on Raspbian has an issue with very conservative power management. A quick google search turned up this forum post that worked me.

Essentially create a text file at  /etc/modprobe.d/8192cu.conf with the contents (may require sudo to write):

Save that, restart, and hopefully all is fixed.

Another small quirk, but still very happy with everything. In all this I also discovered that shairport-sync had been updated to 2.4 with some bugfixes, yay.

Finally, AirPlay Speakers

I’ve been toying with getting my airplay speakers setup with right combination of pieces and parts over the past few years and I’m happy to report that my tinkering has paid off. My last attempt which I didn’t report involved using Rune Audio, an ambitious attempt to consolidate the OS, audio drivers, media integrations (including airplay and spotify!), all wrapped in an easy to access web interface. Who wouldn’t want a one-stop install and get everything setup with all the bells and whistles.

Unfortunately, even with pushing through to the latest on their master branch I wasn’t able to get a solid experience out of Rune Audio. The UI was slightly buggy, and more frustrating was the dropping of airplay, and although it seems they might have fixed some of these issues I wanted something more barebones. The more I thought about it the more I realized I just wanted airplay. I didn’t need a web interface for this thing, I would use my phone or mac and simply use that for controlling, so much easier. If I want spotify, I’ll airplay spotify.

So, it was back to the drawing board. I started with a fresh install of raspbian, setup the wifi dongle, setup the audio configuration to default to my USB DAC, and lastly setup shairport. My previous instructions were for shairport 1.0, but since then there’s a brand new fork in town and it’s awesome. It’s called Shairport Sync and it allows you to sync to multiple sources (ie: multi-room setup) so that every airplay receiver running shairport sync is kept, well, synchronized. I only have the one device, so this wasn’t something I really needed but aside from this new feature it was just so much easier to install and setup. My previous instructions for the 1.x required some interpretation, but this setup worked perfectly based on the github instructions.

Happy to say that my idea of a stable, custom AirPlay speaker setup is finally complete with a raspberry pi, raspbian, audio USB DAC, wifi dongle, and shairport sync.

Callback currying, and futures (or a preface to Promises)

I have been diving into the different patterns that can be used to organize functional code. One pattern being curry, a nod to Haskell Curry, that arises from the need of generating a function that has one, or more, of its parameters already setup. I was looking for some good examples in javascript logic where this is applied and came across this blog post, and I saw a really interesting pattern at the section “Currying the callback”:

This little pattern looked really interesting to me, and I saw some parallels with promises. Essentially, returning a function that you can call at any time to handle the response, just like promises when you call  .then and pass in a function as a parameter.

To satisfy my curiosity and whether I was on the right track, I left a comment and Bruno was kind of enough to leave a response. Bruno confirmed my assumption, and left me with the realization that as we work on these concepts one of the best way forwards is to formalize these patterns. These ideas start with libraries, move to specs, then browser implementation, and maybe even push the language to include new features that can’t simply be polyfilled.

In any case, if you want to get an idea of how currying is used in this particular instance, and to get a little bit of insight in how promises are achieving this management of async, check out Bruno’s blog post. It’s also refreshing to see that the community is supported by people like Bruno to answer questions like mine on his blog.

Cleaning Up Whitespace in your Github Diffs [Bookmarklet]

Ever had those pesky whitespace changes show up in your github diffs? They’re usually just trailing whitespace, or from converting to tabs (or spaces if you prefer). They often aren’t what you are trying to focus on, and luckily Github has a way of removing them.

But adding a ?w=1 query param is a hassle. So here’s a way to shave a few seconds off with a bookmarklet. Drag and drop this link to your bookmarks bar 〉 Remove GitHub Whitespaces

Here’s the code un-minified in case you’re wondering.

While we’re on the topic of whitespace, if you are using SublimeText, there’s a handy plugin called TrailingSpaces that can highlight and delete trailing whitespace from your files.

Placeholder

These days it’s hard to represent all the moving parts of a webpage into a single PDF. We have animations, layout, state changes, and many other dynamic pieces on the page that are too easy looked over when approving a concept. The solution has been to spend more time on the wireframe process and ensure that the idea is built right into the browser to capture what is on the screen, but what is happening on the screen. Our hopes are that this extra time gives us the ability to stay on track with development, and not have one of those “Oh, I didn’t see x in the wireframe” moments.

These wireframes are often without aesthetic treatment so that the idea can be fleshed out before it’s decorated and polished. It is common that any layout or design will anticipate the need for images. At the wireframe stage it’s not likely these images are ready, so we put placeholders.

Placeholder images are easy enough for the designers who build things in photoshop, a grey box takes a few seconds. But what about for us, developers? There are services that are highlighted in this “Top 8 Placeholder Services” article, but they all rely on a physical images.

I can foresee a few problems with these services:

  1. They load from an online resource.
    We aren’t always connected to be able to pull these dynamically created assets
  2. The image is available offline as a physical asset, but requires setup and referencing a file in markup.
    I have to save the file, add it to the web directory, then jump back to my editor and reference the file in-line, track the physical file in source control, etc. Create a placeholder should be fast.
  3. We are left with pixel dimensions.
    We won’t always know how we want the image to fill the space, and often it won’t be confined to specific width and heigh pixels. Why get hung up on dimensions when you can just get something that symbolizes a placeholder, and represents what you do know about your layout (a width/height ratio, width/height percentages, etc).
  4. It’s an image.
    We don’t need an image, we need a placeholder that represents an image. Images bring with it baggage that we aren’t ready for, we can do more and still get the idea of the image.
  5. To address these issues I have come up with a side-project that I am calling, for lack of a better name, placeholder. You can check it out on github.

    In a nutshell this gives you the following ways to define placeholders:

    • SASS mixin
    • Javascript
    • Polymer Element

    Each of these methods give you multiple ways to define the way that the placeholder can take shape, including:

    • Width and Height
    • Width and ratio
    • Just a ratio

    Also, on top of creating these placeholders there are different styling helpers that can be applied to the containers to have the placeholder fit the visual language of your wireframes.

    The biggest benefits of doing it this way is that you are doing everything in code and it’s tracked in source control, it’s fast to bring in, it’s flexible in definition of the placeholder, and it can become part of the process that you and your team use to consistently create and use placeholders.

    I still have features I want to add, and features that need to be documented. If you feel like it’s a good idea give the project a ‘star’ on github, fork it, or just give it a try and give your feedback. I will continue to develop the idea further and smooth out any kinks.

    Thanks for taking a look!

On the go in 2014

It was my intention to have a 2014 resolutions post of various topics and progress I wanted to make for the year. Well, things move quickly and we’re already well into 2014 with lots happening, so I’m going to do a short look back instead.

In January 2013 I started with the guys at Hybrid Forge, here in Edmonton, Alberta. I started off as an E-Commerce Analyst, and ended the year as an E-Commerce Lead, specializing in setting up, theming and developing Magento. What a platform! Magento, based on the Zend Framework, builds upon many layers that can easily have your head spinning. As my start into the web as a profession Magento had no shortage of challenges. As time went on, I ended up working on other projects in the company, including some Drupal work. Outside of work, I was learning Symfony for a project, and also launched a few sites on Drupal. All the different ways these platforms accomplish similar goals gave me the foundation I needed to grasp the mechanisms used by larger codebases and frameworks.

About two months ago, I left Hybrid Forge to try and take the next step into the big world of web development. I was excited to see what else I could learn and accomplish. My friend Adam Braly, who works at Snapshot had a fun opportunity for me to go down to Southern Oregon to work on a website. The guys at Snapshot were great, and gave me a chance to practice my skills with a new team, and bounce ideas about development and process. The website was being built in wordpress and was my first time having seriously touched the platform. It was immediately apparent that there were some parallels between it and Drupal, although Drupal arguably with a more established foundation in the CMS space.

My 5 week Oregon adventure ended in Portland, which is neighboring the town my family lives in. Portland is an amazing and awesome city, which I love to visit. With it’s fun and laid back culture it was the perfect place for the Ember.js conference which I was able to attend. Back in August 2012, when I first decided to get back into web development, I wanted to play with a new idea and javascript seemed like the perfect candidate. I decided to dive into Ember.js making a basic app, getting a feel for how MVC could live in the browser, and how you could rely on data-binding to make your life much, much easier. The guys behind Ember, who also live in Portland, are big names and have contributed to some serious open source projects over the years. It was brilliant to see how far they and their team had come with Ember and where they plan to go. One of the best parts about attending the conference, which is the first tech conference I’ve been to, was to see how welcoming everybody was. It’s easy to feel like you know the least in the room, but it’s empowering when that isn’t what matters.

These past few months, and even through the past year, I have come to realize that everyone is always learning. As we learn, we develop, which is often consumed by others, causing them to learn and develop. It’s this gigantic ever-hungry feedback loop, accelerated by the web and systems like Github. It’s alright not to know, as long as you’re humble and hungry to be better. I am looking forward to 2014 and being able to talk more about what I have learned and contributed. Stay tuned.

Mac OSX 10.9.1 Crash on Sleep or Shutdown

I write this on my Mid-2010 MacBook Pro that is finally able to boot, sleep, and shutdown successfully on 10.9.1. It seems like there are variations of this problem on 10.9 and 10.9.1 that many are experiencing. Normally I wouldn’t take time to blog about random troubleshooting issues but this one in particular was a pain, and it’s my hope this will help someone.

The issue
Whenever my computer would go to sleep (either by itself timing out or from the apple menu), or when I would go to shutdown, the cursor would freeze in place and forcefully restart. Upon booting up the keyboard would be “locked” while I could move the cursor, after about 20 seconds the keyboard would become responsive and I would be able to login. I was able to reproduce the instant restart upon sleep or shutdown each and every boot, it didn’t matter when it happened. I noticed this happening after upgrading to 10.9.1, although I can’t be absolutely certain.

As you might imagine this is very frustrating. Essentially I would have to be okay with losing everything any time I would step away from my computer because the machine would eventually sleep. The weird thing is that when it crashes, it reboots, so I imagine it could be left looping over and over in the this cycle.

Troubleshooting
It wasn’t graceful and didn’t leave much in the system logs. Resetting the nvram and pram didn’t seem to do anything. Other forums had mentioned about turning on/off ‘Automatic Login’ in System Preferences.

The uncertain fix
Turns out the culprit was likely a start-up item. I booted into safe mode and was surprised to find that I was able to put the computer to sleep and wake up without any issues, similarly the computer was able to shutdown gracefully. Looking into the differences between booting into safe mode and normally, and it seemed it was mostly kexts and start up items. Using ‘Clean My Mac’, checking the ‘Login Items’ under System Preferences, and sorting through the LaunchAgents within /Library/ and ~/Library/ I was able to find a bunch of cruft from old applications. I removed these (you could probably rename them or move them to a folder on the desktop I imagine), and rebooted and like that I am able to put my machine to sleep, shut it down, and get back to work.

Hopefully this helps someone, at the very least just try to reproduce the issue by booting up in safe mode to rule out non-essential start up items.

Setting up the Best AirPlay Speakers (with Shairport and Raspberry Pi)

A plan that had started nearly a year ago is finally starting to see fruition (haha). My Raspberry Pi journey has been fun and you definitely end up learning a thing or two along the way, this isn’t your ordinary linux box/virtual machine.

I’ll jump right into the setup and configuration and then leave a few of my side notes after. Since getting started the Shairport project has garnered more support and is now back in active development. For the 1.0 release it was largely rewritten and from my testing is a lot faster and more stable.

Shairport 1.x, AirPlay and Raspberry Pi

Now I won’t re-write what others have done a good job at explaining already. The steps for getting Shairport 1.x installed is more or less the same as the previous tutorials that describe the forked shairport versions (hendrikw82/shairportalbertz/shairport) for Raspberry Pi but from my testing so far the Shairport 1.x release is a lot better.

I recommend reading through this tutorial as the steps are basically the same (relevant section starts from the beginning, which gets your raspberry pi up and running, to Setting up your Raspberry Pi AirPlay receiver), and then coming back and reading the Tutorial Differences for a few minor points so that you know when to follow these specific steps for Shairport 1.0.

Tutorial Differences Specific to Shairport 1.x

Where the steps differ however are:

  • Make sure you install these required debian packages:
  • git clone from 1.x repo
  • Use  ./shairport instead of  ./shairport.pl to test the app.

When you want to ‘daemonize’ shairport the steps are slightly different:

Fixing the init.d

  • From the shairport install directory (the one created from your git clone), go to scripts/debian by typing:  cd scripts/debian. Then to copy the init.d script and configuration enter the following commands:

    Shairport is installed to /etc/local/bin/ but the init.d script is looking for it in the wrong location so use vim or nano to edit the init.d script like:

    Then you will need to find, around line 17, change it
    from

    to

    Save the file (in vim this is done by pressing ‘:’ typing ‘wq’ and hitting enter.

    Change the permissions on this file so that you won’t have to run it via sudo by entering this command:
    sudo chmod a+x /etc/init.d/shairport

  • Configuring Shairport 1.x Daemon

  • In the previous step while we were in the shairport install directory we also copied the default configuration ( shairport-install-directory/default/shairport) to the /etc/defaults directory. Let’s edit that file and do a few minor tweaks: sudo vim /etc/default/shairport (again, use whichever text editor you are comfortable with, and put sudo in front). Since I didn’t feel like setting up a shairport user I uncommented the # USER= line so that it read USER=pi (Note: there is no ‘#’ on this line now). I also commented out the # AP_NAME= so that it reads AP_NAME=OntheAir (Again, the ‘#’ character that comments out this line has been removed). Change the OntheAir to whatever you want your AirPlay setup to be called, this is what you will see when you are choosing the AirPlay speakers from the menu. Save the file.

    This configuration approach is way cleaner than editing the /etc/init.d/shairport script to add your daemon arguments. If you’re looking for a little background reading, the configuration information for the raspberry pi and the shairpoint 1.0 I mostly found from this shairport github issue.

General Tips for the Audio

  • The Raspberry Pi doesn’t have audio set to the 3.5mm audio jack by default, if you’ve got it hooked up via HDMI, force switch to it with: amixer cset numid=3 1
  • You can also set the audio output levels, visually with: alsamixer (use up/down arrows), or by command with: amixer cset numid=1 -- 80%.
  • I am still using the default raspberry pi 3.5mm output, and it doesn’t sound that bad, so I’m really looking forward to what a USB DAC will do for me. That should arrive sometime this week, so I’ll report back my findings. I also plan on writing a short blog on some of the small things I’ve learnt about the Raspberry Pi, that might save others some headaches.

    Hope this provides some useful information, especially since the popular Shairport guides are for pre-1.0. Let me know if you have any questions, in the comments.