Hello, (Real) World

I will be graduating at the end of August after 5(!) years pursuing a Bachelor’s of Mathematics in Computer Science at the University of Waterloo.

I spent the past 6 months or so searching (between classes, assignments, and exams) for full-time work beginning in September. By my calculation, I have participated in approximately 60 interviews (including phone, Skype and on-site) this year. I’ve been extremely lucky to meet many talented engineers and managers at a some very interesting companies. Naturally, I’m vary grateful to all of them for giving me the opportunity to interview for positions with their respective teams.

I’ve ultimately decided to accept a position with Upverter . I am super excited to be joining this talented group. I love the idea of building a way to give other professionals similar sharing, collaboration, and version control tools to those that we programmers take for granted. I’m very happy to be joining the team at Upverter to do just that!

I’m also looking forward to living in Toronto again (except for the looking-for-housing part). It’s a great city with a many great events going on.

Posted in Uncategorized | 2 Responses

Numbers

Yesterday, Josh Matthews posted a very interesting blog about building up data about Mozilla’s community. It caught my eye.

It surprises me a little bit that we have so little data on our community, and I think it’s exciting that Josh has started the ball rolling on gathering data in the area.

I have a few questions for the wider internet about Josh’s post:

1) What do other open source communities do in this area? How do they go about gathering data about the community?
Other large open source projects must have their own strategies for community engagement. I wonder what they are doing to drive engagement. Are they making educated “shots in the dark” as Mozilla has been doing, or are there existing solutions to measure and benchmark the effects of community engagement processes?

2) How can we use the data that we gather to change our goals for community building and involvement?
This is a problem that many startups deal with on a day-to-day basis. There are always more feature and product ideas than there is time to implement them. The key challenge is to decide what to do, and that decision requires careful data analysis. Can we use some of the strategies and tools that for-profit startups use?

3) What other data should we be gathering? What other data visualization will help us make decisions and drive community involvement?
We have a goldmine of data in Bugzilla. How can we make more use of it?

Unfortunately, I don’t have answers to these questions. However, I do think it’s a great opportunity. We know there’s plenty of room to improve community engagement and we know the impact is huge. Let’s continue to explore, and let’s see if we can learn from other projects who have similar problems to solve!

Posted in Mozilla | 2 Responses

Maze Generator

I made a thing! ( Check it out here )

A few years ago I decided it would be a fun project to build a maze generator. My initial attempt was a failure, as is often the case. My second attempt turned out much better. It was written in Python using PyGame for the rendering and controls. It was my first experience using Python to build something, and it was a lot of fun.

This is my third attempt at the project. My main motivation behind restarting was to be able to easily share the work with others. This is the kind of thing that’s (arguably) fun to play around with for a few minutes at most. It’s hardly something that’s worth installing PyGame just to play around with.

So here we are, with a basic JavaScript & Canvas implementation of my Maze Generator and Solver. I’ve implemented two Maze Generation algorithms so far, more to come! Most of the algorithms I’ll be adding in the future are explained nicely here: http://weblog.jamisbuck.org/2011/2/7/maze-generation-algorithm-recap (Which I stumbled upon thanks to Blake Winton). So far I’ve got two generator algorithms:

Depth First Search

This one is a classic. It’s about as simple as it gets to implement. I find it a little bit boring to watch, but the mazes it produces are pretty cool, tending towards longer, winding passages.

Prim’s

I like watching this algorithm in action. The maze it produces tends to have shorter passages. I’ve found that the path to from start to finish is usually very straight, diagonally through the maze.

More to Come!

I’ll be implementing more maze algorithms as I have time. Please check out the source code on github if you’re interested

Posted in Maze | Tagged | 1 Response

Signing Mac Builds

For the past month or so I have been working with Ben to add functionality for the signing of mac builds to our current signing infrastructure. Along the way, we discovered a few things.

What is signing?

Code signing is the process of attaching a digital signature to a piece of software that allows the user’s OS to verify that the software does indeed come from its advertised source, and that it has not been altered since being signed.

Why do we need signing?

Apple’s recently announced Mountain Lion is going to be introducing a few new features. In particular the new Gatekeeper ( Learn more here ) is important for Release Engineering at Mozilla. The default setting for Gatekeeper will *only* allow Applications to be installed from The App Store or from Registered Developers. This means that any applications are not signed by a Registered Apple Developer will show Security warnings when users install them. To ensure that our users have a seamless experience, we need to make sure that our builds are signed correctly according to Apple’s standards.

Gatekeeper isn’t the only reason Code signing is important. Signing verifies that the contents of the Application a) come from the trusted developer and b) have not been altered. OSX also uses the code signatures to determine whether an Application is trustworthy enough to be allowed Keychain access.

Code Signing on OSX

Apple provides a tool for Signing Applications called ‘codesign’. The ‘codesign’ tool will apply a digital signature to an entire ‘.app’ directory.

There are two parts to the signature:

  1. A generated manifest containing hashes of each file in the directory. This file is located in ‘*.app/Contents/_CodeSigning/CodeResources’ and is generated according to a specified rules file (more on this later).
  2. A digital signature attached to the binary specified as ‘CFBundleExecutable’ in Info.plist . This signature also contains the hash of the generated CodeResources file.

‘Codesign’ will verify a given ‘.app’s signature with the -v option. (Be sure to add a second v to get ‘verbose’ output, otherwise the only indication of success is the return code)

$ codesign -vv Firefox.app
Firefox.app: valid on disk
Firefox.app: satisfies its Designated Requirement

Automating Signing

There are a few important details about code signing on OSX which were important to address.

OS Compatibility

Applications which are signed on OS 10.7 (Lion) do not verify on 10.5 (Leopard). Since we want to be running one set of servers to sign all of our builds, we need signed builds to verify on 10.5, 10.6, and 10.7. Luckily, applications signed on OS 10.6 (Snow Leopard) verify correctly on all three versions. We’ll be using 10.6 as the OS on our Mac signing servers at least until 10.8 (Mountain Lion) is released, at which point we’ll need to re-evaluate.

 

Keychain Popup

Apple’s ‘codesign’ command require an signing key, or ID, which is stored in a Keychain. The ID and Keychain are passed as arguments to ‘codesign’. Unfortunately, ‘codesign’ does not offer any way to enter the password at the command line. If it tries to access a locked Keychain, it will pop up a UI prompt, asking the user for the Keychain’s password.

The bane of release engineers worldwide

The bane of release engineers worldwide

Thankfully for those of us who like to automate things, the ‘security’ tool comes to our rescue here. ‘Security’ is another Apple tool that allows command-line access and manipulation of Keychains. And ‘security’ does allow the Keychain’s password to be entered either as an argument or in response to a terminal prompt.

$ security unlock-keychain ***.keychain
password to unlock ***.keychain:

One important thing to note about the ‘security unlock-keychain’ command is that it will only unlock the Keychain for the current security context. In particular this means that if the Keychain has been unlocked by the command being run in a terminal, it is not unlocked for any ssh connections into the machine.
To user ‘security unlock-keychain’ without user interaction, the password needs to be entered at the tty level. Pexpect is one option for python which will wait for the command-line prompt and enter the password.
Now we’ve got all the tools we need to automate signing with the following steps with no user interaction:

  1. Unlock the keychain with ‘security unlock-keychain’ and enter the passphrase using pexpect
  2. Run the ‘codesign’ command
  3. Lock the keychain again with ‘security lock-keychain’

Code Resources

The CodeResources file is used by ‘codesign’ to specify which files in the ‘.app’ directory need to be included in the signature and which files do not. Before signing, the CodeResources file contains a set of rules. After signing, the hashes of all files specified in the rules will be added to the CodeResources file.
This page by Apple provides some description and examples of the CodeResources file, but it fails to explicitly state an important fact: any path specified in the CodeResources rules will be interpreted recursively to include all of the files in that directory and below.
The CodeResources rules must specify all files that need to be included in the final signature. Any files not listed or not in any of the recursive includes will be ignored, and will not have their hashes added to the signed version of the CodeResources file. Any files that are included in a recursive listing of a directory can be explicitly omitted. The file specified as ‘CFBundleExecutable’ in Info.plist is never included in the signed CodeResources file. Instead it is signed directly by the ‘codesign’ command.

The following example will use this simple directory structure:

Test.app
    Contents
        Info.plist
        MacOS
            binary <-- the binary file listed in Info.plist
            anotherfile
        Resources
            omittedfile
            resourcesfile

Before signing, we specify a set of rules in the CodeResources file. These rules will be passed to the ‘codesign’ command when it is called. The following code specifies these rules:

  • Recursively include the ‘MacOS’ directory and the ‘Resources’ directory
  • Omit ‘Resources/omittedfile’ from signing.
  • Include ‘version.plist’ in the signature
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<dict>
        <!-- Recursively include 'MacOS' and 'Resources' dirs -->
		<key>^MacOS/</key>
		<true/>
		<key>^Resources/</key>
		<true/>
        <!-- Omit a file -->
		<key>^Resources/omittedfile$</key>
		<dict>
			<key>omit</key>
			<true/>
			<key>weight</key>
			<real>1100</real>
		</dict>
		<key>^version.plist$</key>
		<true/>
	</dict>
</dict>
</plist>

The after signing, the CodeResources file has a list of the hash of every file specified in the rules. If any of these files are changed, ‘codesign -vv’ will fail and will specify the changed file. Notice that the omitted file does not have its hash listed here, nor does the Info.plist file, which was not included explicitly, nor in any of the recursive includes. The binary file has been signed separately by ‘codesign’ and is not included in this file.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>files</key>
	<dict>
		<key>MacOS/anotherfile</key>
		<data>
		K2QRL8qPGNuexZXewkmnMlmcdKU=
		</data>
		<key>Resources/resourcesfile</key>
		<data>
		PhrNXR4JvoBSBHWVsGOxbosr2po=
		</data>
	</dict>
	<key>rules</key>
	<dict>
		<key>^MacOS/</key>
		<true/>
		<key>^Resources/</key>
		<true/>
		<key>^Resources/omittedfile$</key>
		<dict>
			<key>omit</key>
			<true/>
			<key>weight</key>
			<real>1100</real>
		</dict>
		<key>^version.plist$</key>
		<true/>
	</dict>
</dict>
</plist>

That’s it for now

Look for Mac signing servers in the next few weeks at Mozilla. I hope this helps us provide a more seamless user experience (maybe including Keychain access?), and readies us for the release of OS 10.8 (Mountain Lion).

Posted in Mozilla, Releng | Tagged , , , | 10 Responses

Hello, World

Well, 8 months after it was first suggested to me that I should set up a domain and blog, here I am.

I’m in the 4th(ish) year of my undergrad in Math and Computer Science at the University of Waterloo and I’m (at the time of this writing) in the middle of an internship with the Release Engineering team at Mozilla; . I’m a big supporter of free and open software and I’m excited to be a part of Mozilla.

I’ve spent the last month or so automating the signing of Mozilla’s OS X builds. Keep and eye out for some upcoming blog posts detailing the process and some of the discoveries I’ve made about code signing tools on the Mac!

Aside from Software Development, I enjoy Ultimate Frisbee, Starcraft, and a bit of Squash and Bouldering/Climbing here and there.

Posted in Uncategorized | Tagged , | Leave a comment