06.13.07
Builds suck
Builds suck. Normally I’d rag on Ant, but I don’t really blame the tool (even if it does suck…). Ant doesn’t kill kittens, developers do. I think the fundamental problem is that nobody (ESPECIALLY developers) take a build seriously. The sad reality is that we treat a build system the same way we used to treat regular code. We write it once, maybe throw in a few echos, and run the build. We check to see that the files we expect it to create are there, and then we forget about it. That is, of course, until we notice that it stops working (usually LONG after it really stops working). This is ridiculous. There’s a reason we have jUnit.
I was talking with my partner in crime, Nilanjan about this. The project we are on has an insane build system. The last couple of days I’ve had to immerse myself into it. Ick. The sad thing is that it is not even close to being the exception to the rule. So, we started talking about what we think the best way to avoid this kind of intertwining dependencies and properties nested an insane number of imports away… Here’s the list we came up with (keep in mind that a fair number of things here we haven’t tried, they just seem to make sense. If you try any/all, please let me know. I’d love to know how they work out):
Your build is code. Treat it as such.
- Your build has pretty set requirements that show up over the course of your project. For example:
- given a java file, create a class file
- given this war file and this running app server, deploy the war to the app server
- We need some sort of unit testing framework that allows you to verify that your build still works as expected
- should allow you to fake out all calls to the os and the filesystem so it will be fast and nondestructive (when testing a task to deploy to your production server, you should mock out the sftp call in some way)
- should mock out all other imported build files. I think it would get too complex to test when your build file depends on another build file that depends on another build file… This probably is debatable, however.
- Refactor mercilessly. Please.
Nested builds are a tree
- Nested builds are when you have multiple sub-projects, each with their own build file.
- You should only be able to call build files down the tree, or siblings (more on this in a bit)
- A fair amount of the complexity I personally have seen deals with any build file being allowed to call any other.
- It hurts to figure out who exactly depends on a specific task.
- Following this rule, at least gives you a smaller set to look towards when debugging your build.
All projects should have the EXACT same public interface.
- By public, I mean the tasks that other build files can call
- not just what can be called from the command line
- If one project has a “jar” task, all should. And the jar task should do the same thing, in a common build file (see below)
- That way, when project A requires a jar from project B, it knows that all it needs to do is call the “jar” task and it will be in a known place.
- This doesn’t mean that all build files are limited to the same interface, just that if you need to call a task in another build file, all build files need to provide that task in a consistent way.
- This is probably the weakest suggestion of all of them. Just something I came up with. Blame me, not Nilanjan.
Refactor out commonality.
- When factoring out common tasks, the common build file can not depend on any other files.
- Trust me. One off is best.
- This means that if your pull up the compile-unittest task that depends on the compile task, both need to be pulled up.
- Only use one common build file.
- Again, even though it might seem to make sense, debugging becomes hell, especially when you factor out the common tasks between two common files. Arg.
- If a task to be pulled up is not used in EVERY build file, turn the task into a function (or macrodef in ant) and call it.
- Functions are much easier to test than tasks with dependencies (any of which could possibly be overridden)
- They should not depend on the state of any sort of global variables or properties. Only on arguments passed in.
- No global variables/properties
- This will drive you mad if you depend on an imported build file to define your state.
- Insanely hard to debug.
Due to the fact that I’m not suicidal and don’t like to cry (it really hurts your credibility as a consultant… I’ve… heard…), my initial stab at writing a unit testing framework for a build system will be using rake. I’ll post something when I have anything even started…
Comments?
06.05.07
The Big Apple
This last weekend I went to New York for the first time EVAR. Here’s what happened:
Thursday night
- I took the train from Philly to Manhatten. It was not too expensive, so I went first class. I’m a huge fan of the train. You get on, like, 5 minutes before it leaves, you can use your phone/computer the entire trip, and there’s no crazy “Must have your explosive liquids in a ziplock bag before we can let you thru security” crap going on.
- Reshma picked me up at the station and we went to her place to pick up her best friend Shaun. He’s a cool guy, even if he does work for the enemy…
- We all met up with Tiffany and [insert other members of our party here]
- We decided to go to a restaurant that is super old and has old (sometime famous) people’s pipes on pretty much every wall and the entire ceiling. It was super-tasty
- Went and saw times square at night. I mean, I assume it was night based on my watch and tiredness level, but it was so brite I’m not convinced.
Friday
- Slept in to around 10-ish and then took a shower. Pretty much everybody I knew in NY was going to be working, so I was on my own for the touristy stuff.
- I decided to go to the WTC site (more on that below)
- Next I went to the statue of liberty. I had a phone call I had to be on at 2, so I didn’t take the farie over there, but I did see it from a distance. Turns out, in that case, the photo’s do it justice just fine
- I started walking around the financial discrict, had a street vendor hot dog, and saw the brooklyn bridge
- All in all, it was a good site seeing trip
- I headed back for my phone call, and then I took a nap. I was up a little late chatting the nite before, and Chris needs his beauty sleep
Friday Night
- Had a hot date.
- We went to [insert name of place I can’t remember here]. It had awesome food. I was way impressed.
- Next, we had tickets to see Mary Poppins on Broadway. Now, if you know me well at all, you know that the Mary Poppins movie is consistantly in my top 2-3 favorite moves of all time. I have to say, I was impressed. They took out a few things from the movie, and added a few things from the book (there’s a book?!), and technically it was pretty amazing. Bert even walked all the way up the arch on stage right, along the top of the arch and down on stage left. It was pretty sweet.
Saturday
- Got up and took care of some email and errands I needed to do. Reshma woke up at 2pm. Yes, 2. Seriously.
- Headed to brunch (If you can call 2:30 brunch-time) with Shaun and Reshma. We ate at a french restaurant (they were out of quiche.
) - Next we decided to go to the Museum of Modern Art. I wish I could say it was amazing, but by the time we got over there it was only going to be open for another hour.
- Instead, we went to Central Park, stopping on the way to see the coolest apple store in the world. Awesome.
- We walked thru Central Park for a while. This place is awesome. I was seriously blown away. The insane contrast between how packed and fast paced the city is, compared to the open and mellow park is exquisite (I even looked up how to spell exquisite. That’s how much I love this park)
- We headed back to get ready to meet up with some other friends later that night.
- Met up with the other friends. For the record, Julia is the funniest person in the world. Seriously. I haven’t laughed that hard in a very long time. (Now it’s on the internet, so it must be true)
- We had wine on the terrace (with an amazing view I might add), and then headed out for sushi. Good stuff.
- We went to a little bar in some area or another (I know, very specific, eh?) and had a few drinks.
Sunday
- Went out for brunch again, this time to a place called Cafeteria. It was pretty fun. Good food.
- Next, I was on my own again since my friends had to go back in to work (insane, right?). I decided to go to the moma for real this time.
- Didn’t go to the moma again. I got there, realized that in order to get back to get ready for my train I was only going to have 1 hour to see the museum again, and decided not to (again). Instead, I walked in the park. I totally [heart] Central Park.
- I decided I was going to walk the long way thru the park. I get all the way thru it and reach the street. I was pretty impressed with myself. That was, of course, until I discovered that I had gotten turned somehow and ended up on the west side of the park, only about half way thru it. Dang. I decided to give up, since time was slim and it was starting to sprinkle.
- On my way back, I get a voice mail (phone didn’t ring for some reason) saying my train was canceled. So, I get back to the apartment and find my ticket and get the time changed.
- Went to dinner with Reshma and Tiffany at a chinese restaurant they call Room (only because the chinese characters look like the letters “Room”). It was pretty good.
- Went to the train station and headed back to Philly (after another hour of it being delayed…)
World Trade Center
I have to say that the thing that surprised me the most about this trip was seeing the world trade center. The entire time I was looking at the site, or at the pictures that were on the fence, or the timeline that was there, I didn’t feel a thing. I was really expecting some sort of emotional flailing, but nothing. My guess is it is a little bit of me being out of touch with my emotions (what, me?) and a little bit of complete desensitization by mainstream media, but it really did surprise me. It felt like just another construction site. Weird.
05.31.07
Parenting Lessons from Animal Husbandry
This is pretty funny. This guy used to teach animal husbandry to teen volunteers at a wildlife rehabilitation center. Later he had a kid. He assumed that his experience teaching the teens would help him understand raising his kid. Turns out, his experience working with animals was better training.
Favorite quote? “Besides, the coolest things in life are gross.” Too true…
The take away for me? Babies are animals.
05.11.07
Apache vs. Yaws
Ok. I know I already posted today, but this was just cool. A performance diagram comparing Apache to Yaws (An Erlang webserver). Apache dies at around 4,000 parallel sessions, while yaws still works at 80,000. Amazing.
EUnit vs ErUnit
I recently was pointed to this article by pragdave about test-first development in Erlang with EUnit. I’m going to use his examples to show you how it would be done in ErUnit instead. First, some history…
The reason I wrote ErUnit instead of just using EUnit is for a couple of reasons
- The best way to learn a language is to reimplement xUnit. There’s so much meta stuff going on that it really requires you to dig in the language. And when I say reimplement, I don’t mean, “Follow the xUnit spec exactly.” What I mean is leverage the things that make the language good to create a good fit.
- I didn’t see it on the web. Mostly because I assumed EUnit was for the E programming language. So, I searched for ErUnit and found nothing…
- Even after I found out about EUnit, it didn’t gel with me. Now, that might be due to there not being any good tutorials that I could find, but I just wasn’t seeing it. I don’t like using macros specifically for assertions as it makes the failures hard to debug.
- I really don’t like having your tests live in the same file as your production code… It makes your code much harder to understand if you have to double the size of the file just to test it…
- I’m not a big fan of the standard that I saw of using pattern matching for the tests. I personally think there should be a difference between a failed assertion and a failed pattern match within your code, and I don’t know how to make that distinction.
A huge benefit of going with EUnit is that it is MUCH more mature than ErUnit. I’m still tweaking it as need arises. One thing that I’m not sure if it really matters yet is all tests are run in their own thread, concurrently. I did that because it seemed super easy and fun to do, but I don’t know if you really get any benefit with that yet…
Well, on to creating a word wrap utility, test first, with ErUnit.
ErUnit in >60 Seconds
When creating a ErUnit test, you first name the file starting with “test_”. In our example below, when testing “text.erl”, we create the test file “test_text.erl”. ErUnit, with erunit_suite, has the convention of either:
- If your code and tests are in the same folder, add that folder to the path
- If your code is in the “ebin” folder and the tests are in the “ebin_tests” folder, adding both folders to the path
This allows you to break up the tests from the code it’s testing pretty easily. Within the test class, you have to add the erunit_test behaviour. Behaviours in Erlang (well, technically the OTP library in Erlang) are used to define what callback functions are required. In this case, the erunit_test callback requires a function call “tests” with no arguments to be exported.
I personally also add a no argument “run” function that just delegates to erunit:run() with the tests passed in. I also like to import all of the erunit functions I’m going to be using (test/2, assertEquals/2, run/1) and the functions I’m going to be testing. This makes the code later on easier to read.
The file so far is:
-module(test_text). -behaviour(erunit_test). -export([tests/0, run/0]). -import(erunit, [test/2, assertEquals/2, run/1]). -import(text, [wrap/1]). run() -> run(tests()).
As mentioned earlier, you must declare a no argument function called “tests”. All this returns is an array of tests. So, for the first example we want to test that the “wrap” function works with no words.
tests() ->
[
test("Should wrap no words", fun() ->
assertEquals([""], wrap([]))
end)
].
When you use the erunit:test/2 function, you pass in the description of what you’re testing and a fun/0 of the actual test. assert(Description, true-false-test), assertEquals(First, Second), assertEquals(Description, First, Second), and fail(Description) are all currently implemented. Behind the scenes, the “test” function spawns a new thread that will run the fun/0 and send the answer back to the current test runner thread.
The code for the “text” module is the same as in pragdave’s example, so I won’t repeat it here. In fact, I’m just going to show you the entire test file:
-module(test_text).
-behaviour(erunit_test).
-export([tests/0, run/0]).
-import(erunit, [test/2, assertEquals/2, run/1]).
-import(text, [wrap/1]).
run() ->
run(tests()).
tests() ->
[
test("Should wrap no words", fun() ->
assertEquals([""], wrap([]))
end),
test("Should wrap one word", fun() ->
assertEquals(["cat"], wrap(["cat"]))
end),
test("Should wrap two words", fun() ->
assertEquals(["cat dog"], wrap(["cat", "dog"]))
end),
test("Should wrap when more than 10 chars long", fun() ->
Expected = ["cat dog", "elk"],
assertEquals(Expected, wrap(["cat", "dog", "elk"]))
end),
test("Should allow word larger than 10 on its own line", fun() ->
Expected = ["cat dog", "hummingbird", "ibix"],
Actual = wrap(["cat", "dog", "hummingbird", "ibix"]),
assertEquals(Expected, Actual)
end)
].
The output when running from erl is:
1> test_text:run(). ..... ok 2>
When a test fails with this extra test added:
test("This will fail", fun() ->
erunit:fail("Called erunit:fail/1")
end),
…you get…
1> test_text:run().
....F.
-"This will fail" failed:
Called erunit:fail/1
fail
2>
To show an error, this is the test:
test("This will have an error", fun() ->
erunit:assert("This method does not work with a integer", wrap(1))
end),
…which produces…
1> test_text:run().
....E.
-"This will have an error" had error:
error:function_clause
{erunit,'-test_process/3-fun-0-',3}
fail
2>
…Not the best error message, but I’m working on it.
05.03.07
m15m
I recently rediscovered the wonderful Movies in 15 Minutes. If you’ve never heard of this, you really need to check it out. Cleolinda Jones is a genius. *joygasm*
She also has a book out. The odd thing is that while she’s born and raised here in the united states, you can’t actually buy her book here. I bought my copy from amazon.co.uk.
Here are a couple of excerpts from a couple of the online movies: DUMBLEDORE: We are gathered here now to choose one person (who is not Harry Potter) to represent each school, but only one (who is not Harry Potter) will achieve eternal fame and glory in the Triwizard Tournament, in which Harry Potter is not old enough to compete! [Only characters we’ve already seen in closeup are chosen: Krum, the famous one! Cedric, the pretty one! Fleur, the uppity one!] SNAPE: Professor…? About that… THE GOBLET OF FIRE: *coughHARRYPOTTERcough*
Harry Potter and the Goblet of Fire in Fifteen Minutes
The Champion Is Selected
The Trophy Room [All the professors race downstairs so everyone can shriek at Harry in privacy, while the newly-chosen champions watch uncomfortably.] MOODY: Oh, come on, Potter didn’t do it! Look at the kid, he can’t even comb his hair! HARRY [choking]: This—this is not mellow! FLEUR: L’awkward.
DUMBLEDORE [throttling]: DID YOU DO IT, HARRY? DID YOU PUT YOUR NAME IN THE GOBLET? ANSWER ME, YOU LITTLE SHIT! JUST GO AHEAD AND DIE IF YOU’RE THAT STUPID, SEE IF I CARE!
BORDEN: YOU TAKE THAT BACK! [And thus, a rivalry is born.]
The Prestige in Fifteen Minutes
Julia’s Funeral
ANGIER: WOLVERINE COULD TOTALLY KICK BATMAN’S ASS.
Angier’s Cunning Plan OLIVIA: So what you’re saying is, you want me to go to Borden, offer to work for him, tell him you sent me to find out his secrets, and then somehow actually get him to tell me his secrets? ANGIER: Basically, yeah. OLIVIA: You realize that this is probably going to involve me sleeping with him? ANGIER: Ride ‘em, cowgirl.
Angier’s Plan: Not So Cunning As Previously Thought BORDEN: So what you’re saying is, Angier sent you here to work for me and fake me out by telling me you really came to find out my secrets, but you really really came here to sell him out? And I’m supposed to believe that? OLIVIA: You’re forgetting that I also kind of hate him now. BORDEN: Hey, we have something in common, then. OLIVIA: So, sex? BORDEN: Sure thing.
05.02.07
Chris-mas comes early?
I just received an email from Hammacher Schlemmer announcing their brand new sale:
Words fail me1. I’m just wondering who decided that May 2nd was the best time to sell/buy pre-lit chris-mas trees… I suppose the bigger question is, “Will they stay lit all the way into December, or should I just open my presents now?”
1. Which, of course, won’t keep me from commenting
04.30.07
CITCON North America ‘07
This last weekend I attended CITCON in Dallas. Awesome. I had some amazing conversations, watched Bret Pettichord blow me away with customizing RSpec, actually met Elisabeth Hendrickson (and received a sweet green wristband that says, “test obsessed”), joined in a great conversation about what we are looking for in a testing language, and was surprised to discover there are people out there that actually like ant (to be fair, though, he does work on AnthillPro). The only downside was there were 2 topics at the same time that I wanted to go to, so I attended a talk on code metrics and missed out on the CI server cage match (There were like 6 different CI Servers represented at the conference, and they all got in a room to compare features and benefits of each).
If you’ve never been to a conference in an OpenSpace format, you’re really missing out. The short (-ish) version is that there are no preassigned speakers for the conference. On Friday, we all sat in a circle and people could write ideas down on a 3×5 card stating what they want to talk about. It might be some tool you want to show off, some concept you want to tell people, or, just as likely, something you know nothing about and what to learn. The only requirement is if you suggest the topic, you have to facilitate it and help get the conversation going.
After we had about 30 topics, we set them up on a board showing the rooms and time slots. Then, we go to the next room to drink and chat. As the night moved on, people we encouraged to wander back over to the schedule and rearrange topics, vote on what they want to see, and combine topics together. If you see two topics at the same time you really want to see, you just swap one of the topics with a different one so you can see them both. While you would think this would cause all sorts of problems, it turns out that, in general, people aren’t jerks, so you don’t get a constantly shifting schedule as much as a gradually improving one. A key element of OpenSpaces is the rule of 2 feet. If you don’t feel you’re getting the most out of a session, it’s up to you to walk to a better session. If you suggest a topic and nobody shows up (Actually happened for one of them), feel free to move to a different one. This puts the onus on you to get something from the conference. There are plenty of smart people attending, so if you don’t learn anything, you must have been actively avoiding conversations.
One of the interesting things is that the conference is free and takes place from Friday night to all day Saturday. The effect of this is the people that attend are doing so from a very real desire to learn. They are spending their free time to improve their craft and learn from their peers. It really raises the bar of the attendees. It’s amazing how much a difference actually caring about what you do can make.
Well, I guess that’s about it. There are 2 more CITCONs this year: CITCON Asia/Pacific 2007 in Sydney, July 27 & 28, 2007 and CITCON Europe 2007 in Brussels, October 19 & 20, 2007. I’m pretty sure I’m going to go to the one in Brussels, (London 06 was just as awesome as this last one). If you’re in the area, I can safely recommend attending.
Final thoughts: When I first heard about CITCON, I focused on the “Continuous Integration” part of the name (which I didn’t have much passion around). I think it’s important, however, to understand that Testing, Agile, TDD and other xp-ish stuff are just as represented (if not more so). I’m glad I decided to go.
04.23.07
ErUnit - Unit Testing for Erlang
So, I just wrote about my new favorite language Erlang. When learning a new language, it usually helps me to work on a simple problem I’ve solved before to understand how things work. So, I grabbed one of the coding exercises we use in recruiting at t-dub (no, I’m not going to post about them). Now, as every good xp-er, I started writing my test. Or, I should say, started looking for an xUnit framework for Erlang. Nada.
So, I wrote one myself. Test-first, I might add. (Trust me, it doesn’t get much more meta than that).
Download me. (or svn co http://codeby.bestfriendchris.com/svn/ErUnit/tags/erunit-0.1.0/)
To install, put it in the Erlang code path (on my OSX machine, it is /opt/local/lib/erlang/lib). Also, if you want to use the erlang script, add the bin folder to your PATH.
To write tests, you just need to name your test file test_whatever_you_want and add the following to the file:
-behaviour(erunit_test). -export([tests/0]).
I’d show you some more example test code, but my code highlighter doesn’t support Erlang, and I don’t really have the time to hack a new language file together for it right now. So, I’ll link directly to the test case I used in writing ErUnit: test_erunit.erl
To run tests from the command line, type:
# run all tests in the current folder
erunit
# run tests in the “tests” folder
erunit tests
To run the test from erl, type:
% run all tests in the current folder
erunit_suite:run()
% run tests in the “tests” folder
erunit_suite:run(”tests”)
A few more notes:
- If you use seperate folders ebin and src for your .beam and .erl files respectively, ErUnit has the convention of using ebin_test and test. Again, look at the way ErUnit itself is packaged to get the idea.
- erunit_suite will run all tests in every subfolder of the one it is run in.This allows you to have some sort of hierarchy to your code and tests.
- erunit_suite is pretty smart about loading test files. It will make sure that all of the folders the test.beam files are in are included in the code path and auto-magically handle the case where you are using seperate ebin and ebin_test folders. Subfolders within either ebin or ebin_test are not yet supported.
- You might notice that a bunch of the features of most other xUnit implementations are missing. This is, at least in some cases, on purpose (in others, it’s because we’re talking 0.1 version). I’m only adding features as I need them. For instance, I haven’t added Setup or Teardown methods, mostly because I’m not yet convinced that they will be required in a functional language. And even if they are needed, it’s pretty trivial to wrap your test functions in a setup/teardown pair of funs.
- No Documentation? Yeah. I’ve actually been sitting on ErUnit for at least a week because I wanted to write some good docs for it. Since that hasn’t happened yet, I figured I’d at least put this out there and maybe get around to it eventually.
Comments/Suggestions/Complaints?