Dynamic quality and open source

April 6th, 2008

I was just about to give up on this blogging thing - it’s really easy to put thoughts up, but much more difficult to do write anything with any lasting quality. It’s hard to remember that this medium is, for the most part, _intended_ to be junk.

Now, if something, anything, is intended to be junk, why bother investing time and effort? The result of effort may only be a temporary solution to a problem, like a hack to get code out the door, or scrap wood pulled into use as bracing. Similarly, there may not be value in a high quality solution for a particular problem. Consider most of the products people go to big box stores for.

I can understand this perspective - much of what we do daily does not require quality, regardless of what advertisers say. Focus on getting the best quality breakfast and you’re likely to have a hurried, low quality commute. You’ve got to pick your battles.

Where does this put blogging in daily life? Pretty far down the quality/priority chain, by most measurements. It tends to result in a low quality result, and requires an elective effort to get there. Maybe it’s _supposed_ to be some kind of transient art effort, a way to add an equal voice to the world, giving thoughts of the moment. It’s all logged, indexed, and archived, though, so it’s not very ephemeral. As for art, not a lot of blogs qualify on that front either. They’re mostly junk.

So why do blogs exist? It’s rare that a blog is directly solving a problem, temporarily or permanently. By their nature, they’re more _talk_ rather than _action_, which may be why they’re so popular. Since they don’t _do_ much, little is invested in their creation, which explains the general lack of quality.

By this chain of reasoning, blogs are low quality temporary (but yet permanent) inputs to solutions to problems in which they rarely take any active part in implementing. Hmm. Why am I here again?

On the other hand, aggrandizing is one act blogs can directly take part in, and they frequently do. This probably why they’re so tightly bound to the tech industry. They’re great at aggrandizing both people and products. (Ah, right. That’s why I’m here.)

To the point: I value quality, and this view I have puts blogging low on the value scale. To that end, I intended to give up on the idea again, likely revisiting it in another 8-10 years. (My previous effort in the late 90’s provided nearly the same value as the current attempt.) I just finished “Lila”, though, and had some thoughts I wanted to put forward as a last effort.

Some background - Lila, by Robert Pirsig, is a sequel to his “Zen and the Art of Motorcycle Maintenance”. It’s an expansion of his metaphysics of quality, which is an attempt to get to the roots of quality and value. The first was much better, in my opinion. Lila got further down the road of defining quality, but Zen took a much more interesting path.

Before I get into that, though, here are some Cliff notes. First, on Zen:

Pirsig is attempting to define quality, which is a hell of a difficult task. What makes one song better than another? How about a painting? An essay? Some people prefer high quality clothes to junk, or an iPod to a knockoff. What makes one better? Different people will give different answers, all according to their quality perspective. One person might like a BMW’s leather interior, another the engine characteristics, another the electronics. I see low quality due to the poor fuel efficiency. Sorry, but it proves the point - there is no specific quality metric.

The question here is not whether quality exists, but how does one define it? If it can be quantified, surely we can use this definition to be sure we ‘build quality in’ to what we do.

First, Pirsig finds a cleaving point that can divide quality into two camps - the classical and the romantic. Classical quality boils down to quantifiable categories and hierarchies - like engine parts, components, and subsystems. By looking at these categories, one can define the quality and value of a motorcycle based on the components that make the motorcycle work.

The other half, the romantic quality, has to do with the motorcycle as an ideal - the thought of the open road, wind in the hair, freedom. The intangible, and mostly socially inspired, aspects of the cycle fall under this half of the split.

As the book digs deeper into the roots of how we define quality, Pirsig goes all the way back to Plato, the roots of rhetoric, and the Sophists. He digresses into some interesting areas, but the end result is this:

Quality is not inherent in an object, or the observer. It is _created_ as part of the relationship between an object and the observer. This is why the definition can vary - based on our values and perceptions, each of us perceive quality in different ways. Each of us has a different relationship with the same object.

As he discusses classic(scientific)/romantic split he covers some interesting territory relating to the scientific method. How do we work through a problem? What rules do we use to decide what tools we use? What conscious methods decide our approach, and which are subconscious? The art of problem solving, Zen, and scientific method overlap in some powerful ways - he makes a convincing argument that the scientific method itself can only open up infinitely more possible solutions to problems, and Zen methods can help navigate these infinite possibilities to find and create quality.

The metaphor uses motorcycles, but could have just as easily used software. How do some developers nail the source of problems right off the bat, often without proper tools, or even data? It could be argued that they’re subconsciously using Zen and quality appraisals (gut feel) to come to the proper solution. Maybe a particular area of code is known to have little in the way of quality by some metric inherently understood, and the path to the root of the problem can be intuitively chosen from this.

This is partially why software varies so much in quality. Think about the fact that just to _debug_ it, many developers are searching for the root of a problem like traveling salesmen, trying to avoid local minima (low quality is the global maxima) using a metric (quality) that is arising dynamically between the subject (developer) and the object (code). Attempts to use the scientific method alone simply increase the space infinitely. Very few developers have the time or tools to effectively bisect a diverse system into problem and non-problem areas. Mathematically certifiable systems are a way out of this mess, but unfortunately, there’s more talk than action in this area. (Blogs, again)

On to Lila, before I get too distracted. At this point, Pirsig’s got a decent handle on the quality issue, but wants to dig a little deeper. So, the trip he takes us on isn’t as exciting, but he does get an important point across - he makes the distinction between static and dynamic quality. This is what I wanted to talk about.

Think of static quality as the patterns in life that are good. (high in quality, of course) It doesn’t matter what - it could be a holiday dinner, ’summers spent at the lake’, and so on. If it’s something settled in as a static pattern but with high quality, that’s static quality.

Dynamic quality is the new and exciting stuff that comes along. Pirsig’s got a perfect example of a great new song - the first time you hear it, it’s just fantastic. This is dynamic quality in action. It can come out of nowhere, and rarely follows a predictable pattern. As you hear the song played over and over through the next few months, it becomes less fantastic. Still a great song, but the edge (dynamic quality) is wearing off. This is the conversion from dynamic to static quality. After a few years, it’s just a part of static culture, and can be heard as part of the daily rotation on radios in motorcycle repair shops.

This distinction is really important, because it’s at the root of a lot of technology, at least in software. Static patterns are static because there are social norms around them; big, heavy weights that can be moved, but only over long periods with a lot of effort. Static patterns can be high or low quality , but whatever it is, it’s likely to stay that way for a while - we know where it is, we know where it came from, and to a large degree, it’s clear where it’s going.

Dynamic quality, on the other hand, is all over the map. It can come out of nowhere, on a whim; an idea drawn on the back of a napkin, an idea in the shower, etc. But we all know these ideas can vary greatly in quality, too - a brainstorming session gone awry is probably just as likely to produce complete garbage as it is a billion dollar idea.

Software tries to fall pretty squarely in this dynamic area. It’s so fluid that an idea can be pulled out of nowhere and thrown back almost on a whim, at least compared to trying out a new way to land a rover on Mars. This flexibility results in a large variance in quality. An astounding amount of bad software is written, but for all the bad stuff, some good does come out.

Software’s fun because it’s dynamic, and not bound by all of those static rules. And with the right idea, people, time, and effort, some really high quality work can be done, way out on the edge of the norm, away from all of those static patterns. You can call it the ‘Wild West’ or the ‘Next Big Thing’ or sometimes, maybe just a better way of doing what we already do.

The real fun part is that no one has any idea what in the hell they’re doing. The dynamic edge of quality is an emerging point, where there are no rules yet, no reference points, just a gut feeling about the right way to get done what needs to get done. You can come out with a great idea and execution, if you follow and create quality, down whatever seemingly random path the dynamic quality muse is travelling. If you lose the muse, or your gut’s read on quality isn’t quite right, you’re likely to go down in flames.

I’ve done dynamic development before. When you’re on the right quality trail, the work is just there for you. The right tasks are obvious, largely without surveys, market feedback, and so on. The high quality path is just taken. It may look like no one has any idea what they’re doing, but somehow it’s clear. There are no hard requirements, very few success measurement metrics, but yet the product does the right thing and customers love it. I consider this the high quality dynamic programming environment. (Some specific analysis tools were applied to our work later, and confirmed high quality levels, at least by the tool’s metrics.)

Open source development largely follows this model, with varying results in quality, just like there are other commercial development shops around that try to be dynamic but without the muse. There are some great projects out there, way out on the dynamic edge, and there’s some real trash. Amazing trash sometimes, really - looking at the minefield of what’s out there.

The OSS world isn’t just dynamic, though - some of it is very static. Many projects are just replicating existing software, like many of the core GNU packages - tar and the like. These projects can have high quality, or low, but of the static variety. In fact, most of the GNU utilities are (have?) far higher static quality (is quality a noun?) than what they’re modeling themselves after. There are a number of low quality static projects, but it’s easier to achieve higher quality when you’ve got something to compare to, like an existing cultural and technical static pattern.

This brings me to the other important point from Lila, the concept of latches. All of this quality is important: It’s important that static patterns (cultural (technical)) are of high quality, and it’s important that all this work around emerging dynamic quality is also good. But dynamic quality is, well, dynamic, and our static patterns are very static - it’s hard to change them. Without some way to fold this dynamic work back into the static world, it’s lost.

Here’s where the latches come in - Pirsig’s of the opinion that in order to make use of these new dynamic advantages, we need to have a latch where the static patterns are ‘upgraded’ to fold the advances back in. If this doesn’t happen, the culture doesn’t advance, and eventually stagnates and/or collapses on itself. Or some better culture comes along that has advanced and takes over.

An example latch would be the US Bill of Rights. That document took a number of high quality dynamic patterns and made them static. Without some serious effort, it’d be hard for us to fall back below this point. Sure, there might be efforts to regress, and anything is possible, but it’s not easy.

These latches don’t happen all the time - remember, static patterns don’t want to change! But it’s important that they’re out there, as a collection of new higher quality patterns, with the new lower quality patterns filtered out. When the static culture is ready for a change, it lurches forward, grabbing this latch, and internalizes it. The idea is that when this happens, these dynamic ideas _become_ static, like that new song I mentioned above. Culture can (and does) continue to progress from here, but if it starts to fail for some (environmental, technical, economical) reason, it only falls back to this latching point and not all the way back to the stone age.

I work on a lot of things, but I like to think at the core, this is what I’m doing - creating latches. We stay just ahead of that static edge, right where we can see the new dynamic and higher quality static patterns emerging. Through work and selection, we add to and collect the high quality work that’s out there, and turn it into a ‘latch’. In practical terms, a ‘latch’ for us is a software release. (And no, I’m not saying our Linux releases are equal in importance to the Bill of Rights.)

Once the release is out there, it can serve as a new cultural latch that is slowly internalized a number of ways. Over time, it becomes the static norm. At the same time, new dynamic work and advancements are continuing! Pretty soon, there are a lot of new high quality dynamic advancements out there, along with a solid smattering of improvements to static processes.

While the industry is internalizing the previous latch, we’ve been collecting and crafting these new advancements into a new and better one, letting the cycle begin all over again, improving all the time.

Dharma seems like most apt metaphor to wrap up with, but I’ll leave the process of thinking through the relationship there as an exercise for the reader. Have fun!

Omnivore’s Dilemma

February 6th, 2008

It took months, but I’ve finally finished it. For being such a useful book, it was amazingly hard to get through without falling asleep. Something about Pollan’s writing just puts me out. It’s interesting, well structured, well written, but just not exciting.

I would recommend it, as long as it’s understood that it could take a very long time to get through. The interesting thing is that I wouldn’t want to read it quickly. By absorbing the content over the course of a few months, I was able to think through many of the chains of logic and resources put forth. It’s one thing to read a chapter, it’s another to carry that specific information around for a week when wandering the aisles in the grocery store.

LASIK/PRK, part 4. (one month in)

February 1st, 2008

One month checkup - I can drive now, but still have a lot of halos. PRK can take up to 6 months to fully heal, but I’m definitely pretty far along. My vision is around 20/40.

I can tell that I have the visual acuity to see clearly, but the halos create an artificial fuzz around everything. I can drive now, but seeing street signs is still a bit hard, especially at night.

I only have to scale fonts up about half as big as I did as of day 4 or so, so I’m getting there!

Hydroponics shops in Berkeley

January 30th, 2008

Props to both Berkeley Secret Garden and Berkeley Indoor Garden - great hydroponics stores, and not head shops. (Also really easy to get to!)

Sure, there’s that understanding that some of the people going in are growing a little more than tomatoes, but at least they know what they’re doing, the staff are really helpful, and the place is organized.

That’s a lot more than I can say for the other places visited in the last week. It’s one thing to play reggae in the store (as these two did) but the other shops? Wow - I was surprised that they could operate the cash register.

Yah Rastafari

Mold

January 27th, 2008

Supposedly this is a really wet season for the bay area. It must be - we’ve got mold growing on what’s left outside on the porch. The porch swing is moldy (and torn after the windstorm picked it up and knocked it over to the neighbor’s porch, and the bench has this general dark tone that is growing darker by the week.

Who knew we moved to a rainforest?

Normally I wouldn’t bother writing about something like this, but it’s been raining for days, and I’m waiting for a clear window to show on the radar pattern so I can go for a run.

Good thing we didn’t move to Vancouver after all…

For the record…

January 24th, 2008

If you are how phasing out your DVD collection in favor of rereleased content on Blu-ray, you’ve got problems. I can’t quite go far enough to call someone an idiot without firsthand knowledge, but this is pretty close.

DVD CONTENT IS GOOD ENOUGH, PEOPLE. C’mon. If you really can’t watch movies without complaining of their lack of realism, grow an imagination. It’s a movie. When you get your Blu-ray movies, you’ll probably complain about how overly realistic it is - actors will look like real people, blemishes and all. (Actually, this may be one case of the porn industry _not_ driving technology forward: The rumor is that this coming content is _too_ good - to the point that you can see too much detail. They’re happy with DVDs.)

Besides, for those who really aren’t paying attention, this is likely the last physical format war, the future is software. Hell, Netflix is offering unlimited downloads _now_. By the time you get around to bankrupting yourself buying Superman III for the fourth time, the rest of us will be watching movies from our penny-per-gig hard drives, or who knows, 20 cents-per-gig solid state devices?

Propogating ignorance

January 14th, 2008

Ooof. I don’t know the details of the bill, but if the article is correct and a large portion of the bill’s payout is to people who can’t afford their houses as a result of the subprime mess, Clinton is demonstrating some bad judgement.

Bailing out idiots who didn’t read the terms of their mortgage or didn’t add up their expenses versus income over time isn’t a way to reintroduce confidence in the economy, it’s government-sponsored rewarding of ignorance. Clue - if you’re looking to buy a house, and can barely afford the payments, so when the ARM resets according to the terms you’re signing then don’t buy a house. Don’t understand the terms? Don’t sign.

If housing is too expensive where you live, rent. (We rent.) If you want to buy as an investment but it’s too expensive, move. This isn’t hard. Your job doesn’t pay enough? Then go to where it balances out better. If you can’t do that, do something else or find another employer. I know I’m oversimplifying, but all the presidential candidates are talking up how great of a country we are, and if we really are that great, we should be able to figure out this complex ’spend less money than we bring in’ idea.

Oh wait, that’s the same problem we have at the government level. Hmm. Maybe we are just screwed.

Marathoning

January 13th, 2008

I’ve signed up for the San Francisco Marathon (www.runsfm.com), providing more proof that I’m an idiot. I do believe it’s a lot more fun to promise, then figure out how to get there, rather than planning out in detail and promising what you already know how to do.

Plus, I had to pay $85 to sign up and I’m too cheap to not actually run, now that I’ve paid for it.

LASIK/PRK, part 3. (recovery)

January 6th, 2008

I shouldn’t have taken Percocet after the surgery - as it is, I really can’t say how painful it was right afterwards. By the time I got home, I was functional, but if I didn’t put effort into moving, my head dropped and I’d just fall asleep. The sunglasses were on, the windows were (mostly) closed, and once I took the dogs out, I just laid down and went to sleep.

Jen got home some time later and I got up briefly, but from what she says, I wasn’t really that coherent. Got me. I did notice some sensitivity to light, but there still wasn’t any pain. By about 6pm, the Percocet was worn off, but I was still in a hangover state. The eyes were the least of my issues, and keeping them closed wasn’t hard at all.

By 9pm, they did start to hurt. It wasn’t that bad, all told - I’ve felt far worse. But it’s made worse by the fact that nothing can be done about the pain - no touching the wound, or applying pressure. (I was told not to rub my eyes for 2 months, which is really a tough mental task for someone who stares at a screen all day and fights software.)

Jen started administering drops, which were 4 times a day - morning, noon, evening, night. One drop of an antiseptic (Vigamox) and a anti-inflammatory (Econopred+) in each eye. At this point, we did the night drops, and she also added a ‘comfort drop’ which was some kind of local anesthetic. My worry was that if it did get worse, I wouldn’t be able to open my eye properly. The drops did help, though. (There are some neat docs online at the FDA regarding Econopred+ if you search for them. Looks like some internal docuementation that wasn’t supposed to get online, complete with NDA references.)

I also took another Perc, simply because I had slept most of the day and didn’t think I’d sleep all night. It put me right out, even with the stupid eye goggles I had to wear at night for 10 days. At 3AM, right on schedule, it wore off, and Jen gave me another set of comfort drops and another Perc pill. I slept right through to morning. That was the last time I’ll ever take Percocet, unless I’m really in a lot of pain. This was also the end of the comfort drops.

At this point, my vision was still fuzzy, maybe 20/500, so I could get around, but only inside. Lights were tolerable, but computer screens were a bit much, and I spent most of the day with my eyes closed again, even with sunglasses on. Saturday consisted of a lot of laying around and sleeping, while listening to TV or music. Sunday was more of the same.

Monday I ventured outside, with a hat and sunglasses, but it was pretty painful. An attempt at a walk was aborted early, and while in the car I had my jacket pulled over my head. I was almost functioning, though.

Tuesday I went to work and did some odds and ends - it was slow, being a holiday (I only figured this out afterwards), and I could putz along, with extremely large fonts for everything. By leaving early, before the sun was really up, I avoided any bright lights.

Most of the week went by normally, with only minor disruptions and slowdowns. My eyes didn’t mind light, but seemed to want change - a half hour of looking at the screen would feel tiresome, but putting the sunglasses on helped. After a half hour of that, taking them off helped. Weird. The eyedrop regimen continued, and at this point, while really bright lights were a distraction, they weren’t painful. No dry eyes, either.

Thursday I went in for my first checkup, when the temporary contact lenses would come out. At this point, I could navigate the busses and BART without any trouble, but I wouldn’t have been comfortable driving - everything was still too blurry to do more than 25MPH. Everything looked fine, and my contacts came out.

A side effect of losing the contacts is that my vision degraded for a day or so. This must be because the wound is exposed and gets a bit roughed up, but I don’t know this for sure. The drops continued, but only Econopred+, with a ‘Tears Again’ ointment at night, which appears to have the same makeup as sty medication - it’s oily, and gives me really blurry vision.

Clear sailing so far - I still can’t see that well, but I can get around, and things are improving. The hardest thing at this point is avoiding impacts and rubbing of the eye. After years of having automatic shields up, I’m used to letting things (like dogs) hit me in the face, and it’s hard to get used to not doing that for now.

There hasn’t been an a-ha moment like with normal LASIK, where suddenly everything’s 20/20 and pain-free. It’s been fuzzy, and not the nicest recovery period, but at least it’s getting better!

LASIK/PRK, part 2. (procedure)

January 5th, 2008

OK. Now for the details of the actual LASIK/PRK procedure itself. I’m still recovering, so more details may follow later.

First - the procedure itself. I showed up at 9:15 with an expectation of being around for about 2 hours of prep/surgery/paperwork. At 10:40, I was done. The actual time spent in the chair was probably 5 minutes of active work. (And 5 minutes of the poor doctor and assistants trying to get me to suppress my blink reflex.)

When I got in, I first went through the same barrage of tests as the first day, to make sure the maps are up to date, that my prescription was correct, etc. I spent a lot of time staring at the corneal topographer - I’m sure it was to make sure the map is correct, but the number of trials made me wonder if I was doing it wrong.

A mild anesthetic was used on my eyes, and was told to wait for a bit with my eyes closed. I suspect this is to make sure the last measurements were correct (Regarding the area lased - as I understand it, halos are more common when the area operated on is smaller than the area covered by a fully dilated pupil. This is a bit of a guess, but it makes sense.)

Once I went into the operating room, a couple more minor checks were done (sorry, I don’t know the names of all the machines). Then there were the drops. Lots of them, pouring over the sides of my eyes. Pads were fit into my head covering to soak them up as they ran down the side of my face.

I think there were multiple types of drops, but I’m not sure.

Then I got laid back on the table. This is the surprising part - I was expecting that at some point I’d get a mild sedative like Vicodin. Nothing strong, but something that would allow me to respond to commands while not caring that my eye was being scraped. Nothing of the sort was given to me, and I started to realize that I wasn’t getting it. Oh well.

Once Dr. Hyver came in, the restraints were put in to hold my eye open. This sounds bad, but it really wasn’t - it was hard to suppress my blinking, but eventually, they were able to hook the restraint below both eyelids, and crank it open. (There was some crank mechanism on the side, I didn’t get to see. One eye was done at a time.

Then the chair rotated under the laser, and I could see the red blinking light I was going to have to focus on. Around it was a thin white ring of light. The laser was mounted off to the right. More drops were used.

Here’s the creepy part. First the epithelium has to be removed, so Dr. Hyver said he’d be ‘blocking the light for a few seconds’. Granted, I don’t have good vision (or didn’t) but I swear I saw what looked like a rotating dentist brush come in. Similar, but not the same.

The rotating brush comes down, and fills the field of vision for about 10 seconds or so. This happened a couple times, I suppose while the proper region was covered. Once it’s done, everything’s really blurry.

Now I was warned that the light would get distorted. There was a tool that came across the eye, I suppose to remove the epithelium mess. My vision cleared a bit, but it really did distort in the process. Almost like controlled waves of light, as the tool wiped the mess away. This was kind of neat, actually. I don’t know if I really liked the effect or just knew that the weirdest bit was over.

The laser is the next step. It’d been programmed with my map, and was ready to fire away. I was told to watch the blinking light as well as I could. Once it started, it just pulsed away (at about 2 or 3Hz) as it ablated the cornea surface. The actual time with the laser was about 30 seconds per eye.

During this process, the light seemed to move around, and I was supposed to track it. At the same time, everything was becoming progressively brighter and blurrier. I didn’t know how bad this would get, and it got a bit uncomfortable, but it eventually stopped, at about the point where I could mainly just see a white blur through most of my vision with a little bit of red.

I don’t know if the light was really moving around or if that was a trick of the eye, but it did jump a bit. Following it was a decent task, but the machine is sampling input at 4kHz, so really, if you aren’t keeping up or move to the wrong location, it’s going to know long before you can intentionally go out of tolerance. (Really, how far can you move your eye and/or head in 250 microseconds?)

Once this was done, I had to open the other eye and look at a Q-tip held about 3 inches up, while the final procedures were done. I don’t know what these were, because something blocked my vision in the other eye. I would have asked, but the other eye still had to be done, and I didn’t want to know quite yet.

After whatever that procedure was completed, a contact lens was dropped in, since i really had an open wound on my eye, and the ocular restraint was removed so I could blink.

Those poor people then had to try to get the restraint in my eye, and repeat the procedure. It was largely identical to the first run.

Once both contacts were in and I was out from under the machine, I sat up and was led to an adjoining room, to sit in a chair with my eyes closed for a bit while some last minute items were done. (Paperwork, maybe? I don’t know.) A quick inspection was done, and I passed.

I received some sunglasses, and was all ready to go. At this point, my vision was fuzzy, but better than it would have been without glasses before. I was light sensitive, but not that much. Since I was going to take the BART home, I was advised to take a Percocet pill now, so it would take effect by the time I got there.

This may or may not have been the best idea, but I made it home. I’ll cover the drugs in the next installment. I took a taxi to the BART station (it would have been a 10 minute walk), because I was supposed to keep my eyes closed for 4-6 hours.

Once at the BART, I just took a train to SF, and rode it to the Embarcadero, where I picked up the right east bay train. It was a bit hard to read the station stops and I can never hear the announcer, but it was doable. Once off BART in Oakland, I picked up the 51 bus to Alameda. It was a bit hard to read the signs, but the 51 signs are in blue, and that was easy to trigger on. I did stand there like I was drunk, and could barely open my eyes enough to watch for the bus. This was more Percocet than surgery, I think.

The 51 dropped me off close to home, and after walking a couple blocks, I got in, laid down, and passed out. More later on the recovery - drugs, sleep, and so on.