Substitute for Thinking

I swear that hardly a week goes by that I don't see a story similar to this one I turned up last week via the Irish news service, RTÉ:

Seventy jobs are to be lost as internet security company Symantec confirmed plans to move operations from its Dublin plant to the Czech Republic. The move to the Czech Republic will see 40 initial job losses by the end of October, with another 30 posts expected to be cut in the first half of next year.

I really don't get this.  Everyone I talk to within the industry seems to be equally perplexed by the continued influx of companies into the local market.  Recruiters tell me that they're having a tough time filling the positions coming their way and that a lot of these companies come in with unrealistic expectations as to salary and other employment-related expenses.  Corporate IT managers are being forced to sweeten employment offers in order to compete for the candidates that are available.  And still, they keep coming.

Roman Staněk wrote a piece the other day to serve as sort of a permanent response to all the people who keep asking him about establishing an offshore development center in the Czech Republic. Essentially he says that, yes, this probably was a good deal back before the Czech crown started its massive rise and when competition for technical resources wasn't so fierce, but anyone who comes here now looking for cheap, high-quality tech staff is in for a very rude awakening.

So what's the draw here?  Does anyone have any insight into this?  Obviously CzechInvest is out selling the benefits of setting up shop in the ČR to anyone who'll listen and probably offering some nice subsidies for companies willing to take the plunge and create enough new jobs, but are those companies that are still moving here working off of old information or do they know something that the rest of us don't?

As a recovering news junkie, I’d say I know a little about what makes for great journalism. Yes, the subject matter has to be timely and interesting, and sure, the writing should grab you and help you to feel your way through the complexities of the situation, but first and foremost, the writer needs to provide you with a basic context in which the described events take place. Even the most junior reporter on a small town paper, the guy stuck covering city council meetings and supermarket grand openings, knows how to structure his article in a way that provides all the necessary information up-front in a structured, partially-digested form that ensures comprehension on the part of the reader. Sure, the writing isn’t going to earn him a Pulitzer, but he understands that his audience will get his meaning if he sticks to the most basic rules of thumb for journalists, variously known as “The Five Ws”, “The Six Ws”, or “The Five Ws (and an H)”.

Users of information systems (beta testers especially) could benefit from a bit of self-imposed structure as well when it comes to reporting bugs. I’ve gotten emails with the subject line, “SITE DOESN’T WORK!!!” and a blank body from panicked users more than a few times, but even the messages that go into more detail often fail to provide the basic information needed to begin investigating the problem. Users could help the software support guys (and by extension, themselves) out immensely if they could put themselves in the role of cub reporter for just a moment and describe the problem as if it were a news event.

Who?
From the system’s perspective, you are your computing platform. That basically means the hardware and software your application runs on.

• Hardware – computer make and model and basic specifications (CPU type, amount of RAM, etc.)
• Operating system – type and version
• Other supporting software - Java VM version for Java applications, browser type and version for browser-based applications, etc.

Your name and contact information would be helpful too because fixing a bug often requires getting clarification from the person who reported it. Also because you’re my user and, darn it, I like you.

Where?
It’s not super important for me to know that you’re sitting in a fourth floor office overlooking the river in downtown St. Paul. From the application’s perspective, geographic location is almost completely irrelevant. A network address however, in the form of a dotted quad or a domain name, is often very helpful if the application in question is networked and I need to start paging through gigabytes worth of log files.

When?
Give me the time and date that you observed the problem as precisely as possible, and please make sure to provide your time zone so that I can adjust appropriately when paging through logs.

What?
Here’s where you get to unleash the frustrated writer (and end-user) inside. Describe what behavior you’re seeing in the greatest possible detail. Leave no stone unturned.

• When does it happen?
• Is it consistent or intermittent?
• How long does it last? Does it go away by itself or do you do something to fix it yourself?
• How serious is it? Does it crash your system? Corrupt your data? Cause you to lose work? Is it only a minor annoyance?
• How pissed off are you?

Think of me as your own personal Scooby gang. Anything can be a clue, so if something seems weird to you, don’t be afraid to mention it.

Why?
I don’t mean this in the existential sense, but because you’ve answered the previous four of the five Ws, some ideas might be starting to form as to what’s causing this behavior to happen. If you’ve got a theory, don’t be afraid to shout it out. It might not be right, but it could save your support guy a whole lot of time trying solutions that don’t work.

And if it is right, your support guy owes you a beer. That’s the rule.

How?
Answering the sixth question, the infamous non-W that messes up the otherwise perfect mnemonic, is the job of your tech support guy. He should carefully consider your bug report and apply his knowledge and experience toward its resolution in the most elegant and effective way. He should let you know when the problem has been solved or, if it couldn’t be solved,

Such is the theory anyway. I’ll caution you, though, that guys who do user support for a while tend to become a little, well, bitchy. Or cranky. Maybe even assholes. This leads to all manner of problems – starting with rude dismissal of support requests to end users who can’t stand dealing with IT. It’s all pretty unnecessary. Each party feels pushed around and degraded by the other for no reason when all that’s really required is a little more mutual understanding and detail about the problem and the process.

So to review… The next time you submit a bug report, try to provide as much information as possible including:

• Who? – platform information plus your name and contact info
• Where? – network location (IP address or domain name)
• When? – exact date and time (including your time zone)
• What? – a detailed description of the behavior
• Why? – any theories you have about the cause

The 2008 European Ruby Conference is coming to Prague this weekend -- March 29-30 -- and I'll be there  The program says that we can expect talks on a wide range of subjects including programming, methodology, and infrastructure as well as an opening keynote from Matz himself.

If the e-shop gods smile upon me, I'll have a new (read: viable, holds more than 20 minutes worth of charge) battery for my laptop by the weekend, and I'll try to post an entry or two as the mood strikes me.

One of the first things that impresses a lot of developers when they first start using Rails is the almost complete lack of effort required to implement models using ActiveRecord.  Model objects are super simple to create thanks to the built-in generators, and assuming followed all the relevant naming conventions while designing your database, you’re up and running with a full-featured business object layer in no time flat.  If you’re used to another ORM framework that’s heavier in configuration – Hibernate, I’m looking in your direction! – the benefits of this way of doing things become apparent immediately.

I earned my chops developing two- and three-tiered applications that started with a database model that baked in most of the business requirements for the system as data entities, relationships and constraints.  Over that, you designed and implemented a business objects layer with all the necessary mappings to database objects (and woe unto you if there should be any major differences between the OO and relational models), and over that, you layered your user interface.  The applications often ended up looking like they were designed by a programmer – uninspired and mechanical, front-ends simply slapped onto databases – and you could expect a fun time if some damn user comes along with a requirement change that requires modification of the database.  The word for the process and the applications it produced, people, is B-R-I-T-T-L-E.  No one was happy about this state of affairs, but as developers, we passed the buck to the business stakeholders, blaming them for not being able to stick to their guns when it came to requirements.

For a long time, I didn’t really get why someone writing a serious application would ever want to use Rails migrations, and then I made a point to make a serious attempt at using them, mostly as an experiment.  I didn’t expect to find anything there that would be worth keeping up, but I thought it would be good to know something about it on the chance I ended up taking over a project that used them.  What I found, though, was a tool that has really changed the way I build applications for the better, and I use them from the very beginning of the project.  These are a few reasons why.

1. It puts more of your codebase in Ruby.  There are a few things that everyone out there calling him or herself a developer should know, and how to put together basic SQL queries and build scripts is one of them.  SQL forms the basis for our understanding of relational databases in much the same way that language forms the basis of our thought patterns, and since most business applications require a database of some kind, I’d say that makes it indispensable.  Having said all that though, it’s also a pain in the ass to write.  (Don’t believe me?  Look at all the tools and libraries developed over the years whose only purpose was to avoid having to write SQL code.)  So while it’s an important thing to know, it’s not necessary to practice it daily, and if you can find a way to eliminate the need to write and manage SQL code in your application, especially if you can replace it with code that’s as compact and expressive as that enabled by migrations, all the better.
2. You probably don’t need a graphical design tool.  I was hooked on these things for a long time to the point that I practically forgot how to form proper CREATE TABLE syntax.  I had convinced myself that these things were necessary in order to manage the complexity associated with the database model and to have a graphical model to use as a tool for communicating with business users.  The reality, though, was that I never showed a data model to a non-technical stakeholder and that the models I was working with were usually only a few dozen tables.  In fact though, I was really just trying to get around writing and maintaining SQL build scripts for each phase of development.  I don’t need to do that anymore because migrations builds the database out in parallel with the application.  Any complexity that was there before is now gone because I only need to focus on the parts of the database that I’m actively developing.
3. It promotes iterative development of application features.  Under the old-school, database-first methodology, you would first build out a new iteration of your database model in order to support a new batch of features, fix all the code that you may have just broken, and then extend the application- and presentation-layer code as needed in order to get the desired behavior.  It seemed reasonable at the time, but in practice, the volumes of changes that were being pushed through during a given development cycle made this method a lot closer to the much-derided waterfall model of development than it was to anything that could be called iterative.  The flexibility afforded by Rails migrations is that it’s just as easy to do small changes to the database as it is to do large ones, and all the database versioning is done for you automatically.  So there’s no additional cost associated with making a lot of small changes to the database instead of batching them into one confused mess.
4. Data can be loaded directly from your migrations.  There are a lot of different methods in use out there for inserting data directly into newly created database tables from a migration.  (You can find a good summary of the various possibilities with examples here.)  I use this a lot when I want to load reference data into lookup tables or when I need to insert a few rows into a table in order to bootstrap the application.  (Ex: if I need to have an Administrator account automatically inserted into a Users table in order to provide initial access to a protected application.)  If I’m only dealing with a few objects (3 or fewer) I’ll usually just create new model objects directly in the migration.  For larger collections, I’ll fall back on fixtures or data pulled from delimited text files with a few lines of custom code thrown into the mix to process the files.  Doing it this way allows business users to produce and edit the data files directly in Excel which requires very little in the way of explanation.
5. There are a lot of great plugins available to fill gaps in migrations.  You can probably guess what my initial complaint was about migrations: no foreign key support.  The good news, like most other things within Rails, is that if there’s something that’s bothering you, there’s a good chance that it’s already been bothering someone else, perhaps even enough to write a plugin to solve it.  In this case, you can install redhillonrails_core or sexy_migrations to provide foreign key support along with a number of other nice-to-have features.  The lack of FK support wasn’t a deal breaker for me, but it was nice to know that I could apply a simple fix and get the behavior I wanted.
6. Rake provides tasks that make it easy to manage your migrations.  I tend to use db:migrate:reset a lot during development to simply drop and recreate the database in its latest form.  In production environments, I usually use the more judicious db:migrate.  In both cases though, I find the whole process a helluva lot easier than when I was typing the database username and password, schema, and SQL script location into the command line.

Rails goes a long way toward closing the loop between application and database thanks to the object-relational integration provided by default in ActiveRecord.  Migrations really finishes the job by removing the separation between the two layers from the development process.  I started out looking at it just for the sake of completeness, and in the end, it really led to a massive improvement in my development process.

I just generated a mobile version of my blog using Wirenode, and I'm not exaggerating when I say that it was as easy as the WordPress 5-Minute Install: step 1 -- blah blah blah, step 2 -- blah blah blah, step 3 -- there is no step 3.  You should try it out if you have 30 seconds to spare.

My friend Tomáš Zeman has been working on building up this application for a few years now, and recently things have really started to pop for him.  He just released a new version of the application in January, and he won an award for the application and was written up in Lifehacker in February.  I've known him since I came to the Czech Republic in 2000, and I can attest to the fact that he's a smart guy who's got a great future ahead of him.  Gratuluju, Tomáši!

To those of you out there still checking in here, much appreciated. Much surprised, too. It's been a while since I've found the time to post anything which is, I suppose, a sign that things are going well within the scope of this whole new careerscape. Work and business opportunities have been trickling in, and I've had a chance to meet with many people (probably some of you) to discuss some exciting and potentially very interesting projects. So even though I don't feel all that brokenhearted that I've not posted in an unforgivably long time, I still have a little bit of a lingering unpleasant feeling from knowing that I have an orphaned blog out there somewhere.  He has my name, but still I've abandoned him, and the longer I avoid him, the more embarrassed and less motivated I am to try to reconcile with him.  Damn you, Guilt, why must you torment me so?!

Sorry, I had a little bit of a Charleton Heston moment there.

The point is that, at least intellectually, I think that blogging is a capital-G, capital-T Good Thing® and doubly so for someone who finds himself in shoes like mine.  (But not my actual shoes because, let's face it, the width is going to cause problems for you even if you can get used to the odor.)  It doesn't matter who you are, if you're trying to keep up a blog or any other habit, the inspiration dip is going to hit you too, and when it does, all the good intentions in the world aren't going to do a thing for you.  The difference between taking a couple of days away and having that thing perpetually stuck near the bottom of your to-do list is just a matter of knowing why you're doing it.  So with that in mind, I give you a few of my own reasons why I'm coming back to my blog.

1. Blogging is good for business.  You're about to say, "Well, duh!" but since I started writing this blog, I've heard from a lot of old friends, former co-workers, and past business contacts, and some of those conversations and e-mails have generated projects and leads.  I don't want you to feel that I'm underestimating your intelligence by stating the obvious here.  Rather, I'm stating the obvious so that the obvious doesn't become conspicuous by its absence and, thus, cast suspicion on the entire post.  Blogging is good for business just like giving lectures or writing a book or anything else that builds your public reputation, but if that's your complete list of reasons for writing a blog, you probably won't stick with it for very long.
2. Blogging keeps your writing skills sharp.  I've always been a decent writer, and I've found one thing to be true: good, coherent writing is indicative of a broader tendency toward clear, effective thinking. If someone is able to sit in a chair, think deeply about a given concept, decide what words and sentences are essential to describing that concept, and write it all down in a way that communicates to most other people, that's a person who can do just about any kind of knowledge work you can throw at him.  The guys from 37signals have this to say in their book Getting Real: "If you are trying to decide between a few people to fill a position, always hire the better writer. It doesn't matter if that person is a designer, programmer, marketer, salesperson, or whatever, the writing skills will pay off. Effective, concise writing and editing leads to effective, concise code, design, emails, instant messages, and more."  I couldn't have said it any better myself.
3. Blogging lets you clarify your thoughts on a subject.  Some people pay a therapist, others get a blog, I guess.  While it's pretty unlikely that I would use this as a space for discussing confusion from my childhood or fishing for amateur interpretations of the dream I had last night, I think it can be pretty helpful as a means -- first, through the act of writing, and then, through feedback and revision -- for clarifying what I think and feel about certain subjects (within the scope of this blog's subject matter).
4. Blogging gives you a voice. When Sun announced that they were buying MySQL a couple of months back, I wrote a little bit about it a couple of days after the fact.  I remember feeling good writing that entry because here was this far-away, far-out deal that I had nothing to do with, but by virtue of having an internet connection and an opinion, I could say something about it and join the army of confused people who care about one or both companies, people who rely on their products to make a living, people who couldn't stand to see another screw-up like the one they really hoped they weren't just witnessing.  Moral to the story: you don't need to be an A-lister to have your say.
5. Blogging allows you to give away useful knowledge.  Internet culture is very much a gifting culture.  People open their free web browsers and expect to be able to access services without payment or restriction: news, reference information, applications, and so on and so on.  The open source community is even more about giving away your best work in exchange for fame and status, at least within that small segment of the population who cares. I always got the biggest thrill out of programming from knowing that someone out in the world somewhere might be using some code that I'd written.  They probably didn't even know my name, but they were getting some measure of value from the work I'd produced.  A blog is just another vehicle for that same impulse except that it's not as tightly constrained by syntax and structure as a programming language would be.

When I hear things like Business Open Source Use Up 26% in One Year, I get a little interested.  Excited even.  I like hearing good news about good things, like most people.  But it doesn't override the skeptic in me that forces me to ask, "Says who?"

Follow me if you can.  I followed the link from Slashdot to Computer Business Review -- a publication I'd never heard of before but which I found linked from a trusted news source, so it must be all right, right?  Then on to the main page of the company responsible for the original survey and a short jump to their press page where there was a release that summarized the survey results.  (If you get this far and wonder why the press release looks so familiar, it's because the cub reporter working for CBR basically reworded it and published it as an article.)  There was, however, no information about the methodology used, number of surveys sent out, response rate, respondent demographics, or anything else that would give me a little insight into the meaning of it all.  Just think, before the invention of hypertext I could have just read the blurb on Slashdot, taken it as gospel, and gone to bed like people do.

I also found out that:

• The 26% increase they report is on the basis of the number of open source packages used by companies responding to the survey over similar figures collected in 2006.  As far as I'm concerned, this tells me about as much about real OSS penetration into major companies as the nonsense notifications and invitations I have waiting for me on Facebook.
• The most common license in use was found to be the Apache license which was used in 62% of the cases.  GPL was way down the list with 27% coverage.  This shouldn't come as a surprise to anyone reading a little further because...
• The list of the top open source packages was slanted heavily toward open source libraries and development tools (with a particular bias toward Java-based products) with Hibernate and Struts claiming the two top spots and somewhere in the vicinity of 80% of the top 25 most popular packages being exclusively or predominantly used for Java application development and deployment.

It's this last bit that leaves me feeling a little used.  The rankings are sufficiently screwy -- i.e. Eclipse beating Apache, Jasper Reports making the list while Linux does not, etc. -- that it really makes someone who knows enough about IT and has two brain cells to rub together want to ask whether or not anything useful can be inferred based on a shallow survey of a very limited subset of the technology sector where the results lack any kind of transparency.

Is open source on the rise in the enterprise?  Possibly.  Seems like it.  I can't say for sure based on what I see here, and the echo chamber that is news on the Internet sure isn't helping.

OK, so I've spent the past few days trying to make some sense out of Sun's recent acquisition of MySQL. Where did this come from? What's in it for the involved parties? Is it a good thing? Is it an unmitigated disaster? I'd been holding off writing anything about it until I felt I could answer a few basic questions about the motivations and the implications, but somehow I don't feel like a clear understanding is forthcoming. Fortunately, the lack of plausible theories and coherent ideas isn't prohibitive when it comes to spouting off in the blogosphere.

It's not so hard to understand why this makes sense for MySQL. They've apparently been shopping the company around for some time now (even though all the talk last year was saying IPO), and Sun gave them a lot of good reasons -- a billion of them, to be precise -- to sign on the line which is dotted. They're looking to gain market share in the enterprise with more big installations and, more importantly, by selling more support contracts and tools as a complement to their DBMS. Any potential buyer would need to have an existing presence in the corporate domain with strong global support and sales (especially sales), and if we exclude companies that already have their own competing DB technology, we're really down to just a handful of companies, and Sun just might be the most viable candidate out of all of them.

The incentive to do the deal from the Sun side isn't quite so clear. If we set aside for the moment the possibility that they might be pondering turning MySQL into a commercial product (potentially very profitable, but certainly not consistent with Sun's past behavior and positions on open source) and the additional revenues that would come from selling MySQL support services (the $1 billion purchase price dwarfs the $50 million MySQL took in for 2006), what are you left with? There are only a couple of potential reasons that seem to make any sense here.

1. They could be trying to gain mind share within the open source infrastructure software domain by controlling the 'M' in LAMP. MySQL is already a big favorite of developers working on small applications and in startups, but so far, it hasn't translated into big money for MySQL AB, so why should we expect that it would now that Sun is in the driver's seat?
2. They could be making a play for a bigger chunk of the enterprise database market currently dominated by Oracle, IBM and Microsoft. While they offer some of the same bells and whistles as some of the larger commercial vendors and the Web 2.0 wave has given them some large reference implementations to point to as examples of MySQL in high-volume environments (including YouTube, Flickr, and Second Life among others), CIOs still view most open source largely as a risky, experimental venture. (As the saying goes, "No one ever got fired for buying IBM.") Having a company like Sun pimping for them take them some of the way toward improving MySQL's image in corporate IT departments, but it's anyone's guess as to how seriously they'll be taken.
3. They could be looking at using MySQL as a way of selling other hardware and software products for a profit. Sun produces a diverse line of products, and even though they and MySQL have reaffirmed a commitment to deliver a product that works great on all platforms, there will be a natural temptation to leverage the "close working relationship" between the database development community and Sun's other product development teams to provide a boost.

In the end, it will take years before anyone really knows what to make of this and whether or not it was just simply way, way too much to spend. ($1 billion divided by $50 million per year = 20 years.) It's obvious that the purchase price indicates some speculation about the direction of the market in the coming years on the part of Jonathan Schwartz and Sun's management, all his rhetoric about sharing and supporting notwithstanding, because you've got to sell a hell of a lot of support contracts to pull in that kind of money. Ultimately though, I think that interested observers will find themselves wondering about Sun's Achilles heel since the dot-com days: its ability to execute when presented with a good opportunity. Having just set a $1 billion bet on the table, Sun's investors are going to be looking for some reassurances to that effect very soon.

I found out this morning that Rails really likes databases. It might even have some co-dependency issues.

I was trying to test out a few of the new features of Rails 2.0.2 in isolation - i.e. nothing that required a database in order to function - and I found that Rails was spitting up on me every time because I hadn't configured a database. I tried commenting out all DB connections in the database.yml config file, but that just made it feel like I was making fun of it.

The problem, as it happens, is that Rails loads ActiveRecord by default, whether I've defined any subclasses or not, and part of the module initialization is the creation of a new shared database connection for the application, whether it needs it or not. The simplest way to solve this is to keep Rails from loading the ActiveRecord framework by slipping the following line into one of your environment configuration files - environment.rb or the appropriate initializer for the environment you're working in:

config.frameworks -= [ :active_record ]

Rails is really marvelous at this sort of thing. I guess you could call it opinionated software that's reasonable enough to listen to what you as a programmer have to say.

I'm remembering a conversation I had a with a few of my co-workers during my last few days in the bank.  We began discussing my plans for the future, but at some point, the conversation changed to internet banking applications and why in God's name every bank feels the need to build and maintain its own solution from the ground up.  A couple of different theories were tossed about -- highly specific requirements, the need to differentiate with unique functions -- but in the end, it seemed like no one could think of a good reason, or at least not one that would justify the enormous financial costs and manpower requirements for developing and supporting such a beast.  Most banks offer the same set of functions, so there's very little differentiation at that level, and the requirements for most direct channel financial systems are driven by the same forces in most cases: savings through customer self-service, security requirements, regulatory restrictions, etc.  Hmmm, maybe banks just really like spending money?

I've been thinking about that conversation quite a bit over the holidays, and it seems to me that the big takeaway has less to do with some inherent truth about financial services providers but rather about businesses in general.  On one hand, business stakeholders want to believe that the ideas they're proposing during the conceptualization phase are something new under the sun, something that really blows the competition away.  In many cases, IT departments reinforce or support these ideas, either by not being outward-looking enoug or having a serious case of not-built-here syndrome.  Or maybe just by not caring enough to bring it up.  In the end, the result is often the same regardless of which forces are at work: more coding, more testing, more support, and more cost just to have another been-there-done-that site.

As DHH likes to remind us, "Your application is not a unique snowflake."  You're not Amazon or Google or eBay, and chances are that you are not working on a problem that hasn't already been solved, at least partially, by someone else.  (Unless of course you are Amazon or Google or eBay, in which case thanks for reading my blog and please come again.)  We're already well past the point where having access to a wide range of standardized, high quality, and in many cases, free infrastructure software is interesting to us, and so the next step would seem to be doing the same with more complex systems.  Blogs have already largely succeeded in turning web publishing into a commodity, and only a complete newbie would consider designing and developing a content management system from scratch as his Plan A with some of the great open source packages available today.  But what about other types of systems that are still developed as custom software far too often: e-shops, auctions, e-newsletter publishing, adult websites, and so on?  Why haven't we stopped building these over and over?  Do you really think that you've figured out some new wrinkle for the online shopping cart that hasn't been tried already?

The online banking example remains, in my mind, one of the best as it's practically a "perfect storm" of conditions under which this kind of standardization would produce maximum benefits.  There are a large number of unique implementations which, with the exception of underlying infrastructure, share basically nothing and deliver almost all of the same functions and have many or most of the same non-functional requirements - i.e. high level of security, regulatory requirements, etc.  And applications like this are expensive - to develop, to test, and to maintain.

In the coming weeks, I'm going to be giving some additional time and thought to this concept and trying to come up with a few more posts on topics related to this one.  It's probably a little ambitious to think that one could solve any of these problems once and for all, but the productivity savings for even a few instances of a specific type of application could be tremendous.