This has happened to me several times over the past few weeks.  As someone who has spent several years in the email marketing business, I’m always a bit curious (and irritated) and I often peek under the covers to view the list of subscribers.

Last year I received a broadcast email from CCAP, the state agency that sells the data (a SOAP interface) we use for ForeclosureAlarm.com. The email included all their other customers in the “to:” field.  It was very interesting to see a list of other companies that purchase that data.  Very interesting.

Don’t get me wrong, I don’t take this lightly.  I get a little upset when this happens.  Not this upset,

but I’m not a happy camper that my email address has been shared with hundreds of people I don’t know. Sometimes I’ll even reply back to the sender letting them know about this magical thing called a bcc.

This gets me to the google and their gmail service.  You see, gmail already does some fancy things with their service.  For example, if you say something in the email about an attachment, but you don’t actually attach a file, they tell you.  Pretty slick and a very nice feature to keep people from embarrassment.

Gmail should be checking emails that include a bunch (dozens or more?) addresses in the cc: or to: fields and then provide a friendly note to the sender, with maybe a link to a knowledge base article, that maybe the sender wants to actually put all those addresses in the bcc: field and why. It would help protect their customers from abusing their friends email address and it would make gmail better.  A win win.

There you go Google, keep track of improper use of the “to:” field by your users and it’ll help keep me from feeling like this.

Programming is more about Design than Typing

Over the years I’ve noticed a difference between great programmers and average ones.

Average programmers are often seen typing.

Great programmers are often seen staring at code.

The reason, it seems, is because great programmers are thinking bigger.  They’re thinking about dryness, code consolidation, reuse, simplicity, using the existing framework, and a lot of other things than the specific tracker task they’ve been assigned.  Average programmers focus more on simply creating the code and tests for that task.  As a percentage of their work, average programmers type more.

Thus, focusing on the typing aspect of programming is biased toward the mediocre. It is the “small lever” within software development.  The big lever, the one that effects the long term cost of software development and maintenance is design.  And design comes from thinking.

Design (and thinking) is improved by dialogue

The real power of pair programming comes about due to the dialogue between the developers.  In order to present a design idea, the developer needs to put his/her idea into words and that process alone is often very valuable.  It is very easy to have an idea about how to solve something, yet gloss over some of the details or complex issues.  By taking a vague design thought and putting it into words, the person with the idea is actually learning about his/her idea. Obviously,  this process of verbalizing a design is usually missing when working solo and often a developer will simply head out in a “direction” confident they will figure out the details as they proceed.  With pair programming, a pair usually doesn’t allow this.

This verbalizing is inherently a good thought process.  The gaps in your thinking are exposed.

Along with discussing the solution design, the developer also likely will state their assumptions about why they think something should be done.  The assumptions could be about risks, costs, testability, etc.  And again, that dialogue between developers leads to further knowledge transfer and understanding.

Here’s a quote from The Costs and Benefits of Pair Programming

We often came up with different ideas about how the design should go and the result of arguing over which one was better often led to a truly superior hybrid design.

One last note about design.  I have found that pair programming tends to keep developers near the minimum of the cost curve (written about in Line Count and Maintenance Cost) instead of the arcane and wordy areas of the graph.

To sum it up, forget about all the nitpicky stuff in pair programming, that is not the long lever. It’s about quality and maintainability of software, but that will only happen if the developers are truly pair programming and not simply sitting together at the same keyboard.  But that’s a different post.

They aren’t.

I’ve been implementing both lean and agile processes since the early 90′s.  The concepts aren’t new, but they are relatively new to the software development world.  I’ve read about and taught TPS, SMED, and many other parts of lean production as well as many of the concepts used in agile manufacturing.  Their application to software is incredibly similar.

Lean is an inward focus.

Lean is a measure of the entity separate from its customers and historically is a measurement of waste.  Reduce waste and you are more lean.  The Toyota Production System (one of the oldest lean manufacturing system) has a constant focus on the elimination of waste.

Lean is loved by many executives because it conforms to the “world as machine” view. Taylorism, and it’s descendants such as business process re-engineering, have viewed the business entity and its processes (and the larger scale systems the organization exists within) from a mindset of planning and control – again, a world view most managers seem to prefer.

Agile is an outward focus.

Agile is dirty, organic, unknowable, unplannable, which makes managers and executives uncomfortable.

To be Agile, you do need to be somewhat lean, but the leanness is FOR agility, not the goal.  The difference is important.

Agile is measured not with internal measurements, but as a measure of how well the entity responds to changes in the external environment.  Waste and leanness are not the goal and as Eli Goldrat has shown time and time again, being more lean in the wrong places can make you much, much less agile.

It’s important to understand why agility has become so important over the last several decades.  It’s due to the continual shortening of product life cycles, customer order cycles, and the increase of product competition due to information exchange.  No longer can you release a product once every few years.  Even the idea of software versions is becoming somewhat obsolete (what “version” is the google search engine?).

Which to focus on?

I spend most of my time in the web world where agility, the ability to adapt to a changing competitive landscape, is the number one competitive advantage for a software firm.

Agility is customer focused and these days the customer is king.

So I choose agility over leanness every day.  It isn’t even close.

Over the past few years I’ve been working a lot more with Ruby and the Rails framework.  I’m not sure why, but there seems to be a social norm in the community of impressing others by consolidating code to as few lines as possible (often just one) with the (mistaken) belief that the code is better.  And that sometimes makes me feel like this.

Those programmers that create some of this amazing, yet arcane code are usually some of the best programmers I’ve ever worked with.

The problem is that they’re often misguided in this pursuit of brevity.

The relationship is probably not linear and there is likely not only diminishing marginal returns (which are almost always inescapable), but there is likely a negative relationship at some point.  The reason is that although the advanced programmer can handle the complexity of the “arcane” lines of code, those that maintain code are often not the ones writing it – specifically and also in skill level.  The more arcane the code is, the more time it will take those maintaining the code to understand it.

The relationship is probably like this graph.

It’s been my experience that good programmers do not need to be reminded of writing DRY and concise code.  Quite the opposite.  As a project manager, I believe it is necessary to inform and remind those great programmers of the potential for increasing costs due to an overemphasis on the reduction of lines of code.  Strange but true.

One day I wrote down a list of 25 thoughts on risk management and testing.  Eventually I’ll weave these ideas into articles, but for now, just a <ul>

* Everyone contributes risks

* No risk exists without probability and impact

* No impact is infinite

* All impacts are measurable

* Risks drive level of design

* Risks drive test plans

* Testing directly affects design

* TDD is not 100%. The decision heuristic is risk.

* Exploratory testing approach is publicly known so customers and decision makers can know what has been tested.

* The word “requirements” promotes simplistic thinking. We develop features up until the point at which a person or team decides to deploy. Some features are never implemented. Listing as requirements makes people think in binary mode instead of order of importance to the organization.

* Measure progress based on running features tested. Compare sum of the weights completed with the total targeted feature set weight.

* Features all have a “weight” relative to other features within the same project.

* A feature without a running test is analogous to “works on my computer.”

* In all projects there is a point at which the set of claims tests are a more accurate statement of what the software does than the list of features (or requirements for you old schoolers.)

* Some features like “easy to learn” are not testable claims. Are they?

* A goal of gathering and writing features is to have them written as claims.

* The best types of claims have inputs and outputs because there is less interpretation in the process from documenting the feature to writing the code and tests.

* Tests are either named or described in a way that relates them to a feature.

* The build process generates a list of all test results for a project. The tests for a project can exist in multiple places (web app, shared component, project specific component)

* Risks have the greatest focus of the project team. Not the “requirements” document. It is a fallacy that software development is about creating a 100% complete list of requirements and then coding to that list.

* Risks are the most managed “thing.” Later in the development process, issues become the primary management tool.

* A project has a list of tests and you are able to click a link to view the actual source code of the test.

* Each test has a history of pass/fail.

* When a developer completes an issue, they add a comment in the issue message that can be run by the reactor and will send a result to the nosy list.

* A daily email is sent of verification results. Project name -> className.testName of failures with link.

For many software teams the requirements document is the king of documents.

You know the routine. The software team works with the customer (or customer proxy) to put together a “complete” list of “requirements”. Once that is done, the team writes the software implementing the requirements and voila! the project is done.

Or maybe not.

Every software project I’ve ever been on has involved scope management. This is a process of determining if a requirement is truly a requirement for right now and whether it could be delayed until part of a future version or release. This is performed not only when “requirements” are being first developed, but also during development/testing/deployment in an effort to meet a project timeline constraint.

Words Matter

In the Agile development world, “requirements” is sometimes replaced with “features”.  This is a very important change, mostly related to the meaning of words and cognitive dissonance.

Think about it.  “Requirement” is based on the word “require” – something compulsory or necessary by rule. For someone to agree to postpone a “requirement”, you are basically asking them to admit it wasn’t actually required in the first place – thus, it wasn’t a “real” requirement.  And therein lies the problem…

Asking someone to postpone a “requirement” is asking them to admit a mistake.

This points to the beauty of using the word “feature” when describing the software desired.  Features connote desires.  And it is OK for customers to desire pretty much everything.  “You want features, load me up!”  Throw features on the deck, list them like crazy.  Pile them on, as project managers, we’re not afraid of hundreds or thousands of features.  Knock yourself out – as long as they are prioritized.

Once we start developing, we can then start having conversations such as:

• Do we have enough features to deploy?
• Would you like to change the priorities of the features you’ve listed?
• Any new features?”

Those are much, much easier conversations to have and project decisions will be made much quicker without the debates about whether a “requirement” is actually a requirement.

In what appears to be some form of risk management strategy, these login forms introduce a different risk to the websites that use the default behaviors – in RoR sites, this is often either AuthLogic or RestfulAuthentication.

Both are broken in a big way.

First, the bug.  And you might not even recognize it is a bug because the behaviour is so wide spread, so ingrained, and so “sensible”.

On almost every login form, if I submit either the wrong username OR the wrong password.  I get the same error message.  Usually it’ll be something brief like the following.

Notice the very important word in this message.  OR.  Because the error message is the same if I enter the wrong user name OR the wrong password, I don’t know which piece of data I entered incorrectly.

I’m guessing virtually everyone reading this is now thinking “well yeah, if you tell them which piece is wrong, someone can keep guessing until they get it right.”

Before we get into that, what are the risks being mitigated in this process?  You don’t want someone to hack into one of your user’s accounts, right?

Right.

But what else?  How about the risk that someone, a new user, comes back to your site and has forgotten which password they are using?  Maybe you forced them to use some complex password.  Maybe not.  In any case, maybe they want to use your site, but it isn’t really that important.  As web app builders, we’re constantly thinking about lowering the barriers to join and begin using a web application, right? Why would we add a barrier here.  It is possible that a user with difficulties signing in might just leave.  Never to come back.  Never to pay you for your service.

All because you were not kind in telling him/her why the credentials they gave on login didn’t work.

So, that’s a risk.  And for a startup, this is a significant risk.  You need every new user you can get.  Especially those people that have taken the time to actually create an account.

But let’s get back to the “common sense” reason for not telling the user more about why their credentials didn’t work.

You may notice on the signin form above, that link to “Forgot“.  If you click that link and enter a user name (email address in this case), the system will almost always tell you if that user name exists.  Using the above site, it’ll look like this.

So, lemme just repeat the problem.

Login forms almost never tell you if your user name is wrong

Compared with

Forgot password forms almost always tell you if your user name is wrong

So let’s get back to that risk that is supposedly being mitigated by not telling the form submitter what piece of data is wrong.  We’re trying to keep people from guessing user names, right?  And yet we provide a form for doing exactly that!

The risk is not mitigated in the slightest by the error message.  Not one bit.

Now, I’m not going to go into how this process should be done.  For two reasons.  One, I’m out of time for writing this morning.  And Two, because I want people to disagree with me so I can understand their thought process.  It is possible I’m missing something here.  Unlikely, but possible.

I tried to reply to one of the emails they send each day. And I quickly discovered they send these using a no-reply address. Meaning I COULDN’T reply.

And that broke my brain. This is a company which is all about web 2.0 and the new communication mechanisms available. Yet they have broken the single most used communication process of the internet!!! Why would they do this? People have been sending and replying to emails for decades now. Decades. And it works.

Email replies are free feedback

Letting your customers reply to the emails you sent is like getting free feedback. I don’t know about you, but I LOVE talking with my email marketing customers. I get specific feedback on what they’re looking for, the problems they’re trying to solve, and their business situation. All of that is incredibly useful for me in creating new features and marketing my service.

I would never, ever think of breaking the reply-to of my email newsletter or the welcome emails my system sends when people sign up. NEVER. In fact, I explicitly state that they can simply reply to the confirmation email if they have any questions. Sometimes they do.

And here’s a really important metric. The people that reply to that confirmation email (when they sign up) almost ALWAYS become customers. Why? Because they ask some questions and I reply directly to each of their questions. And then BOOM, they purchase my service.

For me, having people purchase my product is pretty important. And if answering a few emails is all it takes to make a sale, then hell, email away. So why do some small businesses disable the reply-to?

You’re Not That Big

I think small companies look to big companies for how they should do things. They see Amazaon.com or some other big web based business breaking the reply-to and they think “well, this is what the big guys do, so maye I should as well.” At least that is what I’m guessing.

But you’re not that big. Even if you are HUGE, with millions of customers, you should still not break the reply-to, which I’ll explain later.

So, you’re a small web based business. Please, do NOT do what the big lazy businesses do. You should be extremely happy anytime anyone sends you an email.

Dialogue is the best way to get business and email is the easiest way to start a dialogue with a customer.

You should be encouraging replies, not breaking them with a no-reply email.

Even if you ARE big, don’t be Lazy

Any business that sends emails from a no-reply address is either stupid or lazy (or both.)

Before you start ranting about the volume of emails that they send and the cost of processing all those inbound emails, lemme just point out that I’m not stupid (although I am lazy) and I’m also a bit of a programmer.

So, how do you handle all these inbound emails if you can’t have a person or people replying to each of them?

Simple. First, you have software parse the incoming email looking for questions. You run those questions through software which attempts to find items in a knowledge base which address your question. Trouble ticket software does this already. So, if you have a trouble ticket system, you could route these incoming replies into that system. Most of them have email gateways.

Also, smart software would differentiate between emails that have short questions and emails that have multiple paragraphs. Short questions can have an auto response back with links to the knowledge base based on the software’s analysis of the question – exactly like the fancy trouble ticket systems in use today. Long replies (whatever you define as long) are probably worth reading by a person. So you can route those to a real person.

OR, if you’re really cool, you could use the Mechanical Turk to tag those emails and then have software route those emails based on the human tagging of the Turk.

See what I’m getting at?

Don’t be lazy. And don’t be stupid. Be Smart. Email replies are golden. They come from current or potential customers. Let me say that again.

Email replies come from current or potential customers.

And if customers are important to you, then treat these emails according to the value they could provide.

Removing features is very tricky and fraught with problems, even if those features are used by very few people.

New Follower Emails

The first example is when Twitter improved their new follower emails. Many articles where written about how it was a great improvement (and now in HTML!! heh). And the new follower emails are an improvement. However, as I wrote previously about the new follower notification emails, they actually removed a feature. Before the improvement, the emails included my Twitter id. Now they don’t, and that was important to me.

I’m probably not a typical Twitter user. Most people I know have only one Twitter account. Maybe I shouldn’t worry so much about staying “on topic” – that is the email marketer in me.

@replies

Recently, Twitter removed a feature used by a very small population of their user base. However, it caused quite a stir. (Twitter @replies Saga) It is interesting to note that this feature was removed because of the large processing load it placed on the Twitter database.

What is the lesson?

In both of these cases, if Twitter had not initially provided the feature, it probably would not have been a big deal.

And that leads to the lesson. If you are designing new software, keep this in mind. Adding features is easy. Too easy. Software features are really, really easy to add. Once users start using them, they are very difficult to take away.

So, do what the 37Signals guys have been saying – implement less features. You can always add them if your users request them.

Taking features away? Well, that’s a lot more difficult (and painful).

But they are broken.

I’ve read several articles talking about the “richer, HTML version” of the email. Richer? What year is it?  Formatting doesn’t make something “rich”, data does.

Data is rich, markup is lame.

The twitter got it right by adding more data about followers and updates, but HTML? Who cares.

The email still fails to include the person’s bio and their location. And what about maybe the last few tweets at the bottom of the email? They should give me more data, not less. There simply is no reason to not include the basic profile information about the new follower. It isn’t an overwhelming amount of data – include it twitter, please.

But this isn’t why the new twitter notification emails are broken.

The new emails are sent to my profile name. That’s it.

The new notification email is broken because it doesn’t include my twitter account name.

So now, the notifications say “Hi Philip Crawford”, when they used to say “Hi Philip Crawford (wiscoDude)” – or InboxFox or one of my other accounts.

Old style new follower email with almost no information

The improved new follower email with more information

I’m guessing most people have only one twitter account. I don’t. I have multiple twitter accounts because I have multiple “personalities” (I’m sure someone will snicker about that).

I have a twitter account for my email marketing company (@inboxfox), my programming persona (@philip_crawford) and my personal account (@wiscoDude). I keep them separate because I tweet about completely different things for all three. I don’t think my email marketing people want to hear about my drinking escapades or my morel hunting. And I know for a fact my personal friends don’t want to hear about the latest in email marketing reports. I’ve actually unfollowed some people in a subject domain because they tweeted too many times about their personal life – that isn’t why I follow them.

And this points out a current problem with the twitter. Multiple “personas” are not possible to implement with one account. I don’t think that is a huge problem and if I were Biz, I certainly wouldn’t be worried about it, but the email notifications don’t take this into account.

So there you have it. Twitter simply needs to include my twitter id in the email.

And follower bio, location, latest tweets….why are they skimping on text in this email?

As my friend Preston (@gl33p) pointed out, maybe this is simply opportunity for someone else to provide a service that Twitter has chosen not to provide.