Skip to content

Posts by Duncan Hart

Is your existing DNS service reliable? Unlikely!

(This is an occasional rant about how to make things on the Internet just simply work without problem or hinderance).

Many domains fail almost every one of the reliability tests. They have two nameservers (the minimum), but often on adjacent IP addresses, say x.y.z.1 and x.y.z.2. These are certainly on the same IP network, and therefore in the same AS. They’re probably in the same rack, one sitting on top of the other. There is a large number of common failure modes that can make them both temporarily unreachable.

You can use the whois command (or any WHOIS-lookup web site) to look up the NS records by the name of your domain. It should tell you both their names and their IP addresses.

Here’s a very bad example, but very typical:

$ whois example.com
(…)
Domain servers in listed order:

A.EXAMPLE.NET 192.0.34.43
B.EXAMPLE.NET 192.0.34.44

If there are only two of them, and their IP addresses are identical except in the last number (as in the bad example, above), you have a problem.

If they’re not as close together as the example, finding out for sure whether they share an IP network takes local knowledge of the routing topology, which you probably can’t determine on your own. Ask your DNS provider for details.

Finding the AS path for the IP addresses is easier, but the output takes some expertise to interpret. http://nitrous.digex.net/ has some Looking Glass web pages that will let you look up available routing information for any IP address. (Pick a site and choose a BGP Query.) If in doubt, ask your DNS provider to describe their redundancy in terms of IP networks, physical space, and AS diversity. If they don’t even know what you mean, escalate. If they refuse to “disclose” this to you, vote with your feet.

Is DNS important for reliable email delivery?

(This is an occasional rant about how to make things on the Internet just simply work without problem or hinderance).

You bet it is!!!

Your email delivery is dependent upon the wide and reliable visibility of your MX record. In order to assure this, your zone file should be published on at least two DNS servers, or nameservers, and preferably more — four is ideal. This is so that at least one of them is reachable and working correctly, from the standpoint of someone trying to send you email.

If all of your nameservers are unreachable from a given point on the Internet (your correspondent’s originating mailhost), at a given time (when they hit “send”), your domain will appear not to exist, and mail will likely bounce immediately with:

HOST UNKNOWN

This is what we most want to avoid. If your nameservers are up, even if your mailhost is temporarily offline, mail will queue at the origin, rather than bounce, and this is much to be preferred. It gives you a while to fix your mailhost before the senders notice anything is wrong.

Your nameservers should be separated widely, in several ways, to avoid so-called common-mode failures that would otherwise affect them all. They should be:

On multiple hosts - Not on a single host that may be down.

This shouldn’t need much explanation. Computers do not run without incident for long times. Even with no errors and no equipment failures, they need maintenance and upgrades from time to time. DNS requires that every domain have at least two nameservers. Host redundancy is built into the protocol.

Separated in space - Not in the same city, let alone the same building, or the same rack.

To protect against local physical accidents: fire, flood, tripping over the power strip, meteor strike, et cetera.

On separate IP networks - Not on the same IP network or physical (Ethernet) segment.

To protect against common-mode network failures, such as routes being dropped, or border router or LAN switch equipment failures. This is the most commonly violated principle.

In separate Autonymous Systems - Not on networks sharing major routes from the Internet.

Here we get into the core of routing in the Internet. An AS is a group of IP networks run by a single organization, such as an ISP. Where ISPs’ networks connect to one another, they exchange routing information about their networks, but they summarize the exact path that a packet will take. They just advertise the so-called AS Path. It will get your packets to the right ISP, and then a more-detailed system takes over and routes the packet to the exact router and host.

If you have four nameservers, in four cities, on widely different IP networks, but all these networks terminate in the same AS (for example, data centers all run by one ISP), then there is still an important, but subtle, common failure mode. That ISP (that AS) can have a widespread failure that makes all of their networks unreachable. If at least one of your nameservers is in another AS entirely, you are protected against even this.

Should they be on separate nations/continents/tectonic plates?

All the world is not the USA. Though of course there is a point of diminishing returns.

Constant promiscuity

“Our wealth sits upon a very large device that copies promiscuously and constantly. Yet the previous round of wealth in this economy was built on selling precious copies, so the free flow of free copies tends to undermine the established order.”  The new reality. I wonder how many organisations openly acknowledge this? And how many have their head in the sand?

The iPad is unbeatable

This piece by Farhad Manjoo is so spot on….

“Imagine you run a large technology company not named Apple. Let’s say you’re Steve Ballmer, Michael Dell, Meg Whitman, Larry Page, or Intel’s Paul Otellini. How are you feeling today, a day after Apple CEO Tim Cook unveiled the new iPad? Are you discounting the device as just an incremental improvement, the same shiny tablet with a better screen and faster cellular access? Or is it possible you had trouble sleeping last night? Did you toss and turn, worrying that Apple’s new device represents a potential knockout punch, a move that will cement its place as the undisputed leader of the biggest, most disruptive new tech market since the advent of the Web browser? Maybe your last few hours have been even worse than that. Perhaps you’re now paralyzed with confusion, fearful that you might be completely boxed in by the iPad—that there seems no good way to beat it.”

It’s Not Information Overload. It’s Filter Failure.

This piece from Clay Shirky isn’t new, but it’s sure still absolutely relevant.

http://www.youtube-nocookie.com/embed/LabqeJEOQyI?rel=0

The right question

“If you’re not asking the right question, then there is no correct answer.” Merlin Mann

Unacceptable, unacceptable, unacceptable

I think Chris Chant has hit the nail on the head with his recent Institute of Government piece.

“It is unacceptable at this point in time to not know the true cost of a service and the real exit costs from those services: the costs commercially, technically and from a business de-integration standpoint. So, how do we untangle our way out of a particular product or service. I can’t tell you how many times I’ve had the discussion that says, we need to get away from that, and we can’t because of the complexity of getting out from where we are, and of all the things that are hanging on to that particular service, that we can’t disentangle ourselves from.

I think it’s completely unacceptable at this point in time to enter into contracts for longer than 12 months. I can’t see how we can sit in a world of IT, and acknowledge the arrival of the iPad in the last two years, and yet somehow imagine that we can predict what we’re going to need to be doing in two or three or five or seven or ten years time. It’s complete nonsense.”

It fills me with hope that there are people out there working to highlight that the command-and-control approach to technology and systems no longer holds up. Can the few break out of the force field of the incumbent suppliers? I really hope so.

Full audio of Chris’s piece is available here.

Doodlers, unite!

Reading

“The man who doesn’t read has no advantage over the man who cannot read.” Mark Twain.

The wrong questions

“ The most serious mistakes are not being made as a result of wrong answers. The truly dangerous thing is asking the wrong questions.” Peter Drucker

The security sky is falling (but what should I do?)

I’m increasingly becoming irked by lots of security folks who are screaming ‘the security sky is falling.’ Stopping there, with simply the frightening aspect of worrying, is like screaming “Danger!” without knowing what to do or where to run. All that happens is that the screaming causes disturbance but has not told anyone what they can do to avoid the danger.

Communicate

I spend a lot of time thinking about communication systems that have low control, and high expectations. As a result I’m rarely using e-mail these days. Instead, if you want to contact me please do it via Twitter: @duncanhart

Security utopia

The problem I’ve most often met in building secure systems is that this particular subject seems to bring out the utopian in people like no other.

This ‘should’ happen, that ‘should’ happen, its unfair or wrong or wicked that such and such is allowed to continue. Well, yes. But what are we actually going to do about it? So we need realistic problems solvers. That means a pragmatic approach, which can often offend a lot of purists. Peter Gutmann captures the essence beautifully -  I think a lot of purists would rather have PKI be useless to anyone in any practical terms than to have it made simple enough to use, but potentially “flawed”.”

The fallacy of control in all broadcast era organisations

If you think you have control then think again

Mike Bracken

Really pleased to see that Mike Bracken has been appointed Executive Director of Digital Efficiency and Reform Group. Great news. Good luck Mike.

IT Fascism

A friend mentioned that security might well be the last bastion of IT fascism. He could well be right. But surely it doesn’t have to be that way?

Security involves compromise

I was struck by how much this Bryan Lawson quote could actually be about security design decisions and tradeoffs.

“Design security almost invariably involves compromise…. Rarely can the designer security engineer simply optimise one requirement without suffering losses elsewhere…. There are no established methods for deciding just how good or bad solutions are, and still the best test of most design security controls is to wait and see how well it works in practice. Design security solutions can never be perfect and are often more easily criticised than created, and designers security engineers must accept that they will almost invariably appear wrong in some ways to some people.” —Bryan Lawson. Originally observed here.

Trust comes from knowing….

” Trust comes from knowing, not from blind faith. And to know one must understand, and to understand one must have an intimate awareness of what conditions are truly present, what people do and what they don’t, how people do what they do and don’t.” Michael Gerber, “The E-Myth Revisited

Churchill

George Bernard Shaw is meant to have sent Winston Churchill a pair of tickets to the opening night of one of his plays, saying “bring a friend… if you have one”. Churchill is meant to have replied, returning the tickets, “can’t make opening night. will make second. if you have one.”

Bruce Schneier: The security mirage

The feeling of security and the reality of security don’t always match, says computer-security expert Bruce Schneier. At TEDxPSU, he explains why we spend billions addressing news story risks, like the “security theater” now playing at your local airport, while neglecting more probable risks — and how we can break this pattern.

Actionable Problem Statements

Bertrand Russell said, “The greatest challenge to any thinker is stating the problem in a way that will allow a solution.”

Ralph Langner: Cracking Stuxnet, a 21st-century cyber weapon

A superb piece from TED.com with Ralph Langer clearly explaining the internal workings of Stuxnet, a 21st-century cyber weapon.

Security Dick Swinging – post to follow shortly

Nothing here yet. Come back soon.

Steal time, everyday

Another gem from Hugh Macleod and his Evil Plans:

Napoleon once said, “I can always regain lost territory. A single second, never.”

Extinction Management

From Hugh Macleod’s Evil Plans….

“Either get with the programme or hire a consultant in Extinction Management.”

Backwards Maxim

“Most people will assume everything is secure until provided strong evidence to the contrary. Exactly backwards from a reasonable approach.”   Anon.

Irrelevance

I’m continually and constantly amazed about the general myopia from people who have no idea that what they’re doing, or how they’re doing it is becoming irrelevant. If you think change is hard then irrelevance is even more miserable.

Reading

This is so important I’m posting it again.

“No matter how busy you may think you are, you must find time for reading, or surrender yourself to self-chosen ignorance”, Confucius.

Sender Policy Framework and DomainKeys Identified Mail

E-mail – I love to hate it. I could write so much about what is wrong, on so many levels, with e-mail. That’s another matter for another time.

But even with problematic e-mail corporations and enterprises have come to rely on e-mail to support, or even build and maintain their business processes. There’s a lot at stake here, best we get things right and keep them in good order then!

So, with so many additional and complex requirements being loaded onto what is in essence, and was only ever designed to be, a Simple Mail Transfer Protocol it’s no surprise that the system is creaking at the seams.

Thus, if you’re serious about your e-mail, and you give a damn that your message gets through, you need to make sure your e-mail has the edge and advantage. If you’re running your own e-mail service, either as a large corporation or even just as a home enthusiast you really should take a look at using Sender Policy Framework (SPF) and DomainKeys Identified Mail (DKIM). Both are orthogonal and complementary to each other and give your e-mail service the edge.

If you haven’t deployed yourself, or your service provider isn’t offering these options as integral to their service, then you really need to consider how important your communication really is, and whether you can actually afford not to have them.

(Postscript: Or you could take the nuclear option and ditch e-mail completely :-) )

Misjudging risk (and bad decisions)

I’m a huge Seth Godin fan. This is a particularly good post that especially resonates with me: Misjudging risk (and bad decisions).

Right risks

Love this quote:

Safe is good for sidewalks and swimming pools, but life requires risk if you are to get anywhere.

http://twitter.com/simonsinek/status/4545842635603968

Reading or ignorance

I really do believe this quote:

“No matter how busy you may think you are, you must find time for reading, or surrender yourself to self-chosen ignorance”, Confucius.

On sharing

I’m a huge admirer of Steven Pressfield. His book The War of Art made a particularly big impact with me. I’m now a regular reader of his blog and his latest piece (31 December 2010), On Sharing, really caught my eye. Sharing is something that has my attention at the moment. I spend a lot of time thinking about how to do it safely and securely, and also how important and transformative it can be.

With my own emphasis added in italic:

“With Steve’s projects, we’ve tried to get to the point and share information that is relevant to the individuals and outlets receiving it. We’ve avoided generic e-mail press release blasts and postings. Instead, we’ve approached individuals and outlets one by one, with tailored information that speaks to the work they’re doing. The one-by-one approach takes time, but it’s worth it. It’s also a way of showing respect—that we’ve taken the time to learn about the work and interests of others, rather than blanketing everyone with the same release.”

There’s something extra special about taking that extra time and spending the extra effort to ensure that your message, which I’m assuming you value, gets through and is heard and understood. In a World with a far-too-high noise ~ signal ratio it’s going the extra mile like this that really will pay dividends.

Adding manpower to a late (software) project makes it later.

I’m still gripped by my holiday reading – The Mythical Man-Month.

“This then is the demythologising of the man-month. The number of months a project depends upon its sequential constraints. The maximum number of men depends upon the number of independent subtasks. From these two quantities one can derive schedules using fewer men and more months. (The only risk is product obsolescence). One cannot, however, get workable schedules using more men and fewer months. More software projects have gone awry for lack of calendar time than for all other causes combined.”

Amen!

Wait or eat it raw

Spending some of the seasonal holidays rereading
the The Mythical Man-Month, Essays on
Software Engineering. From Gutless Estimating: “Observe that for
the programmer, as for the chef, the urgency of the patron may
govern the scheduled completion of the task, but it cannot govern
the actual completion. An omelette, promised in two minutes, may
appear to be progressing nicely. But when it has not set in two
minutes, the customer has two choices – wait or eat it raw.
Software customers have had the same choices.”