Strengthening substitution ciphers

Fountain in Gatineau

The biggest problem with substitution ciphers (those that replace each letter with a particular other letter or symbol) is that they are vulnerable to frequency analysis. In any language, some letters are more common than others. By matching up the most common symbols with what you know the most common letters are, you can begin deciphering the message. Likewise, you can use rules like ‘a rare letter than almost always appears to the left of one specific more common letter is probably a Q.’ What is needed to strengthen such ciphers is a language in which words have no such ‘personality.’ Here is how to do it:

First, take all the short words (less than three letters) and assign them a random three digit code. Lengthening very short words further strengthens this approach because short words are the most vulnerable to frequency analysis; a single letter sitting with spaces on either side is probably ‘a’ or ‘i.’ Using three digit groups and 26 letters, you can assign 17,576 words. Now, take as many words from the whole language as you want to be able to use. For the sake of completeness, let’s use the entire Oxford English Dictionary. The 456,976 possible four letter groups more than suffice to cover every word in it, leaving some space for technical terms that we may want to encrypt but which might not be included. If we need even more possibilities, there are 11,881,376 five letter combinations.

This approach is cryptographically valuable for a number of reasons. Since the codes representing words have a random collection of letters, the letter frequency in a ‘translated’ message is also random. You no longer need to worry that some English letters are more common than others. Just as important, there are none of the ‘Q’ type rules by which to later attack the substitution cipher. The dictionary of equivalencies would not need to be secret; indeed, it should be widely available. Having the dictionary does not make encrypted messages more vulnerable, since they will have passed through a substitution cipher before being distributed and are fundamentally more robust to the cryptoanalysis of substitution ciphers than a message enciphered from standard English would be.

In the era of modern algorithms like AES, I doubt there is any need for the above system. Still, I wonder if there are any historical examples of this approach being used. If you have a computer to do the code-for-word and word-for-code substitutions, it would be quite a low effort mechanism to increase security.

Botnets

The rise of the botnet is an interesting feature of contemporary computing. Essentially, it is a network of compromised computers belonging to individuals and businesses, now in control of some other individual or group without the knowledge or permission of the former group. These networks are used to spread spam, defraud people, and otherwise exploit the internet system.

A combination of factors have contributed to the present situation. The first is how virtually all computers are now networked. Using a laptop on a plane is a disconcerting experience, because you just expect to be able to check the BBC headlines or access some notes you put online. The second is the relative insecurity of operating systems. Some seem to be more secure than others, namely Linux and Mac OS X, but that may be more because fewer people use them than because they are fundamentally more secure. In a population of 95% sheep, sheep diseases will spread a lot faster than diseases that affect the goats who are the other 5%. The last important factor is the degree to which both individuals and businesses are relatively unconcerned (and not particularly liable) when it comes to what their hijacked computers might be up to.

Botnets potentially affect international peace and security, as well. Witness the recent cyberattacks unleashed against Estonia. While some evidence suggests they were undertaken by the Russian government, it is very hard to know with certainty. The difficulty of defending against such attacks also reveals certain worrisome problems with the present internet architecture.

The FBI is apparently on the case now, though the task will be difficult, given the economics of information security.

A fax is not more secure than email

The way complex organizations assess technology and security is often very silly:

A: “Here is the signed document, as a PDF file that I scanned and emailed.”

B: “No good. We need a hard copy.”

A: “Well, I can mail one to you within about a week.”

B: “That’s far too long. Why don’t you just fax it?”

A boots laptop

A opens PDF file

A clicks ‘print,’ plugs laptop into telephone, sends the fax.

Result: a lower quality version of precisely the same thing is transferred, at greater expense.

Document metadata

It remains somewhat amazing to me that governments and major international institutions so frequently forget what it means to distribute documents in Word format. In particular, people are surprisingly ignorant of how Word tracks changes: making documents into a palimpsest of revisions, not all of which you want the outside world to see. You don’t want the comment about how pointless one of the ‘key items’ in your ‘corporate vision’ is making it into the file that gets passed to the New York Times. Even the early copy of the Summary for Policymakers of the 4th Assessment Report of the IPCC that I have includes a few notes about edits that still need to be done.

Hopefully, closed standards like Word documents will fall by the wayside during the next decade or so. It is insane to be distributing so much information in a proprietary format for no good reason (just one more manifestation of monopolistic dominance). Hopefully, whichever open document format eventually comes to be standard will have better means for assessing and controlling what information you are inadvertantly embedding in your press releases, reports, spreadsheets, etc. Until then, lax security is likely to keep offering some interesting glances into the drafting processes of such publicized documents.

PS. One other thing to remember is that the standard jpg images produced by Adobe Photoshop include thumbnail files that are not edited when you change the image. As such, a face blurred out of the large version may still be recognizable in the embedded thumbnail version. The same goes for areas that may have been cropped from the image entirely. I am sure Cat Schwartz isn’t the only person who has suffered public embarassment because of this. No doubt, many other pieces of software include such counter intuitive and potentially problematic behaviours.

A show of force in the Gulf

No matter how much one tries to focus on the non-security bits of international relations, anyone who reads the news and is concerned about the world will get exposed to it pretty regularly. Yesterday, for instance, nine American warships carrying 17,000 military personnel were sent into the Persian Gulf. Some speculate that this was intended as a corollary to an announcement from the International Atomic Energy Agency (IAEA) about Iran’s ongoing nuclear program. The strike group included two Nimitz carrier battle groups and 2,100 marines in landing ships. The ongoing war games will apparently “culminate in an amphibious landing exercise in Kuwait, just a few miles from Iran.”

According to the IAEA, Iran has about 1,300 centrifuges online at Natanz, with another 600 likely to become available over the summer. Having 3,000 operational centrifuges would produce enough weapons-grade uranium for one bomb per year.

The question of how to deal with challenges to the existing non-proliferation regime is an acute one. More and more states will gain the technical capacity to make bombs in the next few decades. Many will be in dangerous parts of the world, with hostile neighbours who can be plausibly expected to be building bombs of their own. Furthermore, the inability of the current regime to prevent the North Korean test raises the question of how much influence the international community really has, especially when some states are willing to become pariahs.

Millennium Development Goal 7

Church Walk sign

Prompted by my international law and developing world revision, I had another look at the eight Millennium Development Goals which were adopted by the 192 UN member states in 2000, and which are meant to be achieved by 2015. All eight are quite ambitious and represent worthy ambitions and intentions.

Some of the goals give themselves over easily to quantitative evaluation. For instance, reducing the maternal mortality ratio by three-quarters. While there are the ever-present concerns about data quality and the danger of people fudging their numbers, at least there is an empirically verifiable objective being targeted.

The environmental category (MDG7) has the general heading “Ensure environmental sustainability” and among the most vague provisions in the whole list:

  1. Integrate the principles of sustainable development into country policies and programmes; reverse loss of environmental resources.
  2. Reduce by half the proportion of people without sustainable access to safe drinking water.
  3. Achieve significant improvement in lives of at least 100 million slum dwellers, by 2020.

To begin with, ‘sustainable development’ is not as objective a concept as it is sometimes considered. If it requires a society that could continue to operate in its present form indefinitely, then no society that exists today meets the standard. Of course, the term ‘development’ contradicts the idea of stasis. So too does the inclusion of the term in the MDGs generally, since all of them would require large-scale changes in both domestic and foreign policies.

When it comes to sheer vagueness, “reverse loss of environmental resources” must take the cake. What are ‘environmental resources?’ And what would ‘reversing their loss’ involve? With a few exceptions, such as the breakdown and slow recovery of stratospheric ozone, it is not terribly clear what this could mean. Even in cases where the general thrust of the idea seems applicable, such as reforestation or the protection of coral reefs from damaging fishing practices and increasingly acidic oceans, it doesn’t provide much in the way of guidance, or much of a standard for achievement.

Access to water

The second goal, about access to water, is much more in keeping with the qualitative targets that the MDGs generally seek to establish. A map of the world showing who has poor access to water and another showing the incidence of deaths from cholera demonstrates just how unequal quality and availability of water around the world is. All the technology required to provide safe drinking water to everyone exists. The degree to which the present situation is the result of a lack of will makes it a very appropriate target for a high-profile initiative like the MDGs.

While I have never believed that water is a likely cause for large-scale wars (countries that can afford to fight large-scale wars can afford desalination plants, which are expensive but cheaper than wars), there is every reason to believe that water will become a more acute problem in coming decades. One minor example is how a sea level rise of about 100cm could essentially eliminate Malta’s major sources of fresh water. Expect bigger problems in places like India or Bangladesh.

The Economist printed a good Survey on Water back in 2003. Accessing it requires a subscription.

Slum dwellers

Slums were mentioned here quite recently. Improving the lives of 100 million slum dwellers is certainly a worthy aim. As many as 1.2 million people may live in just the Kibera slum in Nairobi. In sub-Saharran Africa, where more than 70% of the urban population already lives in slums, the rate is growing at 4.53% per year. Improving their lives probably requires two sets of approaches. One is based around providing basic needs, including water, health care, sanitation, lighting, security, and education. The other is based around reforming legal systems. Providing secure title to land, for instance, would likely reduce opportunities for bribery, provide access to credit, and generally reduce the level of insecurity in people’s lives. Actually implementing either set of approaches is an awfully tricky proposition, not least because of entrenched interests that value slums as a source of bribes from those who live there as well as a source of cheap labour for the city in which they are embedded. That being said, there are potentially huge improvements in human welfare to be achieved from success in this area.

All told, there seem to be a lot of reasons to be hopeful about the MDGs. They demonstrate, at least, that there is universal awareness within the international system about some of the most pressing problems of the present day. There is likewise at least some energy and initiative being committed to their resolution. The extent to which such efforts are successful will probably have a big impact on the kind of world in which we find ourselves in fifty years time: one in which most of humanity has reached a situation in which their basic needs are met and their basic rights are respected, or one that may be even more unequal and conflict-prone than the situation at present.

Piracy today

Paris graffiti

Today’s Strategic Studies Group meeting was unusually interesting. Lieutenant Commander Nigel JF Dawson was speaking about contemporary piracy and gave me specific permission to discuss it here a little. Basically, there are two hotspots of piracy in the world today: off the coast of Somalia and in the Malacca Strait. The latter waterway carries about 60% of world trade, including all the oil used by Japan and China.

Apparently, there are two major types of piracy happening in Southeast Asia. The first is simple enough: unsophisticated robbery of ships by individuals with few weapons and little organization. The second is much more dramatic: the wholesale capture of ships. Organized gangs steal whole oil tankers, repaint them, produce fake documentation for them, sell the cargo, and then sell or scrap the ships themselves. In the Malacca region, the unsophisticated kind of piracy is the norm south of the third parallel, while the region to the north involves mostly the larger scale sort. The character of piracy off the African coast was less thoroughly discussed. I have heard of an incident where the Tamil Tigers stole a ship containing a consignment of munitions for the Sri Lankan government.

A Piracy Reporting Centre in Malaysia apparently keeps track of all this, though only about one in four incidents are actually reported. I suppose it would make the clients of shipping companies nervous to learn that their cargo faces such perils.

It seems like the easiest way to target the problem would be to deal with the on-shore networks that support the trade. In particular, there must be ways to combat the wholesale expropriation and re-titling of ships. A global registry seems as though it would have a decent chance of being useful, at least when it comes to trade in huge oil and gas tankers.

The Lives of Others

The Lives of Others (Leben der Anderen, Das) is a potent and pertinent film: a reminder of recent history that speaks to ongoing questions about surveillance, as well as the human and inhuman aspects of state security organizations. The film is especially impressive because of the subtlety with which the topic is approached, and the space for contemplation it affords to the viewer.

The cinematography of the film is elegant to the extent that one is in danger of missing subtitles on account of preferring to keep one’s eyes where the film-makers wanted them. The only minor lapse in good judgment is in a few scenes where the use of very wide-angle lenses produces an unwelcome and disconcerting effect. The set designs, costumes, and performs are all extremely well chosen, really managing to convey a certain vision of life under the GDR.

The film struck me as a kind of inversion of Good Morning, Night (Buongiorno, notte) which I saw back in November of last year. One explores the moral dilemma of a member of Stasi, the infamous East German secret police, while the other is about a member of the Red Brigades, an Italian terrorist movement in the 1970s. In a way, both films are comments on how people can and do deal with the structures in which they find themselves. In particular, how exposure to the humanity and vulnerability of others affects one’s pre-existing convictions.

People in Oxford may find it useful to know that it is playing at the Phoenix Cinema on Walton Street until Wednesday May 9th.