Cryptography is full of patterns. Patterns are evident in code-making, code-breaking, and even the history of cryptography itself. Code makers must use some sort of pattern when encrypting any message. Maybe the pattern is as simple as one in a rail fence transposition cipher or as complicated as one in the cipher in the Babington plot. Each encryption will always have some key (or pattern) that it must follow, or else the ciphered message will be as useless to the recipient as it would be to an anonymous third party trying to decipher the message.
Code breakers use similar patterns in order to decipher messages. For Caesar shift ciphers or even mono-alphabetic substitution ciphers, the pattern always begins with frequency analysis. From frequency analysis, all code breakers, whether amateurs or professionals, will attempt to form recognizable words or use clues such as double letter sequences in order to crack the code. Once a few common words, such as “and, the, of,” have been deciphered, the rest of the code can be deciphered with relative ease. Once parts of the code have been deciphered, a pattern tends to emerge in the key used to encipher the text. A key word or phrase may be used, or a pattern such as a shift may be used for the key. This simple pattern works with ciphers as complex as the one used in the Babington plot, as evidenced by the work of Philip Marnix.
The most intriguing pattern is the historical battle between cryptographers and crypt-analysts. Transposition ciphers, which came first, are useless because the text is just as hard to encipher by a crypt-analyst as it is for the intended receiver. Shift ciphers and mono-alphabetic substitutions were very secure until the rise of the Arab caliphate. Since they had so much time and knowledge, new advancements were made in frequency analysis in order to break these ciphers. Soon, nulls were added, and words would be spelled incorrectly in order to throw off potential crypt-analysts. This was largely unsuccessful because the most renowned crypt-analysts of the time could still crack the code with enough words and trying hard enough with frequency analysis. This historical pattern raises a few interesting questions. Will this historical battle ever end? If so, who will emerge the winner? And more importantly, how will important historical events shape this continuous conflict between cryptographer and crypt-analyst?
After the class discussion, I began thinking about the concept that cryptography and cryptanalysis is dependent on exceptional resources. Upon first reading the chapter, I had considered the cryptanalysis that was done at the time as super basic due to the lack of technology such as computers. However, after further reflection upon the concept of resources during the discussion, I now believe that the people of means did have computing power at their disposal. The people that worked under them were their computers. It is true that a computer can easily brute force a simple dictionary password, but that does not mean humans cannot brute force as well. The time differences may be very different but in the end both processes will yield the password.
Every day our computers get more powerful and capable. It is hard to believe, but probably true, that future generations will be discussing the same topic and thinking about our technology in the same way we view that of previous generations. The discussion made me realize that maybe in a few years our level of cryptanalysis will be considered primitive. Cryptanalysis has evolved so rapidly over the past few years, due in part to the advent of the Internet, and it is hard to imagine what advances are bound to occur.
During the class discussion of Singh Chapter 1, we talked about how cryptography and cryptanalysis has developed as societies have advanced. Singh states that “Cryptanalysis could not be invented until a society had reached a sufficiently sophisticated level of scholarship in several disciplines, including mathematics, statistics, and linguistics.” Looking back, many of the ciphers once used to encrypt sensitive political messages now seem dangerously insecure. However, at the time they were unbreakable, or nearly so.
An example of this change in thinking can be seen in the way that many children pass secret messages to their friends. Although they may not know the technical name for it, they are able to understand and use a shift cipher—the sort of cipher once used by rulers and generals—even at a young age. Once they get a bit older, they are able to crack one of these ciphers fairly easily too, as our class saw from the first cryptanalysis worksheet. The reason for this is not that kids today are somehow all being born smarter. The difference is that the “scholarship” once available only to the elite has become much more ingrained in our culture. This seems obvious once you mention it, but in fact it can sometimes be overlooked, and it’s something that should be kept in mind when studying cryptography’s history.
Q3:On page 15 of The Code Book, author Simon Singh writes, "Cryptanalysis could not be invented until a civilization had reached a sufficiently sophisticated level of scholarship in several disciplines, including mathematics, statistics, and linguistics." If such a level of scholarship was required for the development of the frequency analysis approach to solving substitution ciphers, what do you make of the fact that amateur cryptanalysts today often use that approach "on their own," so to speak, without being trained in it?
I think 1)the “sophisticated level” means different things in different time. The “sophisticated level” of scholarship in several disciplines at that time (about A.D. 750) might just be like the level of high-school education today since science develops fast and becomes complicated. This means we might have already had some basic and necessary knowledge in fields like mathematics, statistics and linguistics. So we, compared with people in the past, own the so-called “sophisticated scholarship”
2)Even we never receive any training about cryptanalysis, I believe most of us have gotten in touch with cryptanalysis before, maybe in a movie, in a science-fiction or a detective fiction. So in this way we have, in some way, received some background knowledge of cryptanalysis and have been able to use some basic method of cryptanalysis like frequency analysis, finding one letter words and so on.
3) Of course, the previous two reasons don’t mean that the code in the past is a piece of cake to us. In fact, the two codes we try to break in the class are just two easy uses of cipher(replace letter), a sort of substitution. And if Mr Derek ask us to break some complicated code, it must be a hard work.
I had a general idea about what cryptography and code breaking was. Learning the history of the subject in the first chapter was interesting. When I was in grade school, my friends and I used to wrap pieces of paper around pencils to write small messages; similar to how the Spartans used the scytale. The idea of using the frequency to determine the letters in cipher seems logical, however tedious to do, and it appears that it would only work with the most basic of ciphers. It is fascinating that cryptanalysis can be used when the 26 symbols are not even letters of the english alphabet. The most interesting piece of the chapter was the quote on page 41, "The cipher of Mary Queen of Scots clearly demonstrates that a weak encryption can be worse than no encryption at all." It shed some light on the actual content of the cipher rather then the means of encrypting and decrypting it. Combining the cipher with discretion would be the safest means of communication. It amazed me that some complicated ciphers have been cracked throughout history without the technology we have today. As with most branches of science, more data is better. The more a certain cipher is used, the more likely it is to be cracked because certain patterns begin to emerge. That is why cryptography is a constantly evolving field: adaptation is necessary for survival.
I meant to post this on Friday, but just in case it's still useful...
Here are a few questions about Singh Chapter 1 you might consider. I'm not expecting you to answer these questions (on the blog or in writing), I'm just providing them as a resource.
- On page 41, Singh writes, “The cipher of Mary Queen of Scots clearly demonstrates that a weak encryption can be worse than no encryption at all.” What does Singh mean by this and what does it imply for those who would attempt to keep their communications secret through cryptography?
- On page 15 of The Code Book, author Simon Singh writes, "Cryptanalysis could not be invented until a civilization had reached a sufficiently sophisticated level of scholarship in several disciplines, including mathematics, statistics, and linguistics." If such a level of scholarship was required for the development of the frequency analysis approach to solving substitution ciphers, what do you make of the fact that amateur cryptanalysts today often use that approach "on their own," so to speak, without being trained in it?
- Most of the examples of cryptography in Chapter 1 were associated with well-resourced people—monarchs, military leaders, etc. Is that because those are the only examples that have survived or is that because cryptography and cryptography development is dependent on exceptional resources? If the latter, do you think that has changed over time? What implications does that have for today’s uses of cryptography?
- Given that Singh was presumably trying to write an interesting and engaging book, why do you think he chose these examples for Chapter 1 instead of other potential examples of classical cryptography?
I hope you found today's cryptanalysis activity interesting. As I mentioned in class, you'll get the chance to work on a new ciphertext from start to finish in your first problem set, which I'll post next week.
Here's the worksheet from today's class with all four ciphertexts. The plaintext for the first one (the shift cipher) comes from this news article on Chelsea Clinton's wedding a few years ago. We'll talk about poem codes in a future class.
The plaintext for the second one (the atbash cipher) comes from WIRED writer Mat Honan's story about how his entire digital life was hacked. I'll say a few words about this on Monday, but if you have the time to read the entire article, do so. It's pretty incredible.
Remember, for Monday you should read the first chapter in Singh and create your blog account. Look for those account creation emails soon. And go ahead and finish decrypting the second ciphertext on the worksheet, too.
For your first bookmarking assignment, you're invited to bookmark anything you like that's related to cryptography. Look for a news article or resource on cryptography that's interesting to you. If you're not sure where to go with this, look for inspiration in Singh Chapter 1. Give your bookmark at least two useful tags.
Your bookmark is due by 8:00 a.m. on Friday, August 29th. We'll take a little time in class to share your finds. If you have any questions about using Diigo, don't hesitate to ask.
Image: "Interesting Pin," by me, Flickr (CC)
For your first blog assignment, write a post between 200 and 400 words in which you reflect on how your perspective changed about a topic or question discussed in Singh Chapter 1, after we discussed the chapter in class. You might share something you understand more deeply after the class discussion, or perhaps how your changed your mind about a question raised in the chapter, or perhaps a key takeaway from the class discussion that you want to remember, or even something that's more confusing to you now than it was when reading Singh.
Please give your post a descriptive title, and use the "Student Posts" category for your post. Also, give your post at least three tags, where each tag is a word or very short phrase (no more than three words) that describe the post's content. You're encouraged to use tags already in the system if they apply to your post.
Your post is due by 8:00 a.m. on Wednesday, August 27th. If you have any questions about sharing your first post here on the blog, don't hesitate to ask. Here are some basic instructions for posting to WordPress that you might find useful. Also, via xkcd, here's the secret to using any kind of computer technology.
Image: "Ghost Writer," by me, Flickr (CC)
I'm excited to teach "Cryptography: The History and Mathematics of Codes and Ciphers" again this fall. It's my favorite course to teach, and I hope you find it interesting, too.
Here's a copy of the syllabus. Please read this before class on Friday, when we'll talk about various aspects of the course and I'll take your questions on the syllabus.
I thought I would share a couple of links related to Edward Snowden and the NSA. During class, I shared this graphic from ProPublica summarizing the various NSA programs that we've learned about from Snowden's link. I also recommend listening to this interview with one of the reporters who put together the chart, which aired on ProPublica's podcast.
At the end of class, I showed you an Excel file that's helpful in the cryptanalysis of ciphertext generated by substitution ciphers. Here's a copy of that Excel file in case you would like to use it to work on the ciphertext I gave you today before class on Friday.
I'll also remind you that you should read the first chapter in our textbook, The Codebook by Simon Singh, before class on Monday. We'll talk about other upcoming assignments during class on Friday.